Image classification is one of the supervised machine learning problems which aims to categorize the images of a dataset into their respective categories or labels. Classification of images of various dog breeds is a classic image classification problem. So, we have to classify more than one class that’s why the name multi-class classification, and in this article, we will be doing the same by making use of a pre-trained model InceptionResNetV2, and customizing it.
Let’s first discuss some of the terminologies.
Transfer learning: Transfer learning is a popular deep learning method that follows the approach of using the knowledge that was learned in some task and applying it to solve the problem of the related target task. So, instead of creating a neural network from scratch we “transfer” the learned features which are basically the “weights” of the network. To implement the concept of transfer learning, we make use of “pre-trained models“.
Necessities for transfer learning: Low-level features from model A (task A) should be helpful for learning model B (task B).
Pre-trained model: Pre-trained models are the deep learning models which are trained on very large datasets, developed, and are made available by other developers who want to contribute to this machine learning community to solve similar types of problems. It contains the biases and weights of the neural network representing the features of the dataset it was trained on. The features learned are always transferrable. For example, a model trained on a large dataset of flower images will contain learned features such as corners, edges, shape, color, etc.
InceptionResNetV2: InceptionResNetV2 is a convolutional neural network that is 164 layers deep, trained on millions of images from the ImageNet database, and can classify images into more than 1000 categories such as flowers, animals, etc. The input size of the images is 299-by-299.
Dataset description:
- The dataset used comprises of 120 breeds of dogs in total.
- Each image has a file name which is its unique id.
- Train dataset ( train.zip ): contains 10,222 images which are to be used for training our model
- Test dataset (test.zip ): contains 10,357 images which we have to classify into the respective categories or labels.
- labels.csv: contains breed names corresponding to the image id.
- sample_submission.csv: contains correct form of sample submission to be made
All the above mentioned files can be downloaded from here.
NOTE: For better performance use GPU.
We first import all the necessary libraries.
Python3
import numpy as npimport pandas as pdimport seaborn as snsimport matplotlib.pyplot as pltfrom sklearn.metrics import classification_report, confusion_matrix# deep learning librariesimport tensorflow as tfimport kerasfrom keras.preprocessing.image import ImageDataGeneratorfrom tensorflow.keras import applicationsfrom keras.models import Sequential, load_modelfrom keras.layers import Conv2D, MaxPooling2D, GlobalAveragePooling2D, Flatten, Dense, Dropoutfrom keras.preprocessing import imageimport cv2import warningswarnings.filterwarnings('ignore') |
Loading datasets and image folders
Python3
from google.colab import drivedrive.mount("/content/drive")# datasetslabels = pd.read_csv("/content/drive/My Drive/dog/labels.csv")sample = pd.read_csv('/content/drive/My Drive/dog/sample_submission.csv')# folders pathstrain_path = "/content/drive/MyDrive/dog/train"test_path = "/content/drive/MyDrive/dog/test" |
Displaying the first five records of labels dataset to see its attributes.
Python3
labels.head() |
Output:
Adding ‘.jpg’ extension to each id
This is done in order to fetch the images from the folder since the image name and id’s are the same so adding .jpg extension will help us in retrieving images easily.
Python3
def to_jpg(id): return id+".jpg"labels['id'] = labels['id'].apply(to_jpg)sample['id'] = sample['id'].apply(to_jpg) |
Augmenting data:
It’s a pre-processing technique in which we augment the existing dataset with transformed versions of the existing images. We can perform scaling, rotations, increasing brightness, and other affine transformations. This is a useful technique as it helps the model to generalize the unseen data well.
ImageDataGenerator class is used for this purpose which provides a real-time augmentation of data.a
Description of few of its parameters that are used below:
- rescale: rescales values by the given factor
- horizontal flip: randomly flip inputs horizontally.
- validation_split: this is the fraction of images reserved for validation (between 0 and 1).
Python3
# Data agumentation and pre-processing using tensorflowgen = ImageDataGenerator( rescale=1./255., horizontal_flip = True, validation_split=0.2 # training: 80% data, validation: 20% data )train_generator = gen.flow_from_dataframe( labels, # dataframe directory = train_path, # images data path / folder in which images are there x_col = 'id', y_col = 'breed', subset="training", color_mode="rgb", target_size = (331,331), # image height , image width class_mode="categorical", batch_size=32, shuffle=True, seed=42,)validation_generator = gen.flow_from_dataframe( labels, # dataframe directory = train_path, # images data path / folder in which images are there x_col = 'id', y_col = 'breed', subset="validation", color_mode="rgb", target_size = (331,331), # image height , image width class_mode="categorical", batch_size=32, shuffle=True, seed=42,) |
Output:
Let’s see what a single batch of data looks like.
Python3
x,y = next(train_generator)x.shape # input shape of one record is (331,331,3) , 32: is the batch size |
Output:
(32, 331, 331, 3)
Plotting images from the train dataset
Python3
a = train_generator.class_indicesclass_names = list(a.keys()) # storing class/breed names in a listdef plot_images(img, labels): plt.figure(figsize=[15, 10]) for i in range(25): plt.subplot(5, 5, i+1) plt.imshow(img[i]) plt.title(class_names[np.argmax(labels[i])]) plt.axis('off') plot_images(x,y) |
Output:
Building our Model
This is the main step where the neural convolution model is built.
Python3
# load the InceptionResNetV2 architecture with imagenet weights as basebase_model = tf.keras.applications.InceptionResNetV2( include_top=False, weights='imagenet', input_shape=(331,331,3) )base_model.trainable=False# For freezing the layer we make use of layer.trainable = False# means that its internal state will not change during training.# model's trainable weights will not be updated during fit(),# and also its state updates will not run.model = tf.keras.Sequential([ base_model, tf.keras.layers.BatchNormalization(renorm=True), tf.keras.layers.GlobalAveragePooling2D(), tf.keras.layers.Dense(512, activation='relu'), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dropout(0.5), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(120, activation='softmax') ]) |
BatchNormalization :
- It is a normalization technique which is done along mini-batches instead of the full data set.
- It is used to speed up training and use higher learning rates.
- It maintains the mean output close to 0 and the output standard deviation close to 1.
GlobalAveragePooling2D :
- It takes a tensor of size (input width) x (input height) x (input channels) and computes the average value of all values across the entire (input width) x (input height) matrix for each of the (input channels).
- The dimensionality of the images is reduced by reducing the number of pixels in the output from the previous neural network layer.
- By using this we get a 1-dimensional tensor of size (input channels) as our output.
- 2D Global average pooling operation. Here ‘Depth’ = ‘Filters’
Dense Layers: These layers are regular fully connected neural network layers connected after the convolutional layers.
Drop out layer: is also used whose function is to randomly drop some neurons from the input unit so as to prevent overfitting. The value of 0.5 indicates that 0.5 fractions of neurons have to be dropped.
Compile the model:
Before training our model we first need to configure it and that is done by model.compile() which defines the loss function, optimizers, and metrics for prediction.
Python3
model.compile(optimizer='Adam',loss='categorical_crossentropy',metrics=['accuracy'])# categorical cross entropy is taken since its used as a loss function for# multi-class classification problems where there are two or more output labels.# using Adam optimizer for better performance# other optimizers such as sgd can also be used depending upon the model |
Displaying a summary report of the model
By displaying the summary we can check our model to confirm that everything is as expected.
Python3
model.summary() |
Output:
Defining callbacks to preserve the best results:
Callback: It is an object that can perform actions at various stages of training (for example, at the start or end of an epoch, before or after a single batch, etc).
Python3
early = tf.keras.callbacks.EarlyStopping( patience=10, min_delta=0.001, restore_best_weights=True)# early stopping call back |
Training the model: It means that we are finding a set of values for weights and biases that have a low loss on average across all the records.
Python3
batch_size=32STEP_SIZE_TRAIN = train_generator.n//train_generator.batch_sizeSTEP_SIZE_VALID = validation_generator.n//validation_generator.batch_size# fit modelhistory = model.fit(train_generator, steps_per_epoch=STEP_SIZE_TRAIN, validation_data=validation_generator, validation_steps=STEP_SIZE_VALID, epochs=25, callbacks=[early] |
Output:
Save the model
We can save the model for further use.
Python3
model.save("Model.h5") |
Visualizing the model’s performance
Python3
# store resultsacc = history.history['accuracy']val_acc = history.history['val_accuracy']loss = history.history['loss']val_loss = history.history['val_loss']# plot results# accuracyplt.figure(figsize=(10, 16))plt.rcParams['figure.figsize'] = [16, 9]plt.rcParams['font.size'] = 14plt.rcParams['axes.grid'] = Trueplt.rcParams['figure.facecolor'] = 'white'plt.subplot(2, 1, 1)plt.plot(acc, label='Training Accuracy')plt.plot(val_acc, label='Validation Accuracy')plt.legend(loc='lower right')plt.ylabel('Accuracy')plt.title(f'\nTraining and Validation Accuracy. \nTrain Accuracy: {str(acc[-1])}\nValidation Accuracy: {str(val_acc[-1])}') |
Output: Text(0.5, 1.0, ‘\nTraining and Validation Accuracy. \nTrain Accuracy: 0.9448809027671814\nValidation Accuracy: 0.9022817611694336’)
Python3
# lossplt.subplot(2, 1, 2)plt.plot(loss, label='Training Loss')plt.plot(val_loss, label='Validation Loss')plt.legend(loc='upper right')plt.ylabel('Cross Entropy')plt.title(f'Training and Validation Loss. \nTrain Loss: {str(loss[-1])}\nValidation Loss: {str(val_loss[-1])}')plt.xlabel('epoch')plt.tight_layout(pad=3.0)plt.show() |
Output:
A line graph of training vs validation accuracy and loss was also plotted. The graph indicates that the accuracies of validation and training were almost consistent with each other and above 90%. The loss of the CNN model is a negative lagging graph which indicates that the model is behaving as expected with a reducing loss after each epoch.
Evaluating the accuracy of the model
Python3
accuracy_score = model.evaluate(validation_generator)print(accuracy_score)print("Accuracy: {:.4f}%".format(accuracy_score[1] * 100))print("Loss: ",accuracy_score[0]) |
Output:
Viewing the Test Image
Python3
test_img_path = test_path+"/000621fb3cbb32d8935728e48679680e.jpg"img = cv2.imread(test_img_path)resized_img = cv2.resize(img, (331, 331)).reshape(-1, 331, 331, 3)/255plt.figure(figsize=(6,6))plt.title("TEST IMAGE")plt.imshow(resized_img[0]) |
Output:
Making predictions on the test data
Python3
predictions = []for image in sample.id: img = tf.keras.preprocessing.image.load_img(test_path +'/'+ image) img = tf.keras.preprocessing.image.img_to_array(img) img = tf.keras.preprocessing.image.smart_resize(img, (331, 331)) img = tf.reshape(img, (-1, 331, 331, 3)) prediction = model.predict(img/255) predictions.append(np.argmax(prediction))my_submission = pd.DataFrame({'image_id': sample.id, 'label': predictions})my_submission.to_csv('submission.csv', index=False)# Submission file outputprint("Submission File: \n---------------\n")print(my_submission.head()) # Displaying first five predicted output |
Output:
