Logistic Regression on MNIST with PyTorch

Logistic Regression

Logistic Regression is also known as Binary Classification is one of the most popular Machine Learning Algorithms. It comes under Supervised Learning Classification Algorithms. It is used to predict the probability of the target label. By binary classification, it means that the model predicts the label either 0 or 1. The target variable is categorical data types for example: Yes or No, Survived or Not Survived, Male or Female, Pass or Fail, and so on. Logistic Regression makes use of the Sigmoid Function to make the prediction. Sigmoid Activation Function is a nonlinear function which is defined as:  

y = 1/(1+e-z) 

#the y is in range 0-1
#z = x*w + b where w is weight and b is bias

Logistics Regression of MNIST In Pytorch

Pytorch is the powerful Machine Learning Python Framework. With the Pytorch framework, it becomes easier to implement Logistic Regression and it also provides the MNIST dataset. 

Installation:

pip install torch
pip install torchvision --no-deps

Steps to build a complete MNIST predict model using Logistic Regression

Import Necessary Modules

import torch
import torchvision
import torch.nn as tn
import matplotlib.pyplot as plt
import torchvision.transforms as tt
import torch.utils as utils

Download the Datasets from torchvision

Torchvision module provides the MNIST dataset which can be downloaded by entering this code:

Output:

Dataset MNIST
    Number of datapoints: 60000
    Root location: ./data
    Split: Train
Dataset MNIST
    Number of datapoints: 10000
    Root location: data
    Split: Test

Ok, the training data and test data that we have now are in form of Images:

Output:

(<PIL.Image.Image image mode=L size=28x28 at 0x7FEFA4362E80>, 5)

If you notice the output carefully we get an insight into the train data. The image is made of 28*28 pixels and the first index image is 5. Since it is in form of a tuple where index one is the image and index two is the Code block label. Using Matplotlib we can visualize what image does first train data index contains. 

Visualize

Output:

Data Transformation:

Since the data is in image form, it has to be transformed into Tensor, so that PyTorch neural network can train the data. Torchvision provides a transform method. 

train_data = torchvision.datasets.MNIST('data',train=True,transform=tt.ToTensor())
test_data = torchvision.datasets.MNIST('data',train=False,transform=tt.ToTensor()) 

Arguments Required

input_size = 28*28  #Size of image

num_classes = 10  #the image number are in range 0-10

num_epochs = 5 #one cycle through the full train data

batch_size = 100 #sample size consider before updating the model’s weights

learning_rate = 0.001  #step size to update parameter

Make It Iterable

Using DataLoader we can loop through the tensor of the train data. 

train_dataLoader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,shuffle=True)
test_dataLoader = torch.utils.data.DataLoader(test_data,batch_size=batch_size,shuffle=False)

Build Logistic Regression Model

Create a model of Logistic Regression that fits in features and output data. 

class LogisticRegression(tn.Module):
    def __init__(self,input_size,num_classes):
        super(LogisticRegression,self).__init__()
        self.linear = tn.Linear(input_size,num_classes)
    
    def forward(self,feature):
        output = self.linear(feature)
        return output

Loss And Optimizer

We shall use the CrossEntropyLoss function to calculate the computed loss. The reason to use the CrossEntropyLoss function is that it computes the softmax function and cross-entropy. 

 And Stochastic Gradient Descent is the optimizer used to calculate the gradient and update the parameters. 

model = LogisticRegression(input_size,num_classes)
loss = tn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(),lr=learning_rate)

Train And Predict

Convert images and labels to tensors with gradients and then clear gradients w.r.t parameters(optimizer.zero_grad()). Compute the loss from the gradient and further try to reduce the loss by updating the parameters by adding a learning rate. This process is called Backward Propagation. 

run = 0
for epoch in range(num_epochs):
    for i,(images,labels) in enumerate(train_dataLoader):
        images = torch.autograd.Variable(images.view(-1,input_size))
        labels = torch.autograd.Variable(labels)
        
        # Nullify gradients w.r.t. parameters
        optimizer.zero_grad()
        #forward propagation
        output = model(images)
        # compute loss based on obtained value and actual label
        compute_loss = loss(output,labels)
        # backward propagation
        compute_loss.backward()
        # update the parameters
        optimizer.step()
        run+=1
        
        if (i+1)%200 == 0:
            # check total accuracy of predicted value and actual label
            accurate = 0
            total = 0
            for images,labels in test_dataLoader:
                images = torch.autograd.Variable(images.view(-1,input_size))
                output = model(images)
                _,predicted = torch.max(output.data, 1)
                # total labels
                total+= labels.size(0)
                
                # Total correct predictions
                accurate+= (predicted == labels).sum()
                accuracy_score = 100 * accurate/total
            
            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(run, compute_loss.item(), accuracy_score))

print('Final Accuracy:',accuracy_score)

Final Result:

The accuracy score is 89, which is not bad.