Prerequisites: Auto-encoders
This article will demonstrate the process of data compression and the reconstruction of the encoded data by using Machine Learning by first building an Auto-encoder using Keras and then reconstructing the encoded data and visualizing the reconstruction. We would be using the MNIST handwritten digits dataset which is preloaded into the Keras module about which you can read here.
The code is structured as follows: First all the utility functions are defined which are needed at different steps of the building of the Auto-encoder are defined and then each function is called accordingly.
Step 1: Importing the required libraries
import numpy as np import matplotlib.pyplot as plt from random import randint from keras import backend as K from keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D from keras.models import Model from keras.datasets import mnist from keras.callbacks import TensorBoard |
Step 2: Defining a utility function to load the data
def load_data(): # defining the input image size input_image = Input(shape =(28, 28, 1)) # Loading the data and dividing the data into training and testing sets (X_train, _), (X_test, _) = mnist.load_data() # Cleaning and reshaping the data as required by the model X_train = X_train.astype('float32') / 255. X_train = np.reshape(X_train, (len(X_train), 28, 28, 1)) X_test = X_test.astype('float32') / 255. X_test = np.reshape(X_test, (len(X_test), 28, 28, 1)) return X_train, X_test, input_image |
Note: While loading the data, notice that the space where the training labels are loaded are kept empty because the compression process does not involve the output labels
Step 3: Defining a utility function to build the Auto-encoder neural network
def build_network(input_image): # Building the encoder of the Auto-encoder x = Conv2D(16, (3, 3), activation ='relu', padding ='same')(input_image) x = MaxPooling2D((2, 2), padding ='same')(x) x = Conv2D(8, (3, 3), activation ='relu', padding ='same')(x) x = MaxPooling2D((2, 2), padding ='same')(x) x = Conv2D(8, (3, 3), activation ='relu', padding ='same')(x) encoded_layer = MaxPooling2D((2, 2), padding ='same')(x) # Building the decoder of the Auto-encoder x = Conv2D(8, (3, 3), activation ='relu', padding ='same')(encoded_layer) x = UpSampling2D((2, 2))(x) x = Conv2D(8, (3, 3), activation ='relu', padding ='same')(x) x = UpSampling2D((2, 2))(x) x = Conv2D(16, (3, 3), activation ='relu')(x) x = UpSampling2D((2, 2))(x) decoded_layer = Conv2D(1, (3, 3), activation ='sigmoid', padding ='same')(x) return decoded_layer |
Step 4: Defining a utility function to build and train the Auto-encoder network
def build_auto_encoder_model(X_train, X_test, input_image, decoded_layer): # Defining the parameters of the Auto-encoder autoencoder = Model(input_image, decoded_layer) autoencoder.compile(optimizer ='adadelta', loss ='binary_crossentropy') # Training the Auto-encoder autoencoder.fit(X_train, X_train, epochs = 15, batch_size = 256, shuffle = True, validation_data =(X_test, X_test), callbacks =[TensorBoard(log_dir ='/tmp / autoencoder')]) return autoencoder |
Step 5: Defining a utility function to visualize the reconstruction
def visualize(model, X_test): # Reconstructing the encoded images reconstructed_images = model.predict(X_test) plt.figure(figsize =(20, 4)) for i in range(1, 11): # Generating a random to get random results rand_num = randint(0, 10001) # To display the original image ax = plt.subplot(2, 10, i) plt.imshow(X_test[rand_num].reshape(28, 28)) plt.gray() ax.get_xaxis().set_visible(False) ax.get_yaxis().set_visible(False) # To display the reconstructed image ax = plt.subplot(2, 10, i + 10) plt.imshow(reconstructed_images[rand_num].reshape(28, 28)) plt.gray() ax.get_xaxis().set_visible(False) ax.get_yaxis().set_visible(False) # Displaying the plot plt.show() |
Step 6: Calling the utility functions in the appropriate order
a) Loading the data
X_train, X_test, input_image = load_data() |
b) Building the network
decoded_layer = build_network(input_image) |
c) Building and training the Auto-encoder
auto_encoder_model = build_auto_encoder_model(X_train, X_test, input_image, decoded_layer) |
d) Visualizing the reconstruction
visualize(auto_encoder_model, X_test) |
