LDA and PCA both are dimensionality reduction techniques in which we try to reduce the dimensionality of the dataset without losing much information and preserving the pattern present in the dataset. In this article, we will use the iris dataset along with scikit learn pre-implemented functions to perform LDA and PCA with a single line of code. Converting it into 2D and then visualizing them in two dimensions helps us to identify the patterns present between the different classes of the dataset.
Implementing PCA using Scikit Learn
Python3
from sklearn import datasets import pandas as pd import numpy as np import seaborn as sb import matplotlib.pyplot as plt iris = datasets.load_iris() iris.keys() |
Output:
dict_keys(['data', 'target', 'frame', 'target_names', 'DESCR', 'feature_names', 'filename', 'data_module'])
There are some keys in the dataset that we can use to access particular data. For instance, you can specify iris[‘data’] to get information about the length and width of iris flowers.
Pandas is a fantastic tool for preprocessing and exploring datasets, among other dataset-related tasks. So let’s transform our dataset, which is currently in the form of matrices, into rows and columns.
Python3
iris = pd.DataFrame( data=np.c_[iris['data'], iris['target']], columns=iris['feature_names'] + ['target'] ) iris.head() |
Output:
Iris dataset first five rows
Now, let’s separate the features and the target variable.
Python3
# As we only require the measurements, # we will drop the target and species. X = iris.drop(['target'], axis=1) Y = iris['target'] |
Now from the sklearn.decomposition module we will import PCA and then use it to convert our dataset to 2D from 4D.
Python3
from sklearn.decomposition import PCA pca = PCA(n_components=2) iris_pca = pca.fit_transform(X) |
Now the iris_pca contains the data in the desired format. Let’s plot this on a 2D plane to visualize the pattern between the classes.
Python3
sb.scatterplot(iris_pca[:, 0], iris_pca[:, 1], hue=iris['target']) plt.show() |
Output:
Visualising data obtained by using PCA
Let’s the percentage of the variance or the information of the original dataset retained after reducing the dimensionality of the dataset.
Python3
ret_variance = pca.explained_variance_ratio_[0] ret_variance |
Output:
0.9246187232017271
Implementing LDA using Scikit Learn
We import an LDA model from the Scikit Learn Library in this step.
Python3
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA lda = LDA(n_components=2) iris_lda = lda.fit_transform(X, iris['target']) |
Now let’s plot the lower dimensional data on a 2D plane and try to visualize the distinction between the three classes.
Python3
sb.scatterplot(iris_lda[:, 0], iris_lda[:, 1], hue=iris['target']) plt.show() |
Output:
Visualising data obtained by using LDA
LDA maximizes the distance between different classes, whereas PCA maximizes the variance of the data. When there are fewer samples in each class, PCA performs better. LDA, however, performs better on large datasets with many classes.
