Sentiment classification using NLP With Text Analytics

Loading the data using panda’s read_csv() method is done as follows:

import pandas as pd
import numpy as np
import warnings
warnings.filterwarnings('ignore')
train_data = pd.read_csv('sentiment_train.csv')
train_data.head(5)

We will build different models to classify sentiments.

1. Naive-Bayes classifier

2. TF-IDF VECTORIZER

Now we will discuss one by one and also will see the comparison. Let’s start

First will discuss the Naive Bayes classifier

 Naive-Bayes Model For Sentiment Classification

Naive-Bayes classifier is widely used in Natural language processing and proved to give better results. It works on the concept of Baye’s theorem.

Assume that we would like to predict whether the probability of a document is positive given that the document contains a word awesome. This can be computed if the probability of the word awesome appearing in a document given that it is positive sentiment is multiplied by the probability of the document being positive.

P(doc = +ve | word = awesome) =  P(word = awesome | doc = +ve) * P(doc = +ve)

The posterior probability of the sentiment is computed from the prior probabilities of all the words it contains. The assumption is that the occurrences of the words in a document are considered independent and they do not influence each other. So, if the document contains N-words and words represented as w₁, w₂, w₃………w_n then

P(doc = +ve | word = w₁, w₂, w₃………w_n) =

sklearn.naive_bayes provides a class BernoulliNB which is a Naive -Bayes classifier for multivariate BernoulliNB models. BernoulliNB is designed for binary features, which is the case here.

The steps involved in using the Naive-Bayes model for sentiment classification are as follows:

• Split the dataset into train and validation sets,
• Build the Naive-Bayes model,
• Find model accuracy.

we will discuss these in the following subsections.

Split the dataset into train and validation sets

Split the dataset into a 70:30 ratio for creating training and test datasets using the following code.

from sklearn.model_selection import train_test_split
train_x, test_x, train_y, test_y =  train_test_split(train_ds_features, train_data.Sentiment, 
                                                     test_size = 0.3, random_state = 42)

Build Naive-Bayes Model

Build Naive-Bayes model using the training set.

from sklearn.naive_bayes import BernoulliNB
nb_clf = BernoulliNB()
nb_clf.fit(train_x.toarray(), train_y)

Make a prediction on Test case

The predicted class will be the one that has the higher probability based on Naive-Baye’s Probability calculation. Predict the sentiments of the test dataset using predict() method.

test_ds_predicted = nb_clf.predict(test_x.toarray())

Finding Model Accuracy

Let us print the classification report.

from sklearn import metrics
print(metrics.classification_report(test_y,test_ds_predicted))

 

The Model is classifying with very high accuracy. Both average precision and recall is about 98% for identifying positive and negative sentiment documents. Let us draw the confusion matrix.

cm = metrics.confusion_matrix(test_y, test_ds_predicted)
sn.heatmap(cm, annot=True, fmt = '.2f')

In the confusion matrix, the rows represent the actual number of positive and negative documents in the test set, whereas the columns represent what the model has predicted. Label 1 means positive sentiment and label 0 means negative sentiment.

As per the model prediction, that there are only 13 instances that are wrongly classified as a negative sentiment document and there are only 26 negative sentiment documents classified wrongly as positive sentiment documents. Rest all have been classified correctly.

The next section will discuss TD-IFD Vectorizer Model.

TF-IDF VECTORIZER

TfidfVectorizer is used to create both TF Vectorizer and TF-IDF Vectorizer. It Takes a parameter to use _idf to create TF-IDF vectors. If use _idf set to false, it will create only TF vectors and if it is set to True, it will create TF-IDF vectors.

from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vectorizer =  TfidfVectorizer(analyzer = stemmed_words, max_features = 1000)
feature_vector =  tfidf_vectorizer.fit(train_data.Text)
train_ds_features = tfidf_vectorizer.transform(train_data.Text)
features = feature_vector.get_feature_names()

Tf-IDF is continuous values and these continuous values associated with each class can be assumed to be distributed according to Gaussian distribution. So Gaussian Naive-Bayes can be used to classify these documents. We will GaussianNB, which implements the Gaussian Naive_bayes algorithm for classification.

from sklearn.naive_bayes import GaussianNB
train_x, test_x, train_y, test_y =  train_test_split(train_ds_features, train_data.Sentiment, 
                                                     test_size = 0.3, random_state = 42)

nb_clf = GaussianNB()
nb_clf.fit(train_x.toarray(), train_y)

test_ds_predicted =  nb_clf.predict(test_x.toarray())
print(metrics.classification_report(test_y,test_ds_predicted))

cm = metrics.confusion_matrix(test_y, test_ds_predicted)
sn.heatmap(cm, annot=True, fmt = '.2f')

The precision and recall seem to be pretty much the same. The accuracy is very high in this example as the dataset is clean and carefully curated. But it may not be the case in the real world.

Conclusion

In this article, Text data is unstructured data and needs extensive preprocessing before applying models. Naive-Bayes classification Models are the most widely used algorithm for classifying texts. The next article will discuss some challenges of text analytics using few techniques such as using N-Grams.

About the Author

Hi, I am Kajal Kumari. I have completed my Master’s from IIT(ISM) Dhanbad in Computer Science & Engineering. As of now, I am working as Machine Learning Engineer in Hyderabad. You can also check out few other blogs that I have written here.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.