Sentiment Analysis is a natural language processing technique used to determine emotions from textual data. It’s often performed to know customer requirements, study product sentiment in user feedback, decision-making, and more.
Types of Sentiment Analysis:
- Fine-grained Sentiment Analysis: It is done when we want polarity precision in our results. In this type of analysis, we divide our results into polarity categories such as very negative, negative, positive, very positive, neutral.
- Emotion-Based Sentiment Analysis: This type of analysis detects different types of emotions like happiness, anger, and sadness.
- Aspect Based Sentiment Analysis: If we want to know particular aspects or features users are mentioning in a positive or negative way, then this type of analysis is used. For example, the User review says that the Laptop is very good, but the battery is very poor so here our aspect-based algorithm would be able to determine that there is a negative opinion about the battery, not the whole product.
Approach: In this article, we will be creating a sentiment analysis application using node js that analyses text data from user feedback and we will use a mixture of fine-grained and emotion-based sentiment analysis approach to derive user sentiment. We will be using AFINN (lexicon of English words rated for valence with an integer between minus five (negative) and plus five (positive)) which is included in the natural library which we will include as a npm package in our application.
Below is the step by step implementation:
Node JS Installation: Follow the link to download and install Node.js: Download Node.js. We will ensure that we have node installed in our system by executing the following command :
node -v
Step 1: Create a separate folder and with the help of a terminal or command prompt navigate to this folder and move into that folder.
cd <your folder location>
Step 2: Create package.json by typing the following command in the terminal:
npm init -y
For knowing more about package.json click here.
Step 3: Install the following dependencies.
- Express: Express is a minimal and flexible Node.js web application framework that provides a robust set of features for web and mobile application
- Natural: a Node.js package that supports most of the NLP algorithms
- Stopword: stopword, Node.js package that allows you to strip stopwords from an input text
npm install --save express natural stopword
Project Structure: The project structure will look like this:
Step 4: First, we will create a basic HTTP server.
server.js
// Import packages below const express = require("express"); const natural = require("natural"); const stopword = require("stopword"); // Port and Host const port = 5500; const host = "127.0.0.1"; // Initialise our app let app = express(); // Include Routes // Listen app.listen(port,host,()=>{ console.log("Server is running..."); }); |
Step 5: Create a new route and give it the path /feedback, when users send a POST request to our route with the feedback in their request body, they should receive a response containing its sentiment analysis.
server.js
// Include Routes app.post("/feedback",(request,response)=>{ const { feedback } = request.body; }); |
Step 6: The data we get from our users is filled with a lot of errors and noise that we have to filter out during our analysis. In order to extract meaningful sentiment or information, we have to filter our user’s textual data, this process is called data filtration.
Steps for filtration:
- To maintain the structure of our data, we will convert all words to their standard form. For example, You’re -> You are.
- All our textual data will be lowercased since we want our analysis algorithm to treat brilliant and BriLLiaNt as the same word.
- Removal of special characters and numerical tokens since they are just noisy elements, also their contribution in our results will be null.
- Tokenization is one of the crucial steps in this process of filtration, as this is the process of splitting a text into meaningful units. We will be using the WordTokenizer from our imported Natural package.
- Removal of stopwords as these words contributes nothing to the user’s sentiment, some examples of stop words include: but, a, or, etc.
- Process of word normalization in NLP in which stemming algorithm is expected to reduce the words to their root word. For example “training”, “trained” to “train”. Sentiment Analyzer from the Natural library provides us with the option of supplying a stemmer as a parameter while calling it. During its analysis, the individual words will be converted to their root form.
Sentiment Analysis Using Natural: The sentiment analysis algorithm from the Natural library uses AFINN which is a lexicon of English words rated for valence with an integer between minus five (negative) and plus five (positive). Calculating the sum of the polarity of each word in a piece of text and normalizing it with the length of a sentence, is how the algorithm works. If the algorithm returns a negative value, that means the sentiment is negative, if it returns a positive value, that means the sentiment is positive. Zero value indicates neutral sentiment.
We will create a route /feedback in which we will collect our user’s feedback and implement our analysis. Function named convertToStandard will convert all our data to its standard form. convertTolowerCase function converts all our data to a lowercase form. removeNonAlpha function will remove non-alphabets. Next we will tokenize our data and remove stopwords using the stopword npm package. After all this data filtration code, we will use a natural package, SentimentAnalyzer from Natural will create a sentiment score from our user’s review. At last, we will send this sentiment score based on our analysis as a response to our users.
server.js
// Include npm packages const express = require("express"); const natural = require("natural"); const stopword = require("stopword"); // For conversion of contractions to standard lexicon const wordDict = { "aren't": "are not", "can't": "cannot", "couldn't": "could not", "didn't": "did not", "doesn't": "does not", "don't": "do not", "hadn't": "had not", "hasn't": "has not", "haven't": "have not", "he'd": "he would", "he'll": "he will", "he's": "he is", "i'd": "I would", "i'd": "I had", "i'll": "I will", "i'm": "I am", "isn't": "is not", "it's": "it is", "it'll": "it will", "i've": "I have", "let's": "let us", "mightn't": "might not", "mustn't": "must not", "shan't": "shall not", "she'd": "she would", "she'll": "she will", "she's": "she is", "shouldn't": "should not", "that's": "that is", "there's": "there is", "they'd": "they would", "they'll": "they will", "they're": "they are", "they've": "they have", "we'd": "we would", "we're": "we are", "weren't": "were not", "we've": "we have", "what'll": "what will", "what're": "what are", "what's": "what is", "what've": "what have", "where's": "where is", "who'd": "who would", "who'll": "who will", "who're": "who are", "who's": "who is", "who've": "who have", "won't": "will not", "wouldn't": "would not", "you'd": "you would", "you'll": "you will", "you're": "you are", "you've": "you have", "'re": " are", "wasn't": "was not", "we'll": " will", "didn't": "did not"} const port = 5500; const host = "127.0.0.1"; let app = express(); app.use(express.json()); app.use(express.urlencoded({ extended: false })); app.use("/",express.static(__dirname + "/public")); // Contractions to standard lexicons Conversion const convertToStandard = text => { const data = text.split(' '); data.forEach((word, index) => { Object.keys(wordDict).forEach(key => { if (key === word.toLowerCase()) { data[index] = wordDict[key] }; }); }); return data.join(' '); } // LowerCase Conversion const convertTolowerCase = text => { return text.toLowerCase(); } // Pure Alphabets extraction const removeNonAlpha = text => { // This specific Regex means that replace all //non alphabets with empty string. return text.replace(/[^a-zA-Z\s]+/g, ''); } // Analysis Route app.post("/feedback", (request, response) => { console.log(request.body); // NLP Logic // Convert all data to its standard form const lexData = convertToStandard(request.body.feedback); console.log("Lexed Data: ",lexData); // Convert all data to lowercase const lowerCaseData = convertTolowerCase(lexData); console.log("LowerCase Format: ",lowerCaseData); // Remove non alphabets and special characters const onlyAlpha = removeNonAlpha(lowerCaseData); console.log("OnlyAlpha: ",onlyAlpha); // Tokenization const tokenConstructor = new natural.WordTokenizer(); const tokenizedData = tokenConstructor.tokenize(onlyAlpha); console.log("Tokenized Data: ",tokenizedData); // Remove Stopwords const filteredData = stopword.removeStopwords(tokenizedData); console.log("After removing stopwords: ",filteredData); // Stemming const Sentianalyzer = new natural.SentimentAnalyzer('English', natural.PorterStemmer, 'afinn'); const analysis_score = Sentianalyzer.getSentiment(filteredData); console.log("Sentiment Score: ",analysis_score); response.status(200).json({ message: "Data received", sentiment_score: analysis_score }) }); app.listen(port, host, () => { console.log('Server is running...'); }); |
Step 7: In order to connect frontend with our server, we have to add a single line of code which will be added in our server.js file, which means that all our static HTML, CSS, and JS files will be served at / route.
server.js
app.use("/",express.static(__dirname + "/public"));
Now, we will create an HTML file named index.html which will be our frontend.
index.html
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <!-- Font CDN --> <link rel="preconnect" <link rel="preconnect" <link href= "https://fonts.googleapis.com/css2?family=Open+Sans&family=Outfit&family=M+PLUS+1+Code&display=swap" rel="stylesheet"> <title>Event Feedback Application</title> </head> <style> *{ margin: 0; padding: 0; } .feedback{ transition: all .3s; background-size: cover; min-height: 100vh; } .feedback.active{ filter: blur(8px); } .container{ display: flex; justify-content: center; flex-direction: column; align-items: center; } .container .form{ margin-top: 10%; background: #f1f5f3; padding: 1em 2.5em 1.7em 2.5em; display: flex; justify-content: center; align-items: center; flex-direction: column; } .form > div{ font-family: 'Open Sans',sans-serif; margin-top: 8%; } textarea{ font-family: 'Open Sans',sans-serif; padding: .8em; border: none; outline: none; resize: none; } button{ padding: .7em; cursor: pointer; outline: none; font-family: 'Open Sans',sans-serif; background: #000; border: 1px solid #222; color: #f1f5f3; } .results{ position: absolute; top: 40%; left: 50%; transform: translate(-50%,-50%); z-index: 999999; transition: all .5s; opacity: 0; pointer-events: none; } .results.active{ opacity: 1; pointer-events: visible; } .results .result-box{ background: #f1f5f3; padding: 2em 4em; } .result-box > div{ display: flex; justify-content: center; margin-top: 15%; } .sentiment p{ font-family:'M PLUS 1 Code',sans-serif; } .close-button button{ width: 100%; margin-top: 20%; background: rgb(255, 63, 102); } </style> <body> <!-- Toast Box --> <div class="results"> <div class="result-box"> <div class="emoji"> </div> <div class="sentiment"> </div> <div class="close-button"> <button class="close">❌</button> </div> </div> </div> <section class="feedback"> <div class="container"> <div class="form"> <div class="title"> <h2>Event Feedback ????</h2> </div> <div> <textarea cols="20" rows="5" id="feedbacktext" placeholder="Write..."> </textarea> </div> <div> <button id="submit">Submit</button> </div> </div> </div> </section> </body> <script src="main.js"></script> </html> |
Step 8: Now in /public folder, let’s create a file named main.js, in which we will add some functions for making our API call. Grab all the HTML elements. So our logic is that when our user fills the feedback form, after clicking submit, a function should be called and an API call should be made to our node js server. We will add a click event on our button, on clicking, we will make an API call to our server and we will get response according to our feedback. According to our sentiment score, we will display our results with emojis.
main.js
// Grab all HTML Elements // All containers const feedback = document.getElementById("feedbacktext"); const wholeContainer = document.querySelector(".feedback"); const resultContainer = document.querySelector(".results"); // All controls const submit_button = document.getElementById("submit"); const closeButton = document.querySelector(".close"); // Results const emoji = document.querySelector(".emoji"); const sentiment = document.querySelector(".sentiment"); // Add event listener to submit button, send feedback and // name to our node js server application submit_button.addEventListener("click",()=>{ console.log("Feedback: ",feedback.value); // Send POST request to our server const options = { method : "POST", body : JSON.stringify({ feedback : feedback.value }), headers : new Headers({ 'Content-Type' : "application/json" }) } // Use fetch to request server fetch("/feedback",options) .then(res=>res.json()) .then((response)=>{ console.log(response.sentiment_score); const score = response.sentiment_score; // Separate responses according to sentiment_score if(score > 0){ emoji.innerHTML = "<p>😄</p>"; sentiment.innerHTML = "<p>➕ Positive</p>"; }else if(score === 0){ emoji.innerHTML = "<p>😐</p>"; sentiment.innerHTML = "<p>Neutral</p>"; }else{ emoji.innerHTML = "<p>😡</p>"; sentiment.innerHTML = "<p>➖ Negative</p>"; } // Result Box should appear resultContainer.classList.add("active"); wholeContainer.classList.add("active"); }) .catch(err=>console.error("Error: ",err)); // Clear all inputs after operation feedback.value = ""; }); // Close Button closeButton.addEventListener("click",()=>{ wholeContainer.classList.remove("active"); resultContainer.classList.remove("active"); }) |
Step to run the application: To start this application, execute the following command on the command prompt:
node server.js
Output: Navigate to http://localhost:5500/
Server Logs:
