Intent Recognition using TensorFlow

Intent recognition is a method of natural language processing, which deals with determining intent of a given sentence, or in simple terms “what the sentence means”. It is commonly used in chatbots, virtual assistants, and other conversational AI systems to understand user requests and provide appropriate responses.

In this article, We will show a basic of how Intent Recognition can be made using TensorFlow

TensorFlow : TensorFlow is an open-source library for machine learning developed by Google. It provides various tools and functions to build and train neural networks efficiently, and a neural network is a computational model inspired by the functioning of the human brain, composed of interconnected artificial neurons that can learn and make predictions from input data.

To perform Intent Recognition using TensorFlow, we typically follow these steps:

Import Libraries:

We start by importing all necessary modules.

We import ‘tensorflow' to access the core TensorFlow functionality, ‘Tokenizer' to preprocess text data by tokenizing it into sequences, and ‘pad_sequences' to pad the sequences to a fixed length.

Data Collection:

Then, we need a dataset consisting of labeled examples where each example contains a text input and its corresponding intent label. For example, we consider a dataset from kaggle, to view: Chatbots: Intent Recognition Dataset, it is in JSON format and perfect for intent recognition model.

Output:

dict_keys(['intents'])
<class 'list'>
22
dict_keys(['intent', 'text', 'responses', 'extension', 'context', 'entityType', 'entities'])
{'intent': 'SelfAware',
 'text': ['Can you prove you are self-aware',
  'Can you prove you are self aware',
  'Can you prove you have a conscious',
  'Can you prove you are self-aware please',
  'Can you prove you are self aware please',
  'Can you prove you have a conscious please',
  'prove you have a conscious'],
 'responses': ['That is an interesting question, can you prove that you are?',
  'That is an difficult question, can you prove that you are?',
  'That depends, can you prove that you are?'],
 'extension': {'function': '', 'entities': False, 'responses': []},
 'context': {'in': '', 'out': '', 'clear': False},
 'entityType': 'NA',
 'entities': []}

We load data from JSON file like shown above.

Data Cleaning:

We can’t directly process this data. First, we have to clean it, this usually includes, removing punctuation marks or anything which might produce unwanted results. This is also done to reduce number of tokens, as we don’t want to tokenize everything.

We define a function to clean sentence provided to it, by removing punctuations. This can also be done by using Regular Expression module in python.

Data Preprocessing:

The next step is to preprocess the data to make it suitable for training a neural network. This usually involves tokenization, which means breaking down each input sentence into individual words or sub-words.

But before we do that, we need to prepare data in a format {intent : text data}

We have created a dataset which is ready to be preprocessed, we can see sample data below:

Output:

Intent : Greeting
Number of Intent: 143
Sample Input: Hi
Length of text_input: 143
Sample Response:  ['Hi human, please tell me your GeniSys user',
'Hello human, please tell me your GeniSys user',
 'Hola human, please tell me your GeniSys user']

Tokenization and Embedding

Now, our data is ready to be tokenized, with the help of inbuilt TensorFlow tokenizer, We can make both tokenization and embedding.

Output:

Shape of Input Sequence: (143, 9)
array([[ 0,  0,  0,  0,  0,  0,  0,  0, 52],
       [ 0,  0,  0,  0,  0,  0,  0, 52, 53],
       [ 0,  0,  0,  0,  0,  0,  0,  0, 68],
       [ 0,  0,  0,  0,  0,  0,  0,  0, 39],
       [ 0,  0,  0,  0,  0,  0,  0, 39, 53]], dtype=int32)

Feature Extraction:

Neural network cannot process sentences, so numerical representation of sentences have to be provided to it, this is done by doing Feature Extraction, for that we map all words with their indexes and create a matrix mapping it to its category (intent).

Output:

Number of Intents : 22
{0: 'Greeting',
 1: 'GreetingResponse',
 2: 'CourtesyGreeting',
 3: 'CourtesyGreetingResponse',
 4: 'CurrentHumanQuery',
 5: 'NameQuery',
 6: 'RealNameQuery',
 7: 'TimeQuery',
 8: 'Thanks',
 9: 'NotTalking2U',
 10: 'UnderstandQuery',
 11: 'Shutup',
 12: 'Swearing',
 13: 'GoodBye',
 14: 'CourtesyGoodBye',
 15: 'WhoAmI',
 16: 'Clever',
 17: 'Gossip',
 18: 'Jokes',
 19: 'PodBayDoor',
 20: 'PodBayDoorResponse',
 21: 'SelfAware'}

One-Hot Encoding

Apply One-Hot Encoding for categorical_target

Output:

Shape of Ca (143, 22)
array([[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]],
      dtype=int32)

Model Building:

First step of building a model is defining hyperparameters, in simple words they are settings which we predefine before training our model. They include parameters like no. of epochs, embedding dimensions, size of vocabulary, and target length. They can be adjusted accordingly, to increase performance of model.

Output:

Input Dimension :22,
Output Dimension :22

As both input dimension and output dimension are same, we can proceed with building our model.

Now, we can define the architecture of our neural network using TensorFlow. A common model for intent recognition is the recurrent neural network (RNN) or its variant, the long short-term memory (LSTM) network. These networks can handle sequential data, such as sentences, effectively. We can also use pre-trained models like BERT or GPT to achieve better performance.

Here we are using a RNN, by using ‘Sequential’ model from TensorFlow’s Keras API. It consists of an embedding layer, an LSTM layer for sequence processing, and two dense layers for classification. You can see model summary below.

Output:

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 embedding (Embedding)       (None, None, 300)         35700     
                                                                 
 bidirectional (Bidirectiona  (None, 100)              140400    
 l)                                                              
                                                                 
 dense (Dense)               (None, 50)                5050      
                                                                 
 dropout (Dropout)           (None, 50)                0         
                                                                 
 dense_1 (Dense)             (None, 22)                1122      
                                                                 
=================================================================
Total params: 182,272
Trainable params: 182,272
Non-trainable params: 0
_________________________________________________________________

Training:

With the model architecture defined, we can start training the network using our labeled dataset. During training, the model adjusts its internal parameters to minimize the difference between its predicted intents and the true intents in the training data. This process involves forward propagation, where the model makes predictions, and backward propagation, where the model’s parameters are updated based on the prediction errors.

Output:

<keras.callbacks.History at 0x7f75fff7df10>

Here we train the model by fitting it to the padded sequences and its respective categorical sequence. The model adjusts its internal parameters to minimize the difference between its predicted intents and the true intents in the training data. The training is performed for a specified number of epochs, and might stop early if there is no improvement in it’s loss after 4 consecutive epochs.

Evaluate:

To check if our model works correctly, we give it unseen data along with labels and check if it works correctly or not. Thus, we give text inputs along with it’s intent and we test our model, this is known as evaluation of model. It is important step which helps predict accuracy of model, or if it is overfit or underfit.

Output:

1/1 [==============================] - 1s 572ms/step - loss: 0.4058 - accuracy: 1.0000

Predict:

We have finished training our model and can now use it to make predictions on new, unseen sentences. Given an input sentence, the model processes it using the learned parameters and produces a probability distribution over the possible intents. The intent with the highest probability is considered the predicted intent.

Chatbots: Intent Recognition

We have defined a function response to predict intent and give appropriate response. This is a loop to take input from user recognize intent and give appropriate response.

Output:

Note: Enter 'quit' to break the loop.
You:  Hi, Who are you?
Geek: Hello, how are you? I am great thanks! Please tell me your GeniSys user -- TYPE: CourtesyGreeting

You:  Can you prove you have a conscious?
Geek: That is an difficult question, can you prove that you are? -- TYPE: SelfAware

You:  Can you prove you are self-aware?
Geek: They call you <HUMAN>, what can I do for you? -- TYPE: CurrentHumanQuery

You:  My name is Pawan Gunjan.
Geek: <HUMAN>, what can I do for you? -- TYPE: CurrentHumanQuery

You:  Tell me a Jokes
Geek: A woman goes into a US sporting goods store to buy a rifle. 'It's for my husband', she tells the clerk. 'Did he tell you what gauge to get?' asks the clerk. Are you kidding?' she says. 'He doesn't even know that I'm going to shoot him!' -- TYPE: Jokes

You:  Tell me aJokes
Geek: Let me see -- TYPE: WhoAmI

You:  Tell me a Jokes
Geek: A team of scientists were nominated for the Nobel Prize. They had used dental equipment to discover and measure the smallest particles yet known to man. They became known as 'The Graders of the Flossed Quark...' -- TYPE: Jokes

You:  quit