Rasa is a tool to build custom AI chatbots using Python and natural language understanding (NLU). Rasa provides a framework for developing AI chatbots that uses natural language understanding (NLU). It also allows the user to train the model and add custom actions. Chatbots built using Rasa deployed on multiple platforms like FB messenger, Microsoft bot and slack etc.
Architecture:
Rasa Architecture
Rasa has two main components:
- Rasa NLU (Natural Language Understanding): Rasa NLU is an open-source natural language processing tool for intent classification (decides what the user is asking), extraction of the entity from the bot in the form of structured data and helps the chatbot understand what user is saying.
- Rasa Core: a chatbot framework with machine learning-based dialogue management which takes the structured input from the NLU and predicts the next best action using a probabilistic model like LSTM neural network rather than if/else statement. Underneath the hood, it also uses reinforcement learning to improve the prediction of the next best action.
In other words, the Rasa NLU’s job is to interpret the input provided by the user in the form of structured data and Rasa Core’s job is to decide the next set of actions performed by the chatbot. Rasa Core and Rasa NLU are independent of each other and can be used separately.
How to Install Rasa:
- It is highly recommended to use a virtual environment before installing RASA. For creating the virtual environment, the command in anaconda is:
conda create -n rasa activate rasa
- Now, we will install the Rasa in our environment, we will install using pip install. We also need to install TensorFlow in our environment (it will be installed default by Rasa). Installing rasa will take some time.
pip install rasa
- Now, we will initialize the bot using rasa init. This will create the stating project files for the chatbot and train the initial model.
rasa init
o get started quickly, an initial project will be created. If you need some help, check out the documentation at https://rasa.com/docs/rasa. Now let's start! ???????? ? Please enter a path where the project will be created [default: current directory] ./sample_bot ? Path './sample_bot' does not exist ????. Create path? Yes Created project directory at 'C:\Users\Pawan\Desktop\chatbots\sample_bot'. Finished creating project structure. ? Do you want to train an initial model? ???????? Yes Training an initial model... Training Core model... Processed Story Blocks: 100%|??????????????????????????????????????????????????????| 5/5 [00:00<?, ?it/s, # trackers=1] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 1252.55it/s, # trackers=5] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 250.35it/s, # trackers=20] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 178.79it/s, # trackers=24] Processed trackers: 100%|????????????????????????????????????????????????| 5/5 [00:00<00:00, 1252.03it/s, # actions=16] Processed actions: 16it [00:00, 667.56it/s, # examples=16] Processed trackers: 100%|????????????????????????????????????????????| 231/231 [00:00<00:00, 284.86it/s, # actions=126] Epochs: 0%| | 0/100 [00:00<?, ?it/s]c:\users\Pawan\anaconda3\envs\rasa\lib\site-packages\rasa\utils\tensorflow\model_data.py:386: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray final_data[k].append(np.concatenate(np.array(v))) Epochs: 100%|???????????????????????????????????| 100/100 [00:31<00:00, 3.15it/s, t_loss=0.202, loss=0.131, acc=1.000] 2020-07-31 22:55:59 INFO rasa.utils.tensorflow.models - Finished training. 2020-07-31 22:56:02 INFO rasa.core.agent - Persisted model to 'C:\Users\Pawan\AppData\Local\Temp\tmpk26900vo\core' Core model training completed. Training NLU model... 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Training data stats: 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of intent examples: 43 (7 distinct intents) 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Found intents: 'bot_challenge', 'mood_unhappy', 'mood_great', 'greet', 'affirm', 'goodbye', 'deny' 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of response examples: 0 (0 distinct responses) 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of entity examples: 0 (0 distinct entities) 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component WhitespaceTokenizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component RegexFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component LexicalSyntacticFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component CountVectorsFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component CountVectorsFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component DIETClassifier co get started quickly, an initial project will be created. If you need some help, check out the documentation at https://rasa.com/docs/rasa. Now let's start! ???????? ? Please enter a path where the project will be created [default: current directory] ./sample_bot ? Path './sample_bot' does not exist ????. Create path? Yes Created project directory at 'C:\Users\Pawan\Desktop\chatbots\sample_bot'. Finished creating project structure. ? Do you want to train an initial model? ???????? Yes Training an initial model... Training Core model... Processed Story Blocks: 100%|??????????????????????????????????????????????????????| 5/5 [00:00<?, ?it/s, # trackers=1] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 1252.55it/s, # trackers=5] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 250.35it/s, # trackers=20] Processed Story Blocks: 100%|????????????????????????????????????????????| 5/5 [00:00<00:00, 178.79it/s, # trackers=24] Processed trackers: 100%|????????????????????????????????????????????????| 5/5 [00:00<00:00, 1252.03it/s, # actions=16] Processed actions: 16it [00:00, 667.56it/s, # examples=16] Processed trackers: 100%|????????????????????????????????????????????| 231/231 [00:00<00:00, 284.86it/s, # actions=126] Epochs: 0%| | 0/100 [00:00<?, ?it/s]c:\users\Pawan\anaconda3\envs\rasa\lib\site-packages\rasa\utils\tensorflow\model_data.py:386: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray final_data[k].append(np.concatenate(np.array(v))) Epochs: 100%|???????????????????????????????????| 100/100 [00:31<00:00, 3.15it/s, t_loss=0.202, loss=0.131, acc=1.000] 2020-07-31 22:55:59 INFO rasa.utils.tensorflow.models - Finished training. 2020-07-31 22:56:02 INFO rasa.core.agent - Persisted model to 'C:\Users\Pawan\AppData\Local\Temp\tmpk26900vo\core' Core model training completed. Training NLU model... 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Training data stats: 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of intent examples: 43 (7 distinct intents) 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Found intents: 'bot_challenge', 'mood_unhappy', 'mood_great', 'greet', 'affirm', 'goodbye', 'deny' 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of response examples: 0 (0 distinct responses) 2020-07-31 22:56:03 INFO rasa.nlu.training_data.training_data - Number of entity examples: 0 (0 distinct entities) 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component WhitespaceTokenizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component RegexFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component LexicalSyntacticFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component CountVectorsFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component CountVectorsFeaturizer 2020-07-31 22:56:03 INFO rasa.nlu.model - Finished training component. 2020-07-31 22:56:03 INFO rasa.nlu.model - Starting to train component DIETClassifier ....
Getting started:
The command above will create initial project files in directory sample_bot. Let’s look at these files:
- __init__.py: An empty file that helps python to locate your actions.
- actions.py: This file is used for creating custom actions. In case you want to call an external server or fetch external API data, you can define your actions here.
- config.yml: This file defines the configuration of the NLU and core model. If you are using any model outside the NLU model, you have to define the pipeline here.
- credentials.yml: This file is used to store credentials for connecting to external services such as Facebook Messenger, Slack, etc
- data/nlu.md: In this file, we define our intents (what the user could ask the bot to do? ). These intents are then used in training the NLU model.
- data/stories.md: Stories are the sample conversation between a user and bot in the form of intents, responses and actions. Rasa stories are a form of training data used to train the Rasa’s dialogue management models.
- domain.yml: This file lists the different intent (the things which you expect from the user) with bot’s responses and actions which it can perform.
- endpoints.yml: This defines the details for connecting channels like Slack, FB messenger, etc. for storing chats data in the online databases like Redis, etc.
- models/<timestamps>.tar.gz: the initial model, all the trained models stored in the models folder. For retraining the model, we use rasa train command.
For starting the local environment we use rasa x command. It started with something
Starting Rasa X in local mode... ???? wow wow The server is running at http://localhost:5002/login?username=me&password=s23dafd5436d 2020-08-01 00:27:24 WARNING sanic.root - Sanic tried to use loop.add_signal_handler but it is not implemented on this platform. 2020-08-01 00:27:24 WARNING sanic.root - Sanic tried to use loop.add_signal_handler but it is not implemented on this platform.
- Now, navigate to the address provided by rasa server, we will see the browser window in which a chat window opens, that we can use to chat with the assistant.
Creating a weather bot in Rasa:
- In this tutorial, we will be creating a bot that can tell us the weather of any city, we will also use open weather API for this tutorial.
- We will first initialize the project using the following command, :
rasa init --no-prompt
- This will create the initial project structure as we looked above. For this project, we only need to make changes into the following files:
- data/nlu.md
- data/stories.md
- domain.yml
- endpoints.yml
- actions.py
- Rest all files we leave unchanged.
- First, we will define stories (Note: stories the sample conversation which we used to train our model). These stories are defined in the form of markdown language. We delete some predefine stories and replace them with those conversations which are relevant for our bot.
## say goodbye
* goodbye
- utter_goodbye
## bot challenge
* bot_challenge
- utter_iamabot
## weather long
* greet
- utter_greet
* weather
- utter_ask_location
* city_info
- utter_getting_weather
- action_get_weather
* thanks
- utter_welcome
- utter_goodbye
## weather short
* greet
- utter_greet
* weather_for_location
- utter_getting_weather
- action_get_weather
* thanks
- utter_welcome
- utter_goodbye
## New Story
* greet
- utter_greet
* weather_for_location{"location":"London"}
- utter_getting_weather
- action_get_weather
- Rasa nlu.md contains the list of intents and their possible sample text. Also, we will map our entities stext.separately in this sample. These intents are used to train our NLU model.
## intent:greet - hey - hello - hi - good morning - good evening - hey there ## intent:goodbye - bye - goodbye - see you around - see you later ## intent:bot_challenge - are you a bot? - are you a human? - am I talking to a bot? - am I talking to a human? ## intent:weather - what's the weather - what is the temperature today - what is the temperature - i want to know the temperature - i want to know the weather of today - tell me the weather forecast - hows the weather today ## intent:weather_for_location - what is the weather in [Noida](location) - what is the weather in [New Delhi](location) - weather of [Mumbai] (location) - Today's weather in [Bangalore](location) - [Kolkata](location) weather - Tell me weather of [Hyderabad](location) - Can you tell me weather of [London](location) ## intent:city_info - [New Delhi](location) - [India](location) - [Mumbai](location) - [Noida](location) - [Kolkata](location) - [Bangalore](location) - [London](location) ##intent: thanks - thanks - thank you - OK
- domain.yml file list all intents, bot’s responses with their text, and also the action that can be performed by the bot. Note, any action or response we define (in nlu.md/ stories.md) has to be listed in the domain.yml
session_config:
session_expiration_time: 60
carry_over_slots_to_new_session: true
intents:
- greet
- goodbye
- bot_challenge
- weather
- weather_for_location
- city_info
- thanks
entities:
- location
slots:
location:
type: text
responses:
utter_greet:
- text: Hey!
utter_goodbye:
- text: Bye
- text: Good Bye
utter_ask_location:
- text: of where?
- text: In what city?
utter_getting_weather:
- text: Ok, getting weather of [location] ...
utter_welcome:
- text: Welcome
utter_iamabot:
- text: I am a bot, powered by Rasa.
actions:
- action_get_weather
- Now, we define the “action_get_weather” function in actions.py file. This function takes city name from the entity and get the weather of the city with help of OpenWeatherMap API.
from typing import Any, Text, Dict, List
from rasa_sdk import Action, Tracker
from rasa_sdk.events import SlotSet
from rasa_sdk.executor import CollectingDispatcher
import requests
class ActionCheckWeather(Action):
def name(self)-> Text:
return "action_get_weather"
def run(self, dispatcher, tracker, domain):
api_key = 'Your API Key'
loc = tracker.get_slot('location')
current = requests.get('http://api.openweathermap.org/data/2.5/weather?q={}&appid={}'.format(loc, api_key)).json()
print(current)
country = current['sys']['country']
city = current['name']
condition = current['weather'][0]['main' ]
temperature_c = current['main']['temp']
humidity = current['main']['humidity']
wind_mph = current['wind']['speed']
response = """It is currently {} in {} at the moment. The temperature is {} degrees, the humidity is {}% and the wind speed is {} mph.""".format(condition, city, temperature_c, humidity, wind_mph)
dispatcher.utter_message(response)
return [SlotSet('location', loc)]
- Now, we train our model using rasa train command. This will train our NLU model.
- To check that our model is working correctly, we use the following command:
rasa shell nlu
NLU model loaded. Type a message and press enter to parse it.
Next message:
weather in Noida
{
"intent": {
"name": "weather_for_location",
"confidence": 0.9999433755874634
},
"entities": [
{
"entity": "location",
"start": 11,
"end": 16,
"value": "Noida",
"extractor": "DIETClassifier"
}
],
"intent_ranking": [
{
"name": "weather_for_location",
"confidence": 0.9999433755874634
},
{
"name": "city_info",
"confidence": 5.397644417826086e-05
},
{
"name": "greet",
"confidence": 1.0934791134786792e-06
},
{
"name": "goodbye",
"confidence": 1.0303868975825026e-06
},
{
"name": "thanks",
"confidence": 2.715441951295361e-07
},
{
"name": "weather",
"confidence": 2.5835913675109623e-07
},
{
"name": "bot_challenge",
"confidence": 4.761825422860966e-08
}
],
"response_selector": {
"default": {
"response": {
"name": null,
"confidence": 0.0
},
"ranking": [],
"full_retrieval_intent": null
}
},
"text": "weather in Noida"
}
Next message:
delhi weather
{
"intent": {
"name": "weather_for_location",
"confidence": 0.9984486103057861
},
"entities": [
{
"entity": "location",
"start": 0,
"end": 5,
"value": "delhi",
"extractor": "DIETClassifier"
}
],
"intent_ranking": [
{
"name": "weather_for_location",
"confidence": 0.9984486103057861
},
{
"name": "weather",
"confidence": 0.0012639579363167286
},
{
"name": "thanks",
"confidence": 9.82139608822763e-05
},
{
"name": "goodbye",
"confidence": 8.915668877307326e-05
},
{
"name": "greet",
"confidence": 6.841398135293275e-05
},
{
"name": "city_info",
"confidence": 2.7903372028958984e-05
},
{
"name": "bot_challenge",
"confidence": 3.7094062008691253e-06
}
],
"response_selector": {
"default": {
"response": {
"name": null,
"confidence": 0.0
},
"ranking": [],
"full_retrieval_intent": null
}
},
"text": "delhi weather"
}
Next message:
whats the weather
{
"intent": {
"name": "weather",
"confidence": 0.9636867642402649
},
"entities": [],
"intent_ranking": [
{
"name": "weather",
"confidence": 0.9636867642402649
},
{
"name": "weather_for_location",
"confidence": 0.03303918614983559
},
{
"name": "thanks",
"confidence": 0.0017935443902388215
},
{
"name": "goodbye",
"confidence": 0.0011113830842077732
},
{
"name": "bot_challenge",
"confidence": 0.00020741281332448125
},
{
"name": "greet",
"confidence": 0.0001540367811685428
},
{
"name": "city_info",
"confidence": 7.643929166079033e-06
}
],
"response_selector": {
"default": {
"response": {
"name": null,
"confidence": 0.0
},
"ranking": [],
"full_retrieval_intent": null
}
},
"text": "whats the weather"
}
- As we can notice, the above-trained NLU model correctly classifies the intent with high accuracy. Now, go to the endpoints.yml file and add or uncomment the following lines:
action_endpoint: url: "http://localhost:5055/webhook"
- Now open two command prompt windows, In one window run actions using the following command:
rasa run actions
And in another window, run the rasa shell using following command:
rasa shell
- We can also visualize the sequence graph of bot’s conversation:
rasa visualize
References:
