Recommender Systems (Tweet Ranking) using TensorFlow

Recommender systems are important in many sectors because they assist consumers discover appropriate products or materials based on their preferences and behaviors. These systems analyze user-item interactions and create personalized suggestions using algorithms and approaches. In this post, we will look specifically at tweet ranking and how recommender systems may be used to improve the Twitter user experience.

The purpose is to create a Twitter ranking recommender system using TensorFlow, a well-known deep learning framework. We can develop an effective and efficient tweet grading model using TensorFlow’s rich collection of tools and APIs.

Concepts related to the topic:

• Collaborative filtering is a widely used approach in recommender systems. It leverages the collective behavior of users to make predictions or recommendations. By analyzing the historical interactions between users and items, collaborative filtering can identify patterns and similarities, enabling the system to make relevant recommendations.
• User-item interactions form the foundation of collaborative filtering. These interactions include various types of user actions, such as likes, retweets, and comments on tweets. By capturing and analyzing these interactions, we can gain insights into user preferences and use them to make informed recommendations.
• Matrix factorization is a technique commonly employed in collaborative filtering. It aims to extract latent factors from user-item interaction data. By decomposing the interaction matrix into lower-dimensional matrices, matrix factorization can represent users and items in a latent space, capturing their underlying characteristics.
• Embeddings are essential in capturing the characteristics of users and items. They are low-dimensional vector representations that capture the inherent features or attributes of users and items. Embeddings help to establish relationships between users and items, enabling the system to recommend items based on similar users or items with shared characteristics.

Steps needed:

Step 1: Import the required libraries and modules

Step 2: Load the dataset from a CSV file

Dataset link:https://www.kaggle.com/datasets/thoughtvector/customer-support-on-twitter 

Output:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2811774 entries, 0 to 2811773
Data columns (total 7 columns):
 #   Column                   Dtype  
---  ------                   -----  
 0   tweet_id                 int64  
 1   author_id                object 
 2   inbound                  bool   
 3   created_at               object 
 4   text                     object 
 5   response_tweet_id        object 
 6   in_response_to_tweet_id  float64
dtypes: bool(1), float64(1), int64(1), object(4)
memory usage: 131.4+ MB

Print the first 5 rows

Output:

   tweet_id   author_id  inbound                      created_at  \
0         1  sprintcare    False  Tue Oct 31 22:10:47 +0000 2017   
1         2      115712     True  Tue Oct 31 22:11:45 +0000 2017   
2         3      115712     True  Tue Oct 31 22:08:27 +0000 2017   
3         4  sprintcare    False  Tue Oct 31 21:54:49 +0000 2017   
4         5      115712     True  Tue Oct 31 21:49:35 +0000 2017   

                                                text response_tweet_id  \
0  @115712 I understand. I would like to assist y...                 2   
1      @sprintcare and how do you propose we do that               NaN   
2  @sprintcare I have sent several private messag...                 1   
3  @115712 Please send us a Private Message so th...                 3   
4                                 @sprintcare I did.                 4   

   in_response_to_tweet_id  
0                      3.0  
1                      1.0  
2                      4.0  
3                      5.0  
4                      6.0

Step 3: Extract tweet texts, user IDs, and labels from the DataFrame

Output:

Tweets : @115712 I understand. I would like to assist you. We would need to get you into a private secured link to further assist.
User Id: sprintcare
Labels : False

Step 4: Perform label encoding on user IDs and labels

Output:

sprintcare : 702776
False : 0

Label encoding is used to convert categorical data (user IDs and labels) into numerical representations.

Step 5: Perform min-max scaling on user IDs

Min-max scaling is used to normalize the user IDs to a specific range, typically between 0 and 1.

Step 6: Reduce the dataset size to 500000 samples

This step limits the dataset to the first 500000 samples for faster processing and testing.

Step 7: Split the data into training and testing sets

The dataset is split into training and testing sets using the train_test_split function. The test size is set to 20% of the data, and a random state is specified for reproducibility.

Step 8: Tokenize tweet texts

Output:

Tweets: @189743 Would you mind providing more details about the issue? We'd certainly follow up. ^R https://t.co/iuwZCjz4Or
Encoded Tweeet : [98281, 89, 3, 567, 1119, 74, 91, 65, 2, 86, 136, 715, 161, 58, 882, 11, 9, 10, 2451]

The Tokenizer class from Keras is used to convert the tweet texts into sequences of integers. The tokenizer is fitted on the training tweets and then used to tokenize both the training and testing tweets.

Step 9: Pad sequences to ensure they have the same length

To ensure that all sequences have the same length, padding is applied to the sequences. The maximum sequence length is determined, and both the training and testing sequences are padded to this length.

Step 10: Create the user embedding model

In this step, we create the user embedding model. The user IDs are passed as input to the model using an Input layer. The Embedding layer is then applied to map the user IDs to dense vectors of user_embedding_dim dimensions. The number of unique user IDs is determined by len(user_encoder.classes_). The output of the Embedding layer is flattened using the Flatten layer to obtain a 1D vector. Finally, we define the user embedding model by specifying the input and output layers using the Model class.

Step 11: Create the tweet embedding model using LSTM

In this step, we create the tweet embedding model using the LSTM (Long Short-Term Memory) layer. The input to the model is the sequences of tokenized tweets, represented as integers. An Input layer is defined by a shape of (max_sequence_length,), where max_sequence_length is the maximum length of the padded sequences. The Embedding layer is applied to map the tokenized sequences to dense vectors of tweet_embedding_dim dimensions. The vocabulary size is determined by len(tokenizer.word_index) + 1. The LSTM the layer is then used to process the tweet embeddings and generate a fixed-dimensional output of size 64.

Step 12: Concatenate user and tweet embeddings

In this step, the user embeddings from the user embedding model and the tweet LSTM output are concatenated using the Concatenate layer. This combines the information from both the user and the tweet to capture the relationship between them.

Step 13: Add dense layers for ranking

We add dense layers to the concatenated embeddings for ranking. In this example, two dense layers with 64 and 32 units, respectively, are added. Each dense layer uses the ReLU activation function. The final dense layer has a single unit and uses the sigmoid activation function to output a probability between 0 and 1, indicating the ranking score.

Step 14: Create the model

Output:

Model: "model_5"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_5 (InputLayer)           [(None, 1)]          0           []                               
                                                                                                  
 input_6 (InputLayer)           [(None, 93)]         0           []                               
                                                                                                  
 embedding_4 (Embedding)        (None, 1, 10)        7027770     ['input_5[0][0]']                
                                                                                                  
 embedding_5 (Embedding)        (None, 93, 50)       14688800    ['input_6[0][0]']                
                                                                                                  
 flatten_2 (Flatten)            (None, 10)           0           ['embedding_4[0][0]']            
                                                                                                  
 lstm_2 (LSTM)                  (None, 64)           29440       ['embedding_5[0][0]']            
                                                                                                  
 concatenate_1 (Concatenate)    (None, 74)           0           ['flatten_2[0][0]',              
                                                                  'lstm_2[0][0]']                 
                                                                                                  
 dense_6 (Dense)                (None, 64)           4800        ['concatenate_1[0][0]']          
                                                                                                  
 dense_7 (Dense)                (None, 32)           2080        ['dense_6[0][0]']                
                                                                                                  
 dense_8 (Dense)                (None, 1)            33          ['dense_7[0][0]']                
                                                                                                  
...
Total params: 21,752,923
Trainable params: 21,752,923
Non-trainable params: 0

The model is created by specifying the input and output layers. The inputs are the user embeddings and tweet embeddings, and the output is the ranking score.

Step 15: Compile the model

The model is compiled with the binary cross-entropy loss function, which is commonly used for binary classification tasks. The Adam optimizer is used for optimization, and accuracy is chosen as the evaluation metric.

Step 16: Train the model

In this step, the model is trained using the fit method. The training data consists of the user IDs (train_user_ids) and the padded tweet sequences (train_sequences), with the corresponding labels (train_labels). The number of epochs is set to 10, indicating the number of times the entire training dataset will be passed through the model. The batch size is set to 32, which determines the number of samples processed before the model’s weights are updated.

Step 17: Evaluate the model

Output:

1/63 [..............................] - ETA: 24s - loss: 1.1737 - accuracy: 0.8750
63/63 [==============================] - 1s 6ms/step - loss: 0.1655 - accuracy: 0.9750
Test accuracy: 0.9750000238418579

The trained model is evaluated using the testing data. The testing data consists of the user IDs (test_user_ids) and the padded tweet sequences (test_sequences), along with the corresponding labels (test_labels). The evaluate method returns the loss value and accuracy of the model on the testing data. Finally, the test accuracy is printed to the console.

step 18: Save the trained model

This line saves the trained model to the specified file path (“/content/drive/MyDrive/recommender_model.h5”).

Step 19: Function to recommend tweets based on user preference

This function recommend_tweets takes a user ID and a list of tweets as input and returns a list of recommended tweets based on the user’s preference. It performs the following steps:

• Encodes the user ID using user_encoder.transform.
• Tokenizes and pads the tweet sequences using tokenizer.texts_to_sequences and tf.keras.preprocessing.sequence.pad_sequences.
• Predicts tweet relevance using the trained model by calling model.predict.
• Sorts the tweets based on predicted relevance using np.argsort.
• Returns the sorted tweets.

Step 20: Usage example

Output:

1/1 [==============================] - 0s 16ms/step
['Could be better explained.',
 'Interesting perspective.',
 'Great article! I learned a lot.',
 'Not sure about this one...',
 'I agree with the points made.']

This code sets a user ID and a list of tweets and calls the recommend_tweets function to obtain a list of recommended tweets. Finally, it prints the recommended tweets.

Please make sure you have defined and trained the model before attempting to save it and use it for a recommendation.