The goal of this application is to demonstrate how the NLP task of text style transfer can be applied to enhance the human writing experience. In this sense, we intend to peel back the curtains on how an intelligent writing assistant might function — walking through the logical steps needed to automatically re-style a piece of text while building up confidence in the model output.
We emphasize the imperative for a human-in-the-loop user experience when designing natural language generation systems. We believe text style transfer has the potential to empower writers to better express themselves, but not by blindly generating text. Rather, generative models, in conjunction with interpretability methods, should be combined to help writers understand the nuances of linguistic style and suggest stylistic edits that may improve their writing.
Select a style attribute from the sidebar and enter some text below to get started!
1. Input some text
2. Detect style
Before we can transfer style, we need to ensure the input text isn’t already of the target style! To do so, we classify the sample text with a model that has been fine-tuned to differentiate between subjective and neutral tones.
In a product setting, you could imagine this style detection process running continuously inside your favorite word processor as you write, prompting you for action when it detects language that is at odds with your desired tone of voice.
Distribution Between Style Classes
3. Interpret the classification result
Interpreting our model’s output is a crucial practice that helps build trust and justify taking real-world action from the model predictions. In this case, we apply a popular model interpretability technique called Integrated Gradients to the Transformer-based classifier to explain the model’s prediction in terms of its features.
The visual above displays word attributions using the Transformers Interpret library. Positive attribution values (green) indicate tokens that contribute positively towards the predicted class (subjective), while negative values (red) indicate tokens that contribute negatively towards the predicted class.
Visualizing word attributions is a helpful way to build intuition about what makes the input text subjective!
4. Generate a suggestion
Now that we’ve verified the input text is in fact subjective and understand why that’s the case, we can utilize a text style transfer model to generate a suggested replacement that retains the same semantic meaning, but achieves the neutral target style.
Expand the accordion below to toggle generation parameters, then click the button to transfer style!
5. Evaluate the suggestion
Blindly prompting a writer with style suggestions without first checking quality would make for a noisy, error-prone product with a poor user experience. Ultimately, we only want to suggest high-quality edits. But what makes for a suggestion-worthy edit?
A comprehensive quality evaluation for text style transfer output should consider three criteria:
- Style strength – To what degree does the generated text achieve the target style?
- Content preservation – To what degree does the generated text retain the semantic meaning of the source text?
- Fluency – To what degree does the generated text appear as if it were produced naturally by a human?
Below, we apply automated evaluation metrics – Style Transfer Intensity (STI) & Content Preservation Score (CPS) – to measure the first two of these criteria by comparing the input text to the generated suggestion.
To try this interactive application for yourself, check out the original posting here!
