Editor’s Note: Sameer is a speaker for ODSC East 2022. Be sure to check out his talk, “Natural Language Processing in Accelerating Business Growth,” to learn more about NLP applications in business!
Natural language Processing (NLP) is a sub-discipline within Artificial Intelligence (AI) that enables the synthesis and analysis of speech and text, and in doing so, arms computers with the ability to understand and communicate with humans and other machines.
Today, NLP is broadly adopted by businesses across industries in several forms. In fact, according to recent research, the global NLP market size is expected to reach $35.1 billion by 2026. This ubiquity of the technology form can be accorded to the abundance of text and voice data as well as the shift from human-computer interaction to human-computer conversation.
In my upcoming talk at the Open Data Science Conference (ODSC) East, I am excited to be sharing my thoughts on how NLP is already aiding businesses, trends to keep an eye out for in the near future, and things to keep in mind when it comes to adopting NLP solutions. Outlined below is what you can expect me to discuss in detail during the presentation.
Common NLP Functions Being Used Across Industries
At its core, NLP works with unstructured data, such as text documents, customer feedback forms, user logs, and web pages. As a result, the technology form has myriad applications across industries. Some common examples include:
- Chatbots: Today, NLP-powered chatbots are commonplace in industries like finance and retail to answer frequently asked customer support questions, review account details, and for making payments. Increasingly, such virtual assistance solutions are serving as intelligence augmentation tools. This means any customer request is first intercepted by the virtual assistant, and only after it fails to resolve the request, will it be passed along to a human.
- Sentiment Analysis: Most sectors are keen to improve the quality of the customer’s experience through feedback and regularly identify and limit the risk of customer churn. Increasingly, NLP is being used to assist customer service teams in gauging customer needs or temperament and suggesting appropriate responses.
- Information Retrieval: Increasingly, in industries like banking, HR, and legal, where hundreds and thousands of documents are created and need to be processed every day, information retrieval can aid with time and resource management. Semantic Search, a function of NLP, can be used to extract information from millions of documents even when that exact phrase or search term doesn’t exist. Sometimes this retrieval process is followed by a Q&A system, which highlights the answers to specific queries. NLP’s Document Search or Enterprise Search and Discovery function provides a greater ability to sort and find relevant information more quickly.
- Entity Extraction: Entity Extraction systems have the ability to extract specific portions/entities of interest from within a document, which can later be used to search on a need basis or to generate analytics and decisions. This functionality is widely appreciated in the field of compliance, human resource, and legal. Entity extraction and Clustering Analysis also help identify the most relevant topics that are being discussed, which can be valuable for predicting trends, and valuable for industries like marketing, retail, and media.
What the Future Holds
Business:
- Summarization and categorization: We live in a world where attention spans are increasingly volatile. There are branches of NLP that deal with summarizing large bodies of text like news articles and presenting them as bite-sized bullet points.
Similar systems also work towards categorizing and organizing documents. A large portion of the “tags” you see online (which are very helpful with navigating large websites for example) are generated automatically by an AI system that attempts to understand the content that it is presented with. - Improving online accessibility: One interesting use case that is increasing in popularity pertains to using AI systems to look at an image and caption it appropriately. A large swath of the Internet is inaccessible to people that rely on screen readers because the images do not have their respective captions. These AI systems hope to make the world a teensy bit easier to navigate.
- Transcripts from audio or videos: As NLP applications grow more powerful, we are seeing a trend where we not only extract transcripts from audio and videos, but the transcript is further processed into insights. These insights can be summaries, notes, and action items for participants in a call, or just provide more clarity on who is talking on what part of the recording.
There is also an emerging market for real-time processing in fields like health care. The fast-paced nature of the field in an emergency situation, coupled with the need to document interactions for posterity makes this a very important use case. It had been out of reach in the past due to the domain-specific terms used, but recent advancements have been made to overcome the challenges.
- Customized search engines: We have probably seen advancements in search engines where they can highlight the exact bit of information that we need when posed with a question. We have a trend where corporations are implementing that technology specifically tailored to their own internal documents like HR policies, Employee Handbooks, or patent holdings (3M, n.d).
- Conversational AI: It is common to come across older chatbots that present you with a menu from which you choose appropriate options and, successively travel down the levels. This is indeed a tedious process, and chatbots have become infamous because of it. With the emergence of large language models, sophisticated chatbots are able to directly take you to the information you need by understanding what you’re saying, as opposed to working as a proxy for buttons.
- Text to speech: One emerging trend is the adoption of newer text-to-speech synthesis. We all remember the robotic monotone that used to be the signature of computers talking to us. The reason for the eerie nature of that voice is because the individual words are created by gluing together one syllable at a time. With newer “end to end” systems, we can give the AI system a piece of text and produce audio directly. The latest versions of these AI systems produce audio that is almost indistinguishable from human utterances because of the incredible detail it has.
Research and engineering
- Multimodal Systems: Since 2021, there has been a trend towards Multimodal Systems, where the “multi” stands for dealing with language and vision at the same time. For example, we can use natural language to describe an image that we want, and the AI system will create it. A famous demo is to tell the AI system to “create a picture of an Avocado chair” (Open AI, 2021). As ridiculous as this request might be, the system is able to generate this image. AI uses data to learn, and it is probably the case that there are not a lot of Avocado chairs in its training data that it could just copy from. Hence, it points to the fact that the system actually understands the concept of an Avocado and a chair, and was successful at combining them to form an image.
This development can be used to improve semantic search for both text and images. While the traditional system could only leverage either text or vision at a time, now advanced search engines can increasingly understand the different attributes of a search term and retrieve results accordingly, even when dealing with images. - Transformers: A transformer is the name of an ML system architecture that has become ubiquitous in breakthrough research papers on NLP. It is even being borrowed by other branches of ML such as Computer Vision (Houlsby & Weissenborn, 2020). As a result, for many new kinds of data like text, pixels, audio, and even protein sequences, transformers are increasingly coming into play.
- InstructGPT: GPT-3 is a Neural Network that has gargantuan proportions. It has been trained on 45TB of data and needs an extremely large infrastructure to run because of its sheer size (Cooper, 2021). However, we see a trend in research where we see that a neural network of that size can do multiple tasks. The latest release is InstructGPT where you can give it instructions, like writing a paragraph about a certain topic in a certain way, and it fulfills the request.
- Open NLP Libraries: Like most of Computer Science, there is an increasing trend to open source software for use by the general public. Hugging Face, is one example of one of the most popular NLP libraries, which is free and open to the public. It has a mission to democratize NLP, and has made it so much easier to play with a state-of-the-art model without knowing its intricate details.
- Ethical ML Models: Bias and ethics in AI taking a center stage as more and more models that make decisions for real-life problems and business decisions are being deployed at scale. As we know, biased data leads to an AI system that is biased as well, and data that is generated by modeling real people can be biased in more ways than one. As a result, ensuring fairness and robustness of models or NLP-powered predictions will be a core part of research in the future.
- Multilingual NLP: This refers to pushing NLP systems beyond the scope of one language and has been getting traction lately. There are more and more advanced NLP systems in many different languages like Chinese, Hindi, Russian, and others (Moberg, 2020).
NLP research has been going on for decades at this point. But in the last ten years, we have seen exponential growth in the applications of it in the business domain from document processing for compliance and chatbots for customer service to sentiment analysis for marketing. Unlike before the turn of the century, today NLP systems have become much more accurate producing the accuracy levels that humans can trust in business environments producing value for business in many industries. I foresee applications of NLP in business growing even further in the coming decade.
I look forward to discussing this further in my talk in April at ODSC East 2022, “Natural Language Processing in Accelerating Business Growth.” and sharing my thoughts on the things organizations must keep in mind – challenges and opportunities alike – when adopting NLP.
About the Author/ODSC East 2022 Speaker on NLP applications in business
Sameer Maskey is the Founder & CEO of Fusemachines Inc, an AI talent platform and services provider. Dr. Maskey has more than 18 years of experience in artificial intelligence, natural language processing, machine learning, data science and is an Adjunct Associate Professor at Columbia University. After completing his PhD in Computer Science from Columbia University, he joined IBM Watson Research Center where he invented various statistical algorithms to improve speech-to-speech translation and question answering systems.
Website: https://fusemachines.com/ | Linkedin: https://www.linkedin.com/in/sameer-maskey/ |Forbes Tech Council: https://www.forbes.com/sites/forbestechcouncil/people/sameermaskey1/
