The difference between NLU and NLP

Applying AI Technology to ITSM: The Difference Between NLP & NLU (Part Three)

Next Story

Challenges Ahead on UK Information Technology Hiring Front

In my previous post, I shared examples of how AI and machine learning (ML) technologies are being applied and delivering value in IT today.

We also reviewed how historical and observed data is used to train machine learning models to address the entire IT service delivery lifecycle.

In this post, I’ll discuss how virtual agents that leverage a combination of natural language processing (NLP) and natural language understanding (NLU) capabilities provide unique value in enabling IT to deliver better employee service.

NLP & NLU: what’s the difference?

The terms NLP and NLU both refer to systems that are designed to allow computers and humans to interact with one another using natural language as opposed to keywords and codes. While there are different underlying system architectures (i.e., machine learning, deep learning, neural networks, etc.) that can distinguish NLU from NLP, generally speaking, NLU can be thought of as an advanced form (superset) of NLP.

When considering an AI for ITSM initiative, it’s important to understand the value of both NLP and NLU. Each approach enables unique capabilities in terms of how virtual agents engage employees.

To draw a clear distinction between NLP and NLU, it is helpful to classify them by their quantitative and qualitative abilities.

Natural Language Processing

NLP is a quantitative, statistical, data-driven form of language processing that is based on approximation and probability estimations. While data-driven approaches to language processing are accurate to a certain degree, they rely more on mathematical theory than language and human cognition.

In this way, NLP excels at discovering patterns in data, which allows virtual agents to engage in simple, scripted dialogs. However, due to the fact that NLP based virtual agents can only understand a fraction of everyday spoken language, they tend to misunderstand non-routine questions when a conversation deviates from the script. As a result, virtual agents for employee service that rely solely on NLP only handle a subset of inquiries and require continual maintenance.

Natural Language Understanding

NLU builds upon NLP by adding a qualitative, linguistic approach to language processing. NLU-based virtual agents understand user intent and as a result can have fluid conversations- a capability that is essential in service domains. In IT, for example, a simple password reset request may require multiple clarifying questions about app version, role, or operating system. NLP alone can diagnose the intent is resetting a password. NLP + NLU can both diagnose and resolve the issue.

Better together:

With NLP based virtual agents, as long as employees provide highly structured, predictable requests, the system can reply with scripted answers.

As a baseline, it is essential that virtual agents are trained to recognize standard definitions, terms and responses.

However, in a dynamic environment like the IT service desk, virtual agents must also be able to take unstructured and unpredicted inputs, convert them into a structured form and take action on them. For instance, NLU based virtual agents can identify misspelled words, contractions and colloquialisms to better recognize user intent, allowing them to engage in more personalized and natural conversations.

A hybrid approach to language processing, leveraging both NLP & NLU technologies, strengthens a virtual agent’s ability to interact with humans in a natural way.

Additionally, the most effective virtual agents can access multiple data sources and systems, just as human agents do, in order to fulfill requests and/or answer questions. For example, to assist an employee with gaining access to Salesforce a virtual agent must be able to navigate the employee through all the necessary account creation and approval steps, automate the user configuration in Salesforce, and record a service desk ticket in the ITSM system for tracking purposes.

The ability to deliver IT services to employees in real-time while adhering to process workflows allows IT organizations to reduce MTTR and cost per ticket without sacrificing customer satisfaction or compromising proper IT governance and control.

Stay tuned for my next post where I’ll examine the value of NLP and NLU based virtual agents in scaling IT service delivery to meet the needs of an increasingly mobile and global workforce.


The following two tabs change content below.

Robert Young

Robert Young is the Senior Director of Product Marketing at Astound, a company focused on automating IT service and support using AI and NLP. Robert has two decades of experience in ITSM, including 5 years at IDC as Research Director for ITSM and Client Virtualization Software. He also previously held IT operations and knowledge management positions at several large enterprises and while on active duty with the U.S. Air Force. Robert continually looks at how cloud, mobile, and AI technologies are changing not only how users interact with devices, but also how IT organizations interact with their customers.