Blog

|

August 19, 2024

12 minutes read

Study Shows Traditional Chatbots Are Failing

Conversational AI vs Chatbots

By

Andrew Graf

It comes as no surprise that only 6 percent of IT leaders surveyed believe chatbots are effective and highly adopted for self-service. We’ve all had a bad chat experience when trying to use a traditional chatbot for customer service – the conversations are clunky, your intentions are often misunderstood and the success rate of the interaction is often extremely low.

Chatbots have long been touted as a valuable tool for IT Service Management (ITSM), offering a promising solution for handling common use cases and deflecting traffic from overwhelmed IT service desks.

IT leaders are often keen to implement chatbots to supplement ticketing inquiries, expedite response times and enhance end-user satisfaction, but the outcomes have been mixed. The concept is that these chatbots can simulate human conversations, replacing hands-on-keyboard technicians to deliver faster results for IT customers, from answering basic questions to directing them to the appropriate human resource for escalation or triggering automated fixes to their problems.

IT teams utilize traditional chatbots to improve self-service functionality, reduce ticket volumes and enhance delivery speeds. Nevertheless, most teams have yet to achieve the desired outcomes they had anticipated from their initial chatbot deployments.

Why is this the case?

Traditional chatbots operate by utilizing question/answer pairs based on knowledge base information and manually entered intent-based pairs. However, the conversation often deteriorates quickly because the chatbot misinterprets the utterance or question, and its ability to provide answers is limited to the knowledge base. And it’s this poor experience that results in unhappy customers.

According to a recent market study on chatbots and conversational AI from InformationWeek, 48 percent of respondents say their existing chat technology does not accurately solve issues or gets intent wrong, and 38 percent say it’s time-consuming to manage and doesn’t self-learn. This last point coincides with the fact that 29 percent of IT departments say they must manually load intent/answer pairs into the chat platform.

Additionally, respondents reported on the ways that chat tools break down, with the top answers being:

  • Failure to understand the user query (61%)
  • Incorrect or inaccurate answers (45%)
  • Failing to understand natural language (43%)

So, what’s the alternative?

Chatbots are Dead, the Future is Conversational AI

Conversational AI is a broader term that refers to AI-driven natural language processing (NLP) that powers a range of automated communication vehicles like chatbots, virtual assistants and more advanced automated phone center applications.

Conversational AI platforms use data and NLP to recognize vocal and text inputs, mimic human interactions and facilitate conversational flow. The right conversational AI can potentially help lackluster chatbot deployments overcome their current challenges.

Because conversational AI chatbots understand intent, they are much more effective when it comes to assisting customers and end-users.

And while conversational AI is a much better alternative to traditional chatbots – not all conversational AI is equal.  There are different ways that NLP AI can be “taught.” The difference in capability all comes down to what kind of language “brain” the self-learning AI model uses.

The reason why some chat technologies can get things so wrong all the time is that the brain they use is not quite fully developed. Many of the proprietary models used to create these NLP brains are working with insufficient training data and rudimentary models.

Because of this, it’s recommended that you don’t invest in a conversational AI tool that’s built around a small, proprietary NLP engine. Instead, find a tool that’s been built using a standardized NLP engine developed by the likes of Microsoft, Google, or Amazon Web Services.

Utilizing a conversational AI built with standardized NLP created by a major provider with strong AI science resources ensures chat interactions are intentional and meaningful. This under-the-hood capability gives IT teams a greater shot at higher success rates with their chat, which in turn means people will keep coming back to utilize the tool.

And the higher your adoption rate is the better your conversational AI experience can be. Because the more a self-learning conversational chat channel is used (one with a good NLP brain, that is), the more intelligent it can be—meaning the experience continues to get better and evolve.

The Elements of a Successful Chat for Self-Service

According to the survey, there are four key elements for a successful chat experience when it comes to using conversational AI for self-service. The top cited element was a strong knowledge base for content to feed the chat, which was named by 75 percent of respondents. That was closely followed by the ability to personalize the conversation with details about the customer, named by 63 percent. Third and fourth place were ranked equally, named by 62 percent, who equally weighted the ability to present questions to the employee that drives dynamic content with the ability to automate the fulfillment of requests from the chat interaction.

Conversational AI especially benefits service management teams when paired with enterprise integration and automation. This combination can elevate chat from a glorified knowledge base search engine into an automated, action-centered channel to field requests.

No matter how good the knowledge base, personalized user information can only be uncovered through the inherent ability to connect to business systems via APIs and integrations. This capability is what makes it possible to automatically provide dynamic content and fulfill simple, repetitive requests for action.

One example of this in action would be an employee asking the chatbot how much paid time off (PTO) they have left for the year. A first-generation chatbot may not be able to answer that, instead offering a link to the employee knowledge base article about how much PTO each employee gets annually.

A conversational AI chatbot tied to a well-connected integration and automation layer could personalize the response leveraging Single Sign-On, and then access the employee’s data from another application to deliver an accurate, fast response. In this case, the response may say, “Currently, you have 12 days of PTO left this year.” It might even follow up with a question like, “Do you want to know how many of these days will roll over next year?” or, “Would you like to request time off?” If the end-user response is to request time off, the solution would present a form for the request to be entered and then pass that data back to the PTO tracking platform.

TeamDynamix Conversational AI Chat in Action

Before using TeamDynamix, Bowdoin College used an AI chatbot from a different provider.  

The college is a long-time user of the TeamDynamix ITSM platform and has derived a great deal of value from the platform’s integration and automation capabilities, Jason Pelletier, senior director of client services and technology said. So, when campus leaders learned that TeamDynamix was launching a conversational AI tool that can integrate with hundreds of enterprise systems, it seemed like a natural choice to switch.  

The connectivity is being managed through an integration hub that offers hundreds of pre-built connectors to enterprise systems. If you have a specific solution that is not in the library, there is a connector concierge that will build this for you. Connectivity is ready to go for systems like Workday, HRIS, Salesforce and the Active Directory. 

“Being able to integrate the bot with backend systems is very exciting,” Pelletier said. “With TeamDynamix, we have one platform for ITSM with a chat tool that can integrate with our enterprise systems, and then from there we can build automation – this is something we could never have done with our previous solution.” 

The capabilities of the conversational AI tool allow for simple, drag-and-drop workflow and automation building. You can easily create integrations between various programs—with no special coding knowledge required. The no/low-code nature of the solution offers a rapid time to value and endless use cases. 

Pelletier and his team have taken advantage of this integration and automation capability to create special use cases for their AI chatbot. For example, all employees at Bowdoin receive replacement computers every four or five years, and this is something the organization tracks within the TeamDynamix asset management module. Employees have traditionally reached out to the IT support team to ask when their computer is eligible for replacement.

Now, the bot can answer this question automatically with a tailored personalized answer. By integrating the chat with the asset management system, the query will take the users’ information from SSO to then identify and look up their asset records. Once retrieved, the chat can then take that information back to the chat window.  

“We are answering questions faster, with accurate personalized data,” Pelletier said. “When a customer has a question about their equipment, the chat is integrated back into our asset system so that it can give full details to the user.” 

TeamDynamix conversational AI has enabled IT staff to offload much of the work they do in answering peoples’ questions to the bot, allowing them to focus on more important tasks.  

“We’re just scraping the surface right now,” Pelletier said. “Eventually, I think we could have the bot handle up to 30 percent of the questions our help desk staff are getting now.” 

Not only does the bot reduce the workload for IT support staff; but it also creates a better and more equitable experience for students.  

“Some people are outgoing, and they have no problem in coming to our Tech Hub to get technical support,” Pelletier explained. “Others have no problem communicating by email. But there are also many users with a language barrier or another reason why they might feel uncomfortable interacting with an actual person. There are also some users who are night owls and need support late at night when we’re closed.

The chatbot allows us to meet users where they are and give them the assistance they need—whenever, wherever and however they might want it. If we can relieve even some of the stress they might feel, that enhances their experience.”

Want to learn more about chat vs. conversational AI? Read the full market study: State of Chatbots and Conversational AI.

Andrew Graf

Related Articles