After implementing a Chatbot, companies need to monitor it closely. And since the expectations of any implementation are improved response speed and higher conversion, companies must define the correct metrics.
Analytics are often overlooked and underestimated when it comes to Chatbots. While analytics of statistics and metrics are unlikely to define the success of a Chatbot, they can provide valuable insight into opportunities to improve user experience.
KPIs (Key Performance Indicators) will differ depending on the use case and the type of user, however, several key metrics will provide valuable information for almost any Chatbot.
Monitoring active users is a must for most software applications, and Chatbots are no different.
Total Users Over Time: This is the most basic metric. Capture the number of people using your Chatbot. This is important because its trend shows the change in the number of users over time. Additionally, it delivers critical information regarding market size calculations and potentially the effect of your Chatbot.
Session time: This metric is important to analyze the time since a user started and finished interacting with a chatbot. A very low value indicates poor chatbot performance.
New Users: This metric captures the overall success of the campaign. New users will be needed to justify maintaining a chatbot. Customer preferences change over time, so a continuos decrease would put into question the existence of a chatbot.
User location: It is possible to check the general physical location of users by the first digits of their IP address.
Depending on the objective of the chatbot, the measurement of the results can take into account different variables.
Completed forms: Depending on whether the objective of the chatbot is to generate leads, conduct surveys, reservations, etc., it is necessary to measure the results of the main objective of the chatbot.
Reservations made: In case the objective is to make appointments or reservations through the chatbot.
Message metrics help provide qualitative information about the content of the conversation with users. They range from the number of interactions to the analysis of the sentiment of the conversation.
Interactions: Basic metric that refers to the number of messages received and sent from a chatbot to a specific user.
Answer not found: This metric helps analyze the success of the chatbot in searching for answers in its database and delivering it to users. A high value for no answers found suggests that you need to monitor conversations to add more responses to the response templates.
Sentiment analysis: It is possible to obtain an analysis of the sentiment of the conversation, to monitor frustration and satisfaction through the negative or positive feedback that the user shows in the content of his message.
Other critical metrics to measure the performance of a Chatbot.
Bounce Rate: This measurement helps determine the performance of the chatbot by analyzing the times the user started interacting with the chatbot, or left the conversation after an initial message.
A / B test: You can perform A / B tests to analyze the performance of two scripts based on your goals. For example, two scripts can be alternated for two days to analyze the number of results they produced given the number of times they were shown to a user.
