Chatbot evaluation for effectiveness in customer query resolution

Abstract

Background: The adoption of artificial intelligence technologies, specifically chatbots, has grown tremendously in various industries and is expected to transform how businesses communicate with and resolve customer queries. Yet, fewer empirical studies have been conducted on how chatbots can be assessed for their effectiveness in resolving customer queries.
Objectives: This study aims to uncover and identify research gaps through bibliometric analysis. Bibliometric analysis allows for the exploration of conversational chatbot evaluation in resolving customer queries.
Method: A comprehensive analysis of 27 literature articles published between 2015 and 2024 was conducted using data retrieved from the Web of Science database. The study encompasses various analytical approaches, such as performance analysis and science mapping techniques, and includes keyword co-occurrence analysis and citation network visualisation, to elucidate the distribution of publications, influential authors and institutions, and critical research disciplines.
Results: Findings reveal a growing body of research on chatbot evaluation predominantly in well-developed countries, spanning diverse disciplines such as Business and Economics, Computer Science, and Engineering. Examining citation networks suggests interconnectedness in the literature, with specific articles emerging as central nodes of influence.
Conclusion: The study’s implications for future research include the importance of interdisciplinary collaboration, a deeper examination of aspects of chatbot design, user experience, and interaction dynamics, and prioritising context-sensitive approaches to effective chatbot deployment and evaluation in emerging countries.
Contribution: Overall, this bibliometric analysis offers valuable insights into the current state of research on chatbots and provides a foundation for future endeavours in this rapidly evolving research domain.