University of Twente Student Theses

Login

Artificial Intelligence Conversational Agents: Using Card Sorting To Re-Evaluate The Chatbot Usability Scale

Verschoor, S.L. (2023) Artificial Intelligence Conversational Agents: Using Card Sorting To Re-Evaluate The Chatbot Usability Scale.

[img] PDF
758kB
Abstract:Research previously published on chatbot quality of interaction by Borsci et al. (2021, 2022) resulted in a Bot Usability Scale consisting of 11 items (BUS-11). This scale aims to support the development and assessment of chatbots, and with that increase user satisfaction in the utilization of such systems. With their latest research showing the potential of models that vary in their number of factors (Borsci & Schmettow, 2023) the present research aimed to explore the mental model of chatbot users to determine what model would be the most accurate to use. To uncover the natural mental model of chatbot users this study made use of open card sorting. During card sorting the participants were tasked to organize the items of the BUS-11 in groups. A total of 58 participants took part in the study, resulting in 47 usable results. The results were presented in a heatmap to visualize the found mental model. The heatmap revealed a mental model that was nearly identical to both the 4 and 5-factor models of the BUS-11. Along with these models, the heatmap also presented outlying points of interest. Based on this data the results suggested a model that exists of 4 factors, with one of these factors including 2 sub-factors. Keywords: Chatbots, Usability, Chatbot Usability Scale, User Satisfaction, Card Sort, Heatmap
Item Type:Essay (Bachelor)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:77 psychology
Programme:Psychology BSc (56604)
Link to this item:https://purl.utwente.nl/essays/94244
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page