University of Twente Student Theses


Testing a usability scale for chatbots : the effect of familiarity on satisfaction ratings

Pollmann, N. (2021) Testing a usability scale for chatbots : the effect of familiarity on satisfaction ratings.

[img] PDF
Abstract:Although the use of chatbots as information retrieval assistants becomes more common nowadays, there are many problems that companies encounter when implementing these tools. To find out which facets of those chatbots need improvement to better accommodate the needs of the user, usability scales for chatbots are the most effective tools available. This study aims to perform confirmatory factor analysis on the 15-item Chatbot Satisfaction Scale developed by Borsci et al. (under review). The data of 56 participants, who used an English and a German scale to rate their interaction with 10 different chatbots, were used. The confirmatory factorial analysis suggested a good model fit for the Chatbot Satisfaction Scale that could be further improved by adding a new factor and deleting two items that assessed the same concept as another item. The resulting scale composed of 13 items demonstrated good reliability (α = 0.93) and showed a strong correlation with the UMUX-Lite. The German translation of the new scale was found to have a good reliability (α = 0.95) that was comparable to the original English version. While a moderate to high correlation was found between the two versions, a significant difference suggested that the German version fails to measure the same concepts. Moreover, we also investigated the potential effect of familiarity on the new scale, nevertheless, results suggested no significant effects of familiarity on satisfaction after the interaction with chatbots.
Item Type:Essay (Bachelor)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:77 psychology
Programme:Psychology BSc (56604)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page