University of Twente Student Theses
Testing a usability scale for chatbots : the effect of familiarity on satisfaction ratings
Pollmann, N. (2021) Testing a usability scale for chatbots : the effect of familiarity on satisfaction ratings.
|
PDF
953kB |
| Abstract: | Although the use of chatbots as information retrieval assistants becomes more common nowadays, there are many problems that companies encounter when implementing these tools. To find out which facets of those chatbots need improvement to better accommodate the needs of the user, usability scales for chatbots are the most effective tools available. This study aims to perform confirmatory factor analysis on the 15-item Chatbot Satisfaction Scale developed by Borsci et al. (under review). The data of 56 participants, who used an English and a German scale to rate their interaction with 10 different chatbots, were used. The confirmatory factorial analysis suggested a good model fit for the Chatbot Satisfaction Scale that could be further improved by adding a new factor and deleting two items that assessed the same concept as another item. The resulting scale composed of 13 items demonstrated good reliability (α = 0.93) and showed a strong correlation with the UMUX-Lite. The German translation of the new scale was found to have a good reliability (α = 0.95) that was comparable to the original English version. While a moderate to high correlation was found between the two versions, a significant difference suggested that the German version fails to measure the same concepts. Moreover, we also investigated the potential effect of familiarity on the new scale, nevertheless, results suggested no significant effects of familiarity on satisfaction after the interaction with chatbots. |
| Item Type: | Essay (Bachelor) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 77 psychology |
| Programme: | Psychology BSc (56604) |
| Link to this item: | https://purl.utwente.nl/essays/87031 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page