University of Twente Student Theses

Login

Validating the Bot Usability Scale for Satisfaction with Chatbots: Factor Structure, Convergent Validity and Relation with Workload

Taha, M (2022) Validating the Bot Usability Scale for Satisfaction with Chatbots: Factor Structure, Convergent Validity and Relation with Workload.

[img] PDF
834kB
Abstract:The aim of this study was to validate the Bot Usability Scale (BUS), a standardised scale for chatbot satisfaction. The main objectives were to confirm the BUS’ five-factor structure, high reliability, and correlation with the UMUX-Lite, and to test the BUS’ correlation with workload, which typically has an inverse relationship with satisfaction. For this, 58 participants solved tasks for five chatbots and rated each chatbot on the BUS, UMUX-Lite and NASA-TLX in an online study. 388 BUS-questionnaires were analysed using confirmatory factor analysis and Cronbach’s alpha. The relationships of 297 UMUX-Lite and NASA-TLX questionnaires with the BUS were analysed using Kendall’s Tau and linear mixed models. As expected, the five-factor structure, high reliability, and correlation with the UMUX-Lite were confirmed, and the BUS correlated negatively with workload. These findings support the use of the BUS as a standardized measure for chatbot satisfaction. However, there was a potential Heywood case for item 1, possibly requiring modification or removal. Future research should aim to replicate these findings with chatbots with low satisfaction and high workload rates and using multiple tasks per chatbot, research whether item 1 presents a Heywood case and investigate the relationships between the BUS and variables other than workload.
Item Type:Essay (Bachelor)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:70 social sciences in general, 77 psychology
Programme:Psychology BSc (56604)
Link to this item:https://purl.utwente.nl/essays/91568
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page