University of Twente Student Theses

Login

Testing of a usability assessment tool for chatbots : investigating the effect of believing that a chatbot might be a human

Neumeister, S. (2020) Testing of a usability assessment tool for chatbots : investigating the effect of believing that a chatbot might be a human.

[img] PDF
1MB
Abstract:Information-retrieval chatbots have gained more importance for customer service in recent years, but a reliable and valid usability measure tailored towards chatbots is still missing. The Usability Satisfaction Questionnaire (USQ) might solve this issue. This study tested the reliability and validity of the USQ by comparing it to the UMUX-Lite. Furthermore, a PCA was carried out and the results have been compared to a previous study. Thirty-nine participants interacted with five chatbots each and completed two tasks per chatbot. Additionally, the belief that a human controls a chatbot, as well as trust in relation to the USQ has been investigated. For this, 15 participants were deceived. The PCA suggested a six-component structure with 32 items and a positive correlation has been found between the USQ and the UMUX-LITE. Moreover, a positive correlation has been found between the USQ and trust as well as between the USQ and the belief that a human controlled the chatbot. The results suggest that the USQ is a reliable and valid measure of user satisfaction, and the extracted components overlapped with those of previous studies. However, all outcomes should be seen with caution, as Covid-19 influenced the execution of the study.
Item Type:Essay (Bachelor)
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:77 psychology
Programme:Psychology BSc (56604)
Link to this item:https://purl.utwente.nl/essays/81714
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page