University of Twente Student Theses
Validity and Reliability of the User Satisfaction with Information Chatbots Scale (USIC)
Silderhuis, Imke (2020) Validity and Reliability of the User Satisfaction with Information Chatbots Scale (USIC).
|
PDF
1MB |
| Abstract: | Although the chatbot market is growing, chatbots have difficulty to live up to their potential and often disappear due to disappointing usage (Brandtzaeg & Følstad, 2018). Developers need insight into which chatbot aspects users are satisfied with and which aspects need further improvement to retain their success. As of yet, there are no standardized scales available to assess the user’s satisfaction with chatbots (Balaji & Borsci, 2019). In the current study, we evaluated a promising scale that assesses user satisfaction with information chatbots (USIC). Due to the scale’s multifaceted character, it provides detailed information on various chatbot’s aspects, which is valuable to help chatbot developers improve their chatbots in a targeted manner (Balaji & Borsci, 2019). Balaji and Borsci (2019) provided preliminary evidence for the USIC’s validity and reliability, however the scale needs repeated validity and reliability assessment towards standardization. In this study, we evaluated the USIC’s validity and reliability to further the standardization process. Also, we reduced the scale’s length to make it more feasible to implement. We performed an extended replication of Balaji and Borsci's (2019) usability study. During the study, participants interacted with multiple chatbots and filled out the USIC and UMUX-Lite after each completed chatbot interaction. Results showed evidence for the USIC’s concurrent validity and reliability, measured by the USIC’s factor structure, its relation to the UMUX-Lite and its internal consistency. The findings suggest that the USIC can fulfil the need for a standardized diagnostic scale to measure user satisfaction with information chatbots. The proposed 14-item USIC is especially promising as it is more compact, making it more efficient and more feasible to implement. The USIC enables researchers and chatbot developers to gain more insight into the user’s satisfaction with information chatbots, compare studies and results, and it offers the possibility to improve chatbots in a targeted way. Keywords: Chatbots, user satisfaction, validity, reliability, standardization |
| Item Type: | Essay (Master) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 70 social sciences in general, 77 psychology |
| Programme: | Psychology MSc (66604) |
| Link to this item: | https://purl.utwente.nl/essays/83495 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page