University of Twente Student Theses
Development of a Preliminary Measurement Tool of User Satisfaction for Information-Retrieval Chatbots
Waldera, L. (2019) Development of a Preliminary Measurement Tool of User Satisfaction for Information-Retrieval Chatbots.
|
PDF
1MB |
| Abstract: | Based on the findings of Tariverdiyeva and Borsci (2019), it can be concluded that the implementation of 18 key features of information-retrieval chatbots are directly leading to a change in perceived quality and user satisfaction. Standard measurement tools for the usability of software, like UMUX-Lite (Lewis et al., 2013), are found to be failing at capturing these specific requirements for chatbots. This research aims to reassess and extend the list of relevant key features found by Tariverdiyeva and Borsci (2019). Moreover, this research aims to develop a preliminary list of questionnaire items that can be used in a future measurement tool of user satisfaction for information-retrieval chatbots from an end user’s perspective. This is achieved by conducting a systematic literature review and discussions with possible end users in focus groups in the first experimental phase and conducting a usability testing with different chatbots in the second experimental phase. After the first phase, a total of 14 features were found to be relevant to measure user satisfaction. Questionnaire items for these features were generated and used for the development of a preliminary questionnaire – UMT/C. After the second experimental phase, the UMT/C was found to be correlating with the standard measurement tool UMUX-Lite. A factor analysis was applied to the questionnaire data which showed a total of 9 factors being measured by 25 items. While some of the factors correlated highly with the results of the UMUX-Lite scale, others were found to capture additional information related to user satisfaction. Therefore, the present study recommends to test the quality of this preliminary questionnaire and to refine the pool of items in order to develop a valid and reliable tool for assessing the quality of chatbots from an end user’s perspective. Keywords: information-retrieval chatbots, usability testing, UMUX-Lite, questionnaire development, factor analysis |
| Item Type: | Essay (Bachelor) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 77 psychology |
| Programme: | Psychology BSc (56604) |
| Link to this item: | https://purl.utwente.nl/essays/79258 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page