University of Twente Student Theses
Perceivable Unfairness of Conversational Agents : Implications for Trust, Competence, Helpfulness and Usability
Markiewicz, Nikola (2024) Perceivable Unfairness of Conversational Agents : Implications for Trust, Competence, Helpfulness and Usability.
|
PDF
953kB |
| Abstract: | In the contemporary society, chatbots, a type of AI-based conversational agents (CAs) are one of the vastly developing technologies utilised in multiple domains such as marketing, academia, or customer service. As with any system, there are potential risks associated with using chatbots that can spread inaccurate or biassed (unfair) information. The spreading of bias and misinformation puts minority groups at risk of facing discrimination due to their individual characteristics. On top of that, previous research suggests that male and female chatbots tend to be treated and rated differently. Thus, this study focused on the effect of the level of unfairness and appearance of the chatbot (sex: female vs male) on the quality of the interaction measured through perceived trust, helpfulness, competence, and usability. To test this, first, a pilot study was conducted to create and validate explicit negatively biassed sentences that were used in the main phase of the experiment as chatbot knowledge. Participants interacted with a chatbot and reported their attitude and experience ratings before and after the interaction by filling items about trust, usability, helpfulness and competence. A pre-post, between-subjects design 2 (appearance of the chatbot: male or female) by 3 (level of hallucinations: 100% fair, 50% fair/unfair, or 100% unfair) was employed. The main findings show a significant effect of unfairness (p <.001) and no effect of appearance (p = .267). Significant differences between the pre and post-measurements were identified in the fair and unfair conditions. These results might suggest that people anticipate and thus tolerate a certain level of unfairness in chatbots. When the chatbot is fair, expectations are exceeded, and post ratings are significantly different. When the chatbot is unfair, people tolerate it to a certain extent, as there is no significant difference between pre and post-measures of usability and helpfulness. There is, however, a significant decrease in perceived trust and competence. Future research should further validate the stimuli created during the pilot, manipulate the expressions of chatbot appearance, the effect of previous experience and expectations, and possibly manipulate the levels of fairness by adding more conditions. |
| Item Type: | Essay (Bachelor) |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 77 psychology |
| Programme: | Psychology BSc (56604) |
| Link to this item: | https://purl.utwente.nl/essays/100022 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page