University of Twente Student Theses
A Preliminary Study Examining an Automated Sentiment Analysis on Extracting Sentiment from Session Patient Records in an Eating Disorder Treatment Setting
Huisman, S.M. (2022) A Preliminary Study Examining an Automated Sentiment Analysis on Extracting Sentiment from Session Patient Records in an Eating Disorder Treatment Setting.
|
PDF
828kB |
| Abstract: | Background: Clinicians collect therapy notes within session patient records containing information about patients’ treatment progress. Sentiment analysis is a tool to extract emotional tones from text input and could be used for the evaluation of patients’ sentiment during treatment over time. Therefore, this study aims to investigate the validity of an automated sentiment analysis in extracting sentiment from session patient records within an eating disorder (ED) treatment context against the performance of human raters. Methods: A total of 460 patient session records from eight participants diagnosed with an ED were evaluated on their overall sentiment by an automated sentiment analysis and two human raters separately. The Inter-rater agreement (IRR) between the automated analysis and human raters and IRR among the human raters was analysed by calculating the intra-class correlation (ICC) and weighted Cohen’s Kappa. Further, qualitative differences between the human raters and automated analysis were examined in closer detail. Results: The ICC showed a moderate automated-human agreement (ICC= .55) and the weighted Cohen’s kappa showed a fair automated-human (k = .29) and substantial human-human agreement (k = .68) for the evaluation on overall sentiment. Further, the automated analysis lacks the inclusion of words specific to an ED context. Discussion/Conclusion: This study suggests that the automated sentiment analysis performs worse in discerning sentiment from session patient records compared to human raters and cannot be used within practice. The automated analysis should be further investigated by including context-specific ED words and a more solid benchmark such as patients’ mood should be established. |
| Item Type: | Essay (Master) |
| Clients: | 6Gorillas, Utrecht, Netherlands |
| Faculty: | BMS: Behavioural, Management and Social Sciences |
| Subject: | 77 psychology |
| Programme: | Psychology MSc (66604) |
| Link to this item: | https://purl.utwente.nl/essays/93035 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page