University of Twente Student Theses
Patterns of success and failure : Analysing Large Language Models in Question Answering in Exam Contexts
Burlacu, Felicia (2024) Patterns of success and failure : Analysing Large Language Models in Question Answering in Exam Contexts.
|
PDF
608kB |
| Abstract: | Large Language Models (LLMs) have come to be the spotlight of the general public in the recent years, capturing the attention of people among different industries besides tech. LLMs are artificial intelligence (AI) models designed to interact with, process, generate and analyze human language on a large scale using deep learning techniques. It was through their high performance for complex human-level tasks like sentiment analysis, text summarization and question answering, that these models gathered so much attention. However, despite their impressive results and substantial computational abilities, LLMs are not without drawbacks like biases and lack of interpretability in decision-making. But even with these challenges in mind, there are media articles published with titles saying 'GPT-4 beats 90% of the lawyers trying to pass the bar' , or 'Chat-GPT passes the Radiology Board Exam' citing papers with recent research on GPT performance on such examinations. One reason for such evaluation methods would be to apply the LLMs in the context of real world problems in order to show their applicability but also to make these assessments of performance accessible for the general public without much technology background. This research aims to employ a few-shot learning approach in order to identify and analyze patterns in the performance of LLMs, specifically in the context of answering exam-type questions. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/100884 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page