University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Reducing Hallucinations in Enterprise Generative AI with Retrieval-Augmented Generation
Jong, Sem de (2025) Reducing Hallucinations in Enterprise Generative AI with Retrieval-Augmented Generation.
|
PDF
888kB |
| Abstract: | Hallucination – the generation of plausible but incorrect information by large language models (LLMs) – poses a serious risk for enterprises that use generative AI assistants on internal documentation. This thesis evaluates the effectiveness of Retrieval-Augmented Generation (RAG) in reducing hallucinations in an enterprise context. A custom demo system was developed using Convex and Next.js. This system combines two embeddings models (Open AI and Google) and two chunking strategies. The system was tested under five different configurations, including a baseline configuration without retrieval. Both the automated and manual evaluations were used to measure hallucination severity and frequency. The results demonstrate that all RAG configurations significantly reduced hallucinations compared to the baseline configuration, with Google embeddings and smaller chunk sizes performing best. The inter-rater agreement between the annotators was high, and the trends in the automated evaluations closely followed the manual evaluation patterns. These findings support RAG as an effective strategy to increase factual accuracy of generative AI system on internal documentation in enterprise environments. |
| Item Type: | Essay (Bachelor) |
| Clients: | Canon Medical Europe BV., Amstelveen, Netherlands |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Business & IT BSc (56066) |
| Link to this item: | https://purl.utwente.nl/essays/107394 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page