Reducing Hallucinations in Enterprise Generative AI with Retrieval-Augmented Generation

Hallucination – the generation of plausible but incorrect information by large language models (LLMs) – poses a serious risk for enterprises that use generative AI assistants on internal documentation. This thesis evaluates the effectiveness of Retrieval-Augmented Generation (RAG) in reducing hallucinations in an enterprise context. A custom demo system was developed using Convex and Next.js. This system combines two embeddings models (Open AI and Google) and two chunking strategies. The system was tested under five different configurations, including a baseline configuration without retrieval. Both the automated and manual evaluations were used to measure hallucination severity and frequency. The results demonstrate that all RAG configurations significantly reduced hallucinations compared to the baseline configuration, with Google embeddings and smaller chunk sizes performing best. The inter-rater agreement between the annotators was high, and the trends in the automated evaluations closely followed the manual evaluation patterns. These findings support RAG as an effective strategy to increase factual accuracy of generative AI system on internal documentation in enterprise environments.