University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Improving LLM Accuracy with Knowledge Graphs in Solving Algebra Problems
Nishijima, Shun (2025) Improving LLM Accuracy with Knowledge Graphs in Solving Algebra Problems.
|
PDF
1MB |
| Abstract: | Large Language Models (LLMs) have demonstrated remarkable capabilities in many real-world applications. Recent models also show high performance in solving simple calculations. However, LLMs are often criticized for producing hallucinations when solving problems that require accurate symbolic reasoning, such as algebra tasks. Some researchers have explored approaches that use large knowledge bases or fine-tuning to solve mathematical problems. Others have explored using structured knowledge to generate one-shot answers to general knowledge questions. Still, there is little research on using small, flexible, structured knowledge to improve LLMs’ mathematical reasoning. This research proposes a lightweight, domain-specific knowledge graph (KG)-based prompting method to enhance LLM accuracy in solving high school algebra problems. A manually constructed KG containing key algebraic concepts and procedures is injected into the prompts to provide a structured context. Using 40 algebra problems and two LLMs (GPT-4o and GPT-4.1-mini), the study demonstrates that KG-based prompts improve average factual accuracy from 61.5% to 73.75% with statistically significant gains, particularly in expression simplification tasks. These results suggest that small, targeted KGs may serve as an effective, low-cost alternative to improve reasoning accuracy in LLMs without requiring retraining or external tools. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 50 technical science in general |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/107260 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page