University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Structuring the Literature : Classifying Team Formation Problem Research with Large Language Models
Hajidehabadi, M. and Barrios-Fleitas, Y. and Lalla, E. (2025) Structuring the Literature : Classifying Team Formation Problem Research with Large Language Models.
|
PDF
983kB |
| Abstract: | The Team Formation Problem (TFP) is a widely studied area in operations research and education, yet its literature is fragmented by inconsistent terminology and modeling approaches. This hinders systematic understanding and synthesis of findings across studies. In this work, we explore the use of large language models (LLMs) to automate the classification of academic papers related to TFP using a validated semantic taxonomy. We present a structured classification pipeline that integrates prompt engineering, retrieval-augmented generation (RAG), and schema-based output formatting. Two state-of-the-art LLMs, GPT-4.1 and Gemini 2.5 Flash, are evaluated on a curated dataset of 23 papers using macro-F1 scores and inter-rater agreement with expert annotations. To assess robustness, each model is tested across repeated runs and under both RAG-enabled and non-retrieval settings. Our results show that RAG substantially improves classification accuracy in nuanced dimensions such as modeling constraints and objective structure. While model performance varies across tasks, both LLMs demonstrate strong generalization to papers outside operations research, including those without formal mathematical models. However, consistency issues remain in abstract categories like evolution, highlighting the need for improved prompt calibration and system-level refinement. This study provides evidence that LLMs, when guided by structured inputs and targeted context, can support scalable and interpretable classification in complex academic domains. The proposed framework offers a foundation for automated literature synthesis in areas where methodological diversity and terminological ambiguity have limited traditional approaches. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 50 technical science in general, 54 computer science, 77 psychology, 81 education, teaching |
| Programme: | Business & IT BSc (56066) |
| Link to this item: | https://purl.utwente.nl/essays/107429 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page