University of Twente Student Theses

Login
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.

Structuring the Literature : Classifying Team Formation Problem Research with Large Language Models

Hajidehabadi, M. and Barrios-Fleitas, Y. and Lalla, E. (2025) Structuring the Literature : Classifying Team Formation Problem Research with Large Language Models.

[img] PDF
983kB
Abstract:The Team Formation Problem (TFP) is a widely studied area in operations research and education, yet its literature is fragmented by inconsistent terminology and modeling approaches. This hinders systematic understanding and synthesis of findings across studies. In this work, we explore the use of large language models (LLMs) to automate the classification of academic papers related to TFP using a validated semantic taxonomy. We present a structured classification pipeline that integrates prompt engineering, retrieval-augmented generation (RAG), and schema-based output formatting. Two state-of-the-art LLMs, GPT-4.1 and Gemini 2.5 Flash, are evaluated on a curated dataset of 23 papers using macro-F1 scores and inter-rater agreement with expert annotations. To assess robustness, each model is tested across repeated runs and under both RAG-enabled and non-retrieval settings. Our results show that RAG substantially improves classification accuracy in nuanced dimensions such as modeling constraints and objective structure. While model performance varies across tasks, both LLMs demonstrate strong generalization to papers outside operations research, including those without formal mathematical models. However, consistency issues remain in abstract categories like evolution, highlighting the need for improved prompt calibration and system-level refinement. This study provides evidence that LLMs, when guided by structured inputs and targeted context, can support scalable and interpretable classification in complex academic domains. The proposed framework offers a foundation for automated literature synthesis in areas where methodological diversity and terminological ambiguity have limited traditional approaches.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:50 technical science in general, 54 computer science, 77 psychology, 81 education, teaching
Programme:Business & IT BSc (56066)
Link to this item:https://purl.utwente.nl/essays/107429
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page