University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Optimizing Test Case Generation and Bug Fixing Efficiency Through Hyperparameter Tuning of Local Large Language Models
Linde, Ruben van der (2025) Optimizing Test Case Generation and Bug Fixing Efficiency Through Hyperparameter Tuning of Local Large Language Models.
|
PDF
13MB |
| Abstract: | Large Language Models (LLMs) have become incredibly popular, but using them costs a lot of energy. Hyperparameter tuning is a promising way to reduce this energy consumption. This study investigates the effect that hyperparameter tuning has on local large language models while performing test case generation and bug fixing tasks in Java. To accomplish this, 12 models from four different families were evaluated and compared across multiple configurations. The focus was on commonly tuned parameters—temperature, top-p, and max tokens—and their impact on accuracy and energy consumption. Additionally, the influence of including a method description in the prompt for bug fixing was examined. The results show that temperature has the most noticeable effect on performance, while top-p and max tokens have limited influence, as long as the token limit is high enough to avoid incomplete outputs. Lower bit-width models showed comparable performance across both tasks, and their energy efficiency remained very similar. Furthermore, including the method description led to a significant increase in accuracy and even a small reduction in energy usage. These findings provide a better understanding of how local LLMs behave under different hyperparameter settings and prompt designs |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/107265 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page