University of Twente Student Theses
Automating the Cybersecurity Triage Process : A Comparative Study on the Performance of Large Language Models
Bakker, Pascal (2024) Automating the Cybersecurity Triage Process : A Comparative Study on the Performance of Large Language Models.
|
PDF
142kB |
| Abstract: | Security analysts have the task of inspecting cybersecurity alarms to filter false positives and identify their severity: triage. The problem with this process is that it is complicated and time-consuming, limiting the depth and speed of investigations. Whereas other proposed optimizations and automations appear to be very promising, rapid advancements in the development of Large Language Models (LLMs) opened up new possibilities to speed up parts of the triage process that previously required human judgment. This research aims to identify ways in which LLMs can optimize triage, evaluate the performance of these techniques and offer a comparison between different LLMs including GPT-4, Aya, Code Llama, Gemma, Llama 3, Mistral and Phi-3. The study shows that GPT-4 is the most capable model, while Llama 3 and Mistral achieve competitively similar results. The findings in this study are expected to help security teams make informed implementation decisions when optimizing the triage process. The data and scripts used are available in the GitHub repository (https://github.com/PascalNB/llm-triage-automation). |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 02 science and culture in general, 50 technical science in general, 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/100966 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page