University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Using LLMs as Assistants for Maintaining Rule-Based IOC Extractor Tools
Milev, Konstantin (2025) Using LLMs as Assistants for Maintaining Rule-Based IOC Extractor Tools.
|
PDF
502kB |
| Abstract: | Indicators of compromise (IOCs) are forensic artifacts, such as malicious IP addresses, URLs, file hashes, or malware names, that signal a likely system breach. Accurate detection and extraction of such indicators from open threat reports is crucial for timely defense, as delays in identifying IOCs can lead to missed opportunities to contain or mitigate a threat on time. Rapid IOC recognition enables real-time alerts, automated blocking, and faster incident response. Traditionally, open-source tools and research prototypes rely on hand-crafted regular expressions (regex) and rule-based extractors to identify IOCs in text. This constraint is problematic as human analysts face an overwhelming volume of unstructured reports. These static patterns struggle with variable threat report syntax, obfuscation (e.g., defanged URLs), and novel IOC formats. Recent advancements in AI, especially Large Language Models (LLMs), offer powerful natural language understanding that can identify entities and relationships in text. By leveraging LLMs to suggest or adapt regex patterns based on new threat reports, we aim to increase IOC coverage and adaptability while reducing manual effort. This paper concludes that, when augmented with gemma3:27b-generated regexes, the rule-based extractor’s average recall jumps from 37.9% to 69.1% and its F1 score climbs from 41.0% to 55.3%, while precision increases modestly from 46.6% to 50.4%. By contrast, the smaller Regex-AI-Llama-3.2-1B:F16 model yielded only marginal gains (mean recall 39.2%, F1 32.6%). These results show that larger LLMs can substantially broaden IOC coverage, yet the broader patterns they generate can introduce false positives. As a result, maintaining high extractor reliability in a dynamic threat landscape still depends on a human-in-the-loop workflow to review and refine LLM-suggested rules. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/107455 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page