Designing A Framework for Process Modelling using Large Language Models in a Multi-Agent Architecture

Author(s): Monsma, W.H. (2026)

Abstract:

This thesis addresses a recurring challenge in business process management: domain experts often have strong operational knowledge but struggle to translate it into process models without consultancy support. In OrangeSpot’s onboarding context, this creates a bottleneck between scalable self-service and high-quality consultant-led modelling.
Following a design science approach combining stakeholder interviews, a literature review on process generation and evaluation methods, and four iterative design cycles, a prototype was developed for a process modelling assistant. Early prototypes explored different levels of automation and guidance, gradually converging on a hybrid approach that combines an agent-based architecture with deliberate interaction design. The final assistant employs multiple specialised agents and a human-in-the-loop workflow, where users approve suggested changes, control the modelling scope, and switch between process and entity perspectives.
A between-subjects evaluation compared AI-assisted modelling with manual modelling using a standard onboarding scenario. While expert-rated model quality was on average comparable between conditions, the AI-assisted results showed a wider range of outcomes. Participants reported higher perceived efficiency and ease of use, suggesting the assistant effectively reduces the modelling barrier. However, the limited sample size restricts statistical confidence, and consistency remains a key challenge. Overall, the results highlight clear potential for LLM-based modelling support, with propositions for future work focusing on more consistent outcomes.

Document(s):

Monsma_MA_EEMCS.pdf