University of Twente Student Theses

Login

Facilitating Industrial B2B e-Auctions through Multi-Agent and Retrieval Augmented Large Language Models

Mansour, Noor (2024) Facilitating Industrial B2B e-Auctions through Multi-Agent and Retrieval Augmented Large Language Models.

[img] PDF
7MB
Abstract:Purchasing professionals often face challenges in the auction design process due to the time-intensive and complex nature of the task. The intricacies of auction dynamics, and the negative impact of subpar auction design choices on realized savings, highlight the need for advanced tools that can facilitate more informed decision-making, optimize auction strategies, and automate the design process. This research project addresses these challenges by designing, implementing, and evaluating a multi-agent and retrieval augmented conversational chatbot system, based on Large Language Models (LLMs) to assist in the auction design process. Leveraging a proprietary dataset from a case company of executed reverse e-auctions, the study first conducts an empirical analysis of theoretical auction design models and examines the impact of several key factors like the number of bidders, price dispersion, and the risk aversion of bidders on the auction design choice and realized savings. A new empirically optimized recommendation model that demonstrates the potential for achieving higher expected savings compared to existing models is proposed. Notably, the study defines specific operationalizations for the practical use of the recommendation models and finds that the empirical evidence supports the theoretical recommendations in a small majority of cases. The insights derived from this empirical analysis are integrated into the chatbot system’s knowledge base alongside the dataset of e-auctions, allowing it to generate customized auction design recommendations. The effectiveness of this system is evaluated using the AuctionEval dataset, created specifically to assess the performance of the system on common use cases around auction design. The experiments explore how information volume, type, and various prompt engineering techniques impact the chatbot’s performance. The findings reveal that smaller models significantly benefit from tailored, focused corpora, while larger models gain from a more diverse corpus. Additionally, the implementation of chain of thought prompting has markedly improved the relevance, quality of answers, and reasoning accuracy of the system. The study is limited by the analysis of only single-phase auctions and a limited range of auction types in the dataset, which restricts the applicability of the validation of theoreti- cal models. Furthermore, the multi-agent conversational chatbot has limitations regarding performance in data retrieval tasks, and the evaluation procedure can benefit from a larger experimental setup and evaluation dataset. These limitations highlight the need for future research to encompass multi-phase auctions and a larger experimental setup and evaluation of the chatbot application. The practical and scientific implications of this research are twofold: first, filling the research gap of missing empirical evidence on auction design recommendation models; second, delivering an innovative, adaptable, and robust LLM system that simplifies the auction design process for purchasing professionals and highlights the reasoning capabili- ties of LLMs for currently under-explored game theoretical use cases. Through the large adaptability of the system to new data and tasks, the architecture promises to be beneficial for a range of tasks in the purchasing domain beyond auction design.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science, 83 economics
Programme:Computer Science MSc (60300)
Link to this item:https://purl.utwente.nl/essays/98821
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page