University of Twente Student Theses

Login

Unlocking domain-specific image annotations with AI : A human-in-the-loop approach for generating interior-design insights using large language models and vision language models

Balasubramanian Yamuneswari, Sandhiyaa (2024) Unlocking domain-specific image annotations with AI : A human-in-the-loop approach for generating interior-design insights using large language models and vision language models.

[img] PDF
18MB
Abstract:The home-furnishing brand IKEA prioritizes its core ability to analyze the interior-design elements of rooms. These analyses enhance its digital capabilities such as image tagging, product recommendations, and personalised interior-design advice. This study attempted to automate such room analyses using automated room image annotations with the GPT-4 series of large language models (LLM) and vision-language models (VLM). It followed a three-pronged approach that included generating taxonomies with the LLM, designing multistage prompts, evaluation with domain-experts and incorporating other human-in-the-loop strategies. The resulting AI-generated room analyses, though with a margin of error, were deemed sufficient for automating various room-analysis tasks. Furthermore, they demonstrated the effectiveness of the prompting strategies and human-in-the-loop practices used through the process, suggesting broad applicability in AI development beyond IKEA and interior design alone. Recommendations such as co-prompt-design sessions, domain-specific evaluation criteria, large-scale evaluations, were made to improve the process.
Item Type:Essay (Master)
Clients:
IKEA
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Interaction Technology MSc (60030)
Link to this item:https://purl.utwente.nl/essays/103786
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page