Unlocking domain-specific image annotations with AI : A human-in-the-loop approach for generating interior-design insights using large language models and vision language models

The home-furnishing brand IKEA prioritizes its core ability to analyze the interior-design elements of rooms. These analyses enhance its digital capabilities such as image tagging, product recommendations, and personalised interior-design advice. This study attempted to automate such room analyses using automated room image annotations with the GPT-4 series of large language models (LLM) and vision-language models (VLM). It followed a three-pronged approach that included generating taxonomies with the LLM, designing multistage prompts, evaluation with domain-experts and incorporating other human-in-the-loop strategies. The resulting AI-generated room analyses, though with a margin of error, were deemed sufficient for automating various room-analysis tasks. Furthermore, they demonstrated the effectiveness of the prompting strategies and human-in-the-loop practices used through the process, suggesting broad applicability in AI development beyond IKEA and interior design alone. Recommendations such as co-prompt-design sessions, domain-specific evaluation criteria, large-scale evaluations, were made to improve the process.