RechtBERT : Training a Dutch Legal BERT Model to Enhance LegalTech

The introduction of generative AI has got massive user interest and sparked discussions about AI adoption in the legal domain. Many AI applications in the legal field are built upon the BERT transformer architecture, but little research has been conducted on adapting such models to the Dutch language. This research aims to design a domain-specific legal Dutch BERT model that outperforms generic Dutch BERT models, enabling legal professionals to perform tasks more efficiently and advance legal tech through NLP applications. It introduces a set of domain-specific legal Dutch BERT models called RechtBERT. We conclude that further pre-training existing Dutch BERT models does not yield better performance on legal NLP tasks than using the Dutch BERT models out of the box. This research addresses a gap in literature regarding the development and use of domain-specific legal Dutch BERT models.