Blacklist, do you copy? Characterizing information flow in public domain blacklists

In this paper, we will analyse the information flow of public domain blacklists. Various vendors maintain a list of public domain blacklist to prevent access to domains containing malware, phishing, and counterfeit/ fake webshops. Both malware and phishing can have a disastrous impact on society when critical companies or infrastructure are affected. We will explore the information flow in public domain blacklists to make good decisions which blacklist to use, to prevent access to as many malicious domains as possible and not prevent access to benign domains. Research into the overlap between blacklists was already a focus of a couple of studies. However, there was not much attention into the information flow between blacklists, and if there are occurrences of blacklists that copy from each other. We created several metrics to identify occurrences of copying behaviour of blacklists: we will do a pairwise comparison using data from crawled public domain blacklists, looking at intersections, correlations, and finding interesting overlapping domains. In this research, we have identified that it is indeed possible to show that some blacklists copy from another blacklist. We verify this by using data from blacklists which openly mention that they copy from another blacklist.