Measuring a country’s digital industrial structure: commercial websites and weakly supervised classification to the rescue

Abstract In this paper, we propose the use of commercial websites and a contextualized weak supervision framework as an alternative to industrial taxonomies to identify and classify digital industrial activity. Despite the crucial importance of industrial taxonomies for government and research, their static nature leaves taxonomies unable to accurately capture a country’s industrial structure. This is particularly problematic in the context of firms producing novel, digital outputs, which are nowadays classified into the wrong industrial sectors and thus rendered almost invisible to official statistics. To address this issue we show how commercial websites can complement—or even substitute—industrial classification surveys and ultimately yield a more complete, up-to-date understanding of a country’s industrial structure evolution. In the process, we compare our classification results using only commercial websites’ landing pages versus using full websites for classification, finding that a company’s landing page is a better predictor of industrial classes than their full website. To demonstrate the applied potential of our proposed working pipeline, we propose a case study on the Greater London region (the UK), providing previously invisible insights into existing geographical inequalities in the data economy. We also suggest that our framework could support longitudinal analyses by proposing a pipeline using archival websites. This method can be used by policymakers to identify classes of industries from a bottom-up perspective, while at the same time advocating for the usage of state-of-the art NLP techniques in economics and business research. Keywords: industrial classification, microeconomic data, natural language processing, web data