A method to identify textual features of superior integrated reports issued by Japanese listed companies, using BERT

An integrated report is a report that contains information on a company’s ESG activities. For institutional investors who make ESG investments, integrated reports are an important source of information that enables them to learn about a company’s ESG activities. On the other hand, it is not easy for institutional investors to evaluate integrated reports because they are long. Under these circumstances, the objective of this study is to propose a method that identifies characteristics of superior integrated reports using BERT: Bidirectional Encoder Representations from Transformers. To do so, a pre-trained BERT is finetuned to be able to identify texts in superior integrated reports. Then, by looking at the fine-tuned BERT attention, words that contributed to model’s decision can be revealed. To be specific, since these words are labeled with word embeddings, principal component analysis is applied for the word embeddings to calculate the first and second principal component scores for each word. Then, the elbow method is used for the calculated scores to estimate clusters. Finaly, these clusters are considered to represent characteristics of superior integrated reports. This study focuses on integrated reports issued by Japanese listed companies.