Multimodal Document Analytics for Banking Process Automation

Traditional banks are increasingly challenged by FinTechs, particularly in leveraging advanced technologies to enhance operational efficiency. Our study addresses this by focusing on improving the efficiency of document-intensive business processes in banking. We review the landscape of business documents in the customer banking segment, which often includes text, layout, and visuals, indicating that more than plain NLP is required for effective document analytics and process automation. We evaluate multimodal models that integrate these elements, analyzing the interplay of text, layout, and visual components. Our comparative analysis features multimodal, cross-lingual models such as LayoutXLM and includes comparisons with large language models like GPT-4, contrasting their capabilities with traditional unimodal models like BERT and GPT-3.5. The results confirm that LayoutXLM significantly outperforms both unimodal classifiers and other large multimodal models in processing complex banking documents, leveraging integrated visual and layout cues effectively. Remarkably, with as little as 250 document pages covering 10 different token classes used for fine-tuning, LayoutXLM achieves an F1 score exceeding 75%, demonstrating its efficient few-shot learning capabilities. In our ablation studies, we demonstrate that multimodal systems can achieve significant performance improvements with limited labeled data, effectively managing class imbalance. This enhances their deployment and scalability, highlighting their transformative potential in document-intensive processes. Overall, this paper contributes original empirical evidence on the superior efficiency and effectiveness of layout-aware multimodal models like LayoutXLM, along with practical banking document analytics insights. • Explore the banking document landscape for NLP-based process optimization. • Investigate the need of multimodal models in complex banking document analytics. • Compare multimodal, cross-lingual models like LayoutXLM and GPT-4. • Evaluate the importance of visual and layout components in multimodal models. • Study class imbalance and compare multimodal integration with GPT-3.5 and BERT.