Investigating differentially private learning for document key information extraction

Abstract Modern deep learning models have driven significant performance breakthroughs in document key information extraction (KIE) systems. At the same time, however, their data-driven nature exposes these systems to a new class of privacy vulnerabilities. Since KIE systems directly interact with sensitive information in documents, they are potential targets for privacy attacks. Despite this, research on privacy-preserving multimodal document KIE remains scarce. To address this gap, we conduct the first systematic investigation into privacy-preserving training for multimodal document KIE models, comparing differential privacy (DP), federated learning (FL), and differentially private federated learning (DP-FL) approaches. Through extensive experimentation across six benchmark datasets, we demonstrate that while DP training typically causes substantial performance degradation, this can be effectively addressed through: (1) the use of pretrained document foundation models; (2) DP-suited hyperparameter tuning; and (3) task-specific pretraining that aligns target entity types with the private data distribution. In particular, this combination allows up to 77–93% retention of non-private baseline performance in both standalone and federated setups (via global server-side optimization) while maintaining strong privacy guarantees ( $$\varepsilon \in $$ ε ∈ {8, 20}). Notably, task-specific pretraining alone contributes major performance gains of 2–6.5% when using out-of-distribution public data (partial entity-type overlap) and up to 20% when using in-distribution synthetic data (full overlap). To empirically assess the privacy strength of each approach, we evaluate them against membership inference attacks, and show that non-private KIE models can be highly vulnerable, with the attack reaching near-perfect success rates on some datasets, whereas both FedAVG and DP substantially mitigate this vulnerability. Finally, through systematic analysis of hyperparameters, architectural choices, and pretraining strategies, we provide practical guidelines for achieving optimal privacy-utility trade-offs, enabling practitioners to deploy privacy-preserving document KIE systems in real-world applications.

Authors

Institutions

Publication Details

Journal
International Journal on Document Analysis and Recognition (IJDAR)
Published
2026-11-13
DOI
https://doi.org/10.1007/s10032-026-00626-1
Primary Topic
Privacy-Preserving Technologies in Data
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Investigating differentially private learning for document key information extraction

Stefan Agne, Ahmed Sheraz, Saifullah Saifullah, Andreas Dengel
1 links
International Journal on Document Analysis and Recognition (IJDAR)
Privacy-Preserving Technologies in Data
article

Investigating differentially private learning for document key information extraction

Stefan Agne, Ahmed Sheraz, Saifullah Saifullah, Andreas Dengel
article en

Abstract

Abstract Modern deep learning models have driven significant performance breakthroughs in document key information extraction (KIE) systems. At the same time, however, their data-driven nature exposes these systems to a new class of privacy vulnerabilities. Since KIE systems directly interact with sensitive information in documents, they are potential targets for privacy attacks. Despite this, research on privacy-preserving multimodal document KIE remains scarce. To address this gap, we conduct the first systematic investigation into privacy-preserving training for multimodal document KIE models, comparing differential privacy (DP), federated learning (FL), and differentially private federated learning (DP-FL) approaches. Through extensive experimentation across six benchmark datasets, we demonstrate that while DP training typically causes substantial performance degradation, this can be effectively addressed through: (1) the use of pretrained document foundation models; (2) DP-suited hyperparameter tuning; and (3) task-specific pretraining that aligns target entity types with the private data distribution. In particular, this combination allows up to 77–93% retention of non-private baseline performance in both standalone and federated setups (via global server-side optimization) while maintaining strong privacy guarantees ( $$\varepsilon \in $$ ε ∈ {8, 20}). Notably, task-specific pretraining alone contributes major performance gains of 2–6.5% when using out-of-distribution public data (partial entity-type overlap) and up to 20% when using in-distribution synthetic data (full overlap). To empirically assess the privacy strength of each approach, we evaluate them against membership inference attacks, and show that non-private KIE models can be highly vulnerable, with the attack reaching near-perfect success rates on some datasets, whereas both FedAVG and DP substantially mitigate this vulnerability. Finally, through systematic analysis of hyperparameters, architectural choices, and pretraining strategies, we provide practical guidelines for achieving optimal privacy-utility trade-offs, enabling practitioners to deploy privacy-preserving document KIE systems in real-world applications.

International Journal on Document Analysis and Recognition (IJDAR)
University of Kaiserslautern (DE), German Research Centre for Artificial Intelligence (DE), Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau (DE)
Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau
Privacy-Preserving Technologies in Data
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon