Deep learning evaluation of human and machine translation quality with cross-domain adaptation

Translation quality assessment requires localized error feedback, semantic evaluation, and sensitivity to document context and application domain. This study evaluates a shared-encoder framework integrating word-level error detection, sentence-level semantic alignment, document-level modeling, source-associated representation learning, and MAML-based domain adaptation. Three research questions examine multi-granularity assessment, cross-task representation learning, and adaptation across support-set sizes. On the official WMT23 MQM-scored subsets, the full model reaches Pearson correlations of 0.769 for Chinese–English sentences, 0.781 for English–German paragraphs, and 0.762 for Hebrew–English sentences. The corresponding numerical gains over xCOMET are 0.023–0.024 correlation points. Human–machine classification accuracy is 92.7% in distribution and 86.4% under leave-one-MT-system-out evaluation. On 2000 machine-origin post-edited translations, machine-class recall is 71.3%. Frozen-representation probes recover four linguistic properties with instance-level R 2 values of 0.52–0.72. With 50 labeled support items and 500 labeled validation items per domain, the model reaches correlations of 0.778, 0.765, and 0.748 for medical, legal, and literary translation, respectively. Token-level error-detection F1 ranges from 0.748 to 0.766. Component ablations link the quality-assessment and source-classification tasks through shared representations, while the adaptation results quantify performance under a fixed target-domain model-selection protocol.

Authors

Institutions

Publication Details

Journal
Discover Artificial Intelligence
Published
2026-11-17
DOI
https://doi.org/10.1007/s44163-026-02554-9
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Deep learning evaluation of human and machine translation quality with cross-domain adaptation

Fei Lv, Yan Zhang
2 links
Discover Artificial Intelligence
Natural Language Processing Techniques
article

Deep learning evaluation of human and machine translation quality with cross-domain adaptation

Fei Lv, Yan Zhang
article en

Abstract

Translation quality assessment requires localized error feedback, semantic evaluation, and sensitivity to document context and application domain. This study evaluates a shared-encoder framework integrating word-level error detection, sentence-level semantic alignment, document-level modeling, source-associated representation learning, and MAML-based domain adaptation. Three research questions examine multi-granularity assessment, cross-task representation learning, and adaptation across support-set sizes. On the official WMT23 MQM-scored subsets, the full model reaches Pearson correlations of 0.769 for Chinese–English sentences, 0.781 for English–German paragraphs, and 0.762 for Hebrew–English sentences. The corresponding numerical gains over xCOMET are 0.023–0.024 correlation points. Human–machine classification accuracy is 92.7% in distribution and 86.4% under leave-one-MT-system-out evaluation. On 2000 machine-origin post-edited translations, machine-class recall is 71.3%. Frozen-representation probes recover four linguistic properties with instance-level R 2 values of 0.52–0.72. With 50 labeled support items and 500 labeled validation items per domain, the model reaches correlations of 0.778, 0.765, and 0.748 for medical, legal, and literary translation, respectively. Token-level error-detection F1 ranges from 0.748 to 0.766. Component ablations link the quality-assessment and source-classification tasks through shared representations, while the adaptation results quantify performance under a fixed target-domain model-selection protocol.

Discover Artificial IntelligenceVol. 6(1)
Anhui Wenda University of Information Engineering (CN)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon