Controlled Evaluation of Document-Level Inter-Mention Links for Named-Entity Type Classification in Ukrainian Texts

Document-level links between repeated mentions can provide contextual evidence unavailable to a local entity-type classifier, a potentially important capability for Ukrainian because of its limited annotated resources and rich inflectional morphology. However, their specific contribution is difficult to isolate because graph-based systems often simultaneously alter the classifier architecture. We evaluate named-entity type classification with known mention boundaries on NER-UK 2.0 under controlled conditions, using frozen XLM-RoBERTa-base representations, 10 optimization seeds, and 270 graph-head training runs. The evaluation design combines a no-message baseline with matched head parameterization, degree-preserving topology shuffling, simple averaging, oracle diagnostics, and controlled context-weakening interventions; no novel graph architecture is claimed. On intervention-free data, the exact-repeat graph changed macro-F1 by +0.0041 (95% CI [−0.0042,0.0127]); thus, an overall benefit was not established, although the interval did not rule out a practically meaningful effect. A Ukrainian token-level lemmatization rule increased graph coverage from 34.41% to 42.07% but did not improve performance over exact repetition (Δ=−0.0033, 95% CI [−0.0135,0.0055]). When only the target mention’s context was artificially weakened, with no explicit degradation cue, graph refinement increased target accuracy by 0.0784 (95% CI [0.0544,0.1021]). The advantage reversed when the entire repeat component was weakened, while simple averaging performed at least as well as the learned graph head. Overall, the results support graph-based refinement as a conditional context-recovery mechanism rather than a universal replacement for local classification. The evidence remains controlled but not independently confirmatory and does not establish gains for end-to-end NER or robustness to naturally degraded context.

Authors

Institutions

Publication Details

Journal
Applied Sciences
Published
2026-10-13
DOI
https://doi.org/10.3390/app162010121
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Controlled Evaluation of Document-Level Inter-Mention Links for Named-Entity Type Classification in Ukrainian Texts

Volodymyr Shymkovych, Damian Grela, Oleksii Hryshyn
Applied Sciences
Topic Modeling
article

Controlled Evaluation of Document-Level Inter-Mention Links for Named-Entity Type Classification in Ukrainian Texts

Volodymyr Shymkovych, Damian Grela, Oleksii Hryshyn
article en

Abstract

Document-level links between repeated mentions can provide contextual evidence unavailable to a local entity-type classifier, a potentially important capability for Ukrainian because of its limited annotated resources and rich inflectional morphology. However, their specific contribution is difficult to isolate because graph-based systems often simultaneously alter the classifier architecture. We evaluate named-entity type classification with known mention boundaries on NER-UK 2.0 under controlled conditions, using frozen XLM-RoBERTa-base representations, 10 optimization seeds, and 270 graph-head training runs. The evaluation design combines a no-message baseline with matched head parameterization, degree-preserving topology shuffling, simple averaging, oracle diagnostics, and controlled context-weakening interventions; no novel graph architecture is claimed. On intervention-free data, the exact-repeat graph changed macro-F1 by +0.0041 (95% CI [−0.0042,0.0127]); thus, an overall benefit was not established, although the interval did not rule out a practically meaningful effect. A Ukrainian token-level lemmatization rule increased graph coverage from 34.41% to 42.07% but did not improve performance over exact repetition (Δ=−0.0033, 95% CI [−0.0135,0.0055]). When only the target mention’s context was artificially weakened, with no explicit degradation cue, graph refinement increased target accuracy by 0.0784 (95% CI [0.0544,0.1021]). The advantage reversed when the entire repeat component was weakened, while simple averaging performed at least as well as the learned graph head. Overall, the results support graph-based refinement as a conditional context-recovery mechanism rather than a universal replacement for local classification. The evidence remains controlled but not independently confirmatory and does not establish gains for end-to-end NER or robustness to naturally degraded context.

Applied SciencesVol. 16(20)
National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute” (UA), Cracow University of Technology (PL)
Openalex Percentile: Top 13%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.