Factual2: Extending a Romanian Fact-Checking Corpus with Speaker Credibility Histories for Leak-Aware Veracity Classification

Credibility history, the per-speaker count of past verdicts, is an important metadata feature in fact-checking benchmarks. This feature can be misused: including credibility history for all examples in the corpus embeds the truthfulness label in the feature vector. We present Factual2, a Romanian fact-checking corpus of 1958 political statements by 417 speakers from factual.ro, annotated with five veracity classes. To our knowledge, Factual2 is the first Romanian corpus to deal with credibility history in a controlled leakage setting. The count vectors are created using Leak-Free Aggregation (LFA), with self-exclusion, cold-start policy, and a stricter causal variant (LFA-temporal). Two leaky approaches are presented for diagnostic purposes; four label schemes and three segmentation protocols are available. The key results are: (i) Pre-Split Aggregation (PSA) overestimates the classification performance of LFA counts by 0.20 macro-F1 in the 5-class setting (0.484 vs. 0.285). (ii) In the hard binary setup, the leaky classifier achieves a macro-F1 of 0.798 in the speaker-disjoint setting, while the honest classifier achieves 0.393, which implies that the gap is due to leakage. (iii) Using the temporal protocol, the honest counts achieve 0.784 macro-F1, equal to the speaker-identity baseline; the maximum gain of the honest approach across the grid is 0.051.

Authors

Institutions

Publication Details

Journal
Electronics
Published
2026-11-12
DOI
https://doi.org/10.3390/electronics15225248
Primary Topic
Misinformation and Its Impacts
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Factual2: Extending a Romanian Fact-Checking Corpus with Speaker Credibility Histories for Leak-Aware Veracity Classification

Bogdan Drăgulescu, Marian Bucos
Electronics
Misinformation and Its Impacts
article

Factual2: Extending a Romanian Fact-Checking Corpus with Speaker Credibility Histories for Leak-Aware Veracity Classification

Bogdan Drăgulescu, Marian Bucos
article en

Abstract

Credibility history, the per-speaker count of past verdicts, is an important metadata feature in fact-checking benchmarks. This feature can be misused: including credibility history for all examples in the corpus embeds the truthfulness label in the feature vector. We present Factual2, a Romanian fact-checking corpus of 1958 political statements by 417 speakers from factual.ro, annotated with five veracity classes. To our knowledge, Factual2 is the first Romanian corpus to deal with credibility history in a controlled leakage setting. The count vectors are created using Leak-Free Aggregation (LFA), with self-exclusion, cold-start policy, and a stricter causal variant (LFA-temporal). Two leaky approaches are presented for diagnostic purposes; four label schemes and three segmentation protocols are available. The key results are: (i) Pre-Split Aggregation (PSA) overestimates the classification performance of LFA counts by 0.20 macro-F1 in the 5-class setting (0.484 vs. 0.285). (ii) In the hard binary setup, the leaky classifier achieves a macro-F1 of 0.798 in the speaker-disjoint setting, while the honest classifier achieves 0.393, which implies that the gap is due to leakage. (iii) Using the temporal protocol, the honest counts achieve 0.784 macro-F1, equal to the speaker-identity baseline; the maximum gain of the honest approach across the grid is 0.051.

ElectronicsVol. 15(22)
Polytechnic University of Timişoara (RO)
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon