Factual2: Extending a Romanian Fact-Checking Corpus with Speaker Credibility Histories for Leak-Aware Veracity Classification
Credibility history, the per-speaker count of past verdicts, is an important metadata feature in fact-checking benchmarks. This feature can be misused: including credibility history for all examples in the corpus embeds the truthfulness label in the feature vector. We present Factual2, a Romanian fact-checking corpus of 1958 political statements by 417 speakers from factual.ro, annotated with five veracity classes. To our knowledge, Factual2 is the first Romanian corpus to deal with credibility history in a controlled leakage setting. The count vectors are created using Leak-Free Aggregation (LFA), with self-exclusion, cold-start policy, and a stricter causal variant (LFA-temporal). Two leaky approaches are presented for diagnostic purposes; four label schemes and three segmentation protocols are available. The key results are: (i) Pre-Split Aggregation (PSA) overestimates the classification performance of LFA counts by 0.20 macro-F1 in the 5-class setting (0.484 vs. 0.285). (ii) In the hard binary setup, the leaky classifier achieves a macro-F1 of 0.798 in the speaker-disjoint setting, while the honest classifier achieves 0.393, which implies that the gap is due to leakage. (iii) Using the temporal protocol, the honest counts achieve 0.784 macro-F1, equal to the speaker-identity baseline; the maximum gain of the honest approach across the grid is 0.051.
Authors
- Bogdan Drăgulescu (ORCID: https://orcid.org/0000-0002-1285-202X)
- Marian Bucos (ORCID: https://orcid.org/0000-0001-8205-6697)
Institutions
- Polytechnic University of Timişoara (RO)
Publication Details
- Journal
- Electronics
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3390/electronics15225248
- Primary Topic
- Misinformation and Its Impacts
- Type
- article
- Field-Weighted Citation Impact
- 0.00