From replicates to casework: a systematic methodological review of data independence and transportability in artificial intelligence models for complex forensic chemical and biological evidence

Artificial intelligence (AI), particularly machine-learning models, is increasingly applied to complex forensic chemical, material, and biological evidence. High reported performance does not itself demonstrate generalization to new forensic material. We conducted a systematic methodological review of 446 reports and 1,005 eligible evaluation regimes, examining independence of final evaluation, physical and biological parent grouping, information leakage, external or shifted testing, and transportability. Following source-based methodological assessment and human adjudication, source-supported leakage was classified in 284/446 reports (63.7%); 36/446 (8.1%) showed strong potential concern, 85/446 (19.1%) had uncertain boundaries, and 41/446 (9.2%) had no identified leakage concern. External or shifted evaluation was reported in 278/446 reports (62.3%), although only 94/278 (33.8%) had an evaluation satisfying five strict criteria for transportability of an unchanged pipeline. At regime level, 135/521 external or shifted evaluations (25.9%) satisfied these criteria. Source-supported leakage occurred in 195/308 Biological reports (63.3%) and 87/133 Chemical/Material reports (65.4%): prevalence ratio 1.033 (95% CI 0.889–1.200). The primary comparison did not identify a significant difference across these domains. Casework claims should therefore be evaluated against the independence of the relevant forensic unit, the complete development–test boundary, the shift actually examined, any target-data adaptation, and the applicable performance denominator.

Authors

Institutions

Publication Details

Journal
Frontiers in Analytical Science
Published
2026-11-18
DOI
https://doi.org/10.3389/frans.2026.2015605
Primary Topic
Law, AI, and Intellectual Property
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

From replicates to casework: a systematic methodological review of data independence and transportability in artificial intelligence models for complex forensic chemical and biological evidence

Giulia Vignali, Riccardo Primavera, Paolo Bailo, Andrea Piccinini et al.
5 links
Frontiers in Analytical Science
Law, AI, and Intellectual Property
article

From replicates to casework: a systematic methodological review of data independence and transportability in artificial intelligence models for complex forensic chemical and biological evidence

Giulia Vignali, Riccardo Primavera, Paolo Bailo, Andrea Piccinini, Carlotta Virginia Di Francesco, Lorenzo Franceschetti
article en

Abstract

Artificial intelligence (AI), particularly machine-learning models, is increasingly applied to complex forensic chemical, material, and biological evidence. High reported performance does not itself demonstrate generalization to new forensic material. We conducted a systematic methodological review of 446 reports and 1,005 eligible evaluation regimes, examining independence of final evaluation, physical and biological parent grouping, information leakage, external or shifted testing, and transportability. Following source-based methodological assessment and human adjudication, source-supported leakage was classified in 284/446 reports (63.7%); 36/446 (8.1%) showed strong potential concern, 85/446 (19.1%) had uncertain boundaries, and 41/446 (9.2%) had no identified leakage concern. External or shifted evaluation was reported in 278/446 reports (62.3%), although only 94/278 (33.8%) had an evaluation satisfying five strict criteria for transportability of an unchanged pipeline. At regime level, 135/521 external or shifted evaluations (25.9%) satisfied these criteria. Source-supported leakage occurred in 195/308 Biological reports (63.3%) and 87/133 Chemical/Material reports (65.4%): prevalence ratio 1.033 (95% CI 0.889–1.200). The primary comparison did not identify a significant difference across these domains. Casework claims should therefore be evaluated against the independence of the relevant forensic unit, the complete development–test boundary, the shift actually examined, any target-data adaptation, and the applicable performance denominator.

Frontiers in Analytical ScienceVol. 6
Università di Camerino (IT), University of Milan (IT)
Law, AI, and Intellectual Property
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon