Checking Ai Output In Police Open-Source Investigations: A Step-By-Step Methodological Review

This methodological review asks, for each step of a police open-source investigation, what can go wrong and what can go unexpectedly right when AI tools are used, and which checks protect the investigation against the first while keeping it open to the second. It reviews studies, tests, standards and EU law across four steps: collection, processing, analysis and verification, and reporting. Reported performance varies at every step, and a published figure does not show whether the tool will do worse or better on a new case. In this paper's reasoning, based on info-gap decision theory, checks that rely on the tool, such as its confidence, a second model's agreement or a reviewer who sees only the AI output, weaken when the tool is worse than believed. By the same reasoning, checks that bring outside information, such as the original file, an independent source or a second method, hold up better, since their judgement does not rest on the tool's output. The tool may also perform better than believed, including finding what nobody asked for, and the investigation keeps that gain if AI output enters first as a lead and becomes evidence only after a check that brings outside information. The review proposes that units place such checks before AI output grounds an arrest, a warrant, a data request, a publication or a court filing, and record every AI use so that others can check the work. None of these checks has been tested in police casework.

Authors

Publication Details

Journal
CrimRxiv
Published
2026-11-13
DOI
https://doi.org/10.21428/cb6ab371.bd2d3f36
Primary Topic
Artificial Intelligence in Law
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Checking Ai Output In Police Open-Source Investigations: A Step-By-Step Methodological Review

Filippo Tansini
2 links
CrimRxiv
Artificial Intelligence in Law
preprint

Checking Ai Output In Police Open-Source Investigations: A Step-By-Step Methodological Review

Filippo Tansini
preprint en

Abstract

This methodological review asks, for each step of a police open-source investigation, what can go wrong and what can go unexpectedly right when AI tools are used, and which checks protect the investigation against the first while keeping it open to the second. It reviews studies, tests, standards and EU law across four steps: collection, processing, analysis and verification, and reporting. Reported performance varies at every step, and a published figure does not show whether the tool will do worse or better on a new case. In this paper's reasoning, based on info-gap decision theory, checks that rely on the tool, such as its confidence, a second model's agreement or a reviewer who sees only the AI output, weaken when the tool is worse than believed. By the same reasoning, checks that bring outside information, such as the original file, an independent source or a second method, hold up better, since their judgement does not rest on the tool's output. The tool may also perform better than believed, including finding what nobody asked for, and the investigation keeps that gain if AI output enters first as a lead and becomes evidence only after a check that brings outside information. The review proposes that units place such checks before AI output grounds an arrest, a warrant, a data request, a publication or a court filing, and record every AI use so that others can check the work. None of these checks has been tested in police casework.

CrimRxiv
Peace, justice, and strong institutions
Artificial Intelligence in Law
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon