The (Im)Proficiency of Firearms Examiners

For decades, the nation’s largest provider of forensic proficiency tests, Collaborative Testing Services, structured its firearms examination exercises around bullet and cartridge case comparisons so simplistic as to nearly guarantee vanishingly low misidentification rates. Practitioners, in turn, enthusiastically relied on these results during increasingly vigorous admissibility challenges to defend their alleged ability—using little more than their training, experience, and subjective judgment—to divine the precise gun used in a crime. By and large, they succeeded. Although outside scholars (lawyers, statisticians, human factors experts, and more) have remained constant in their criticisms of the field’s methods and the design of its efforts to measure accuracy (in the form of proficiency testing and beyond), untroubled admissibility and untroubled qualification of firearms examiners as “experts” have remained the norm. In 2023, however, CTS (albeit unintentionally) bucked the trend of simplistic testing, releasing an exercise, CTS 23–5262, that forced participants to confront damaged and poorly marked bullets. The results should trouble even those unfamiliar with firearms examination and the debates about its validity. Although the questioned bullets provided to participants were not fired by the same gun as the exemplar bullets in their packets, 22% of test takers failed by “matching” at least one questioned item to the exemplars, and participants on the whole committed misidentifications 37.4% of the time they offered definitive conclusions about the source of the “challenging” bullets in the proficiency test. To highlight the importance of these results, and to emphasize the extent they depart from past proficiency tests, this Article both analyzes CTS bullet exercises dating back to 2006 and dives deeply into CTS 23–5262 itself. Examiners’ performance failures on the latter, this Article ultimately argues, (1) contradict claims that analysts will rarely err when assessing the source of fired ammunition; (2) raise questions about the validity of comparisons performed on damaged bullets; and (3) force litigants and the courts to grapple with the significance of test error when qualifying individual examiners as experts. Indeed, given how grossly CTS 23–5262 runs counter to the claims made by defenders of firearms examination (and the judges who have parroted them), it may be that the test can (or at least should) finally push the field over the edge and out of criminal courts.

Authors

Institutions

Publication Details

Journal
UCLA Criminal Justice Law Review
Published
2026-11-16
DOI
https://doi.org/10.5070/cj8.70319
Primary Topic
Legal processes and jurisprudence
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

The (Im)Proficiency of Firearms Examiners

Richard E. Gutierrez
3 links
UCLA Criminal Justice Law Review
Legal processes and jurisprudence
article

The (Im)Proficiency of Firearms Examiners

Richard E. Gutierrez
article en

Abstract

For decades, the nation’s largest provider of forensic proficiency tests, Collaborative Testing Services, structured its firearms examination exercises around bullet and cartridge case comparisons so simplistic as to nearly guarantee vanishingly low misidentification rates. Practitioners, in turn, enthusiastically relied on these results during increasingly vigorous admissibility challenges to defend their alleged ability—using little more than their training, experience, and subjective judgment—to divine the precise gun used in a crime. By and large, they succeeded. Although outside scholars (lawyers, statisticians, human factors experts, and more) have remained constant in their criticisms of the field’s methods and the design of its efforts to measure accuracy (in the form of proficiency testing and beyond), untroubled admissibility and untroubled qualification of firearms examiners as “experts” have remained the norm. In 2023, however, CTS (albeit unintentionally) bucked the trend of simplistic testing, releasing an exercise, CTS 23–5262, that forced participants to confront damaged and poorly marked bullets. The results should trouble even those unfamiliar with firearms examination and the debates about its validity. Although the questioned bullets provided to participants were not fired by the same gun as the exemplar bullets in their packets, 22% of test takers failed by “matching” at least one questioned item to the exemplars, and participants on the whole committed misidentifications 37.4% of the time they offered definitive conclusions about the source of the “challenging” bullets in the proficiency test. To highlight the importance of these results, and to emphasize the extent they depart from past proficiency tests, this Article both analyzes CTS bullet exercises dating back to 2006 and dives deeply into CTS 23–5262 itself. Examiners’ performance failures on the latter, this Article ultimately argues, (1) contradict claims that analysts will rarely err when assessing the source of fired ammunition; (2) raise questions about the validity of comparisons performed on damaged bullets; and (3) force litigants and the courts to grapple with the significance of test error when qualifying individual examiners as experts. Indeed, given how grossly CTS 23–5262 runs counter to the claims made by defenders of firearms examination (and the judges who have parroted them), it may be that the test can (or at least should) finally push the field over the edge and out of criminal courts.

UCLA Criminal Justice Law ReviewVol. 10(1)
University of Illinois Chicago (US)
Peace, justice, and strong institutions
Legal processes and jurisprudence
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon