Identifying data anomalies in milk component measurements from partial-day milking records

High-quality milk and milk component data are crucial for accurate genetic evaluations and effective herd management. However, data recording errors can compromise the validity of downstream decisions. In a recent study, we proposed using intraclass correlation coefficients as a herd-level metric to assess the consistency of milk components from single milkings, thereby effectively identifying farms with potential data quality concerns. A key challenge, however, is whether potentially erroneous records can be detected at the cow-day level. In this study, we introduce a novel metric-individual-level intraclass correlations-to assess data consistency at the cow-day level and evaluate its performance against 3 commonly used anomaly-detection methods. We further introduce a 2-step approach to estimate percentile thresholds for flagging outliers. The results demonstrate the superior performance of this new metric over the conventional univariate and multivariate methods in identifying anomalies in correlated partial daily milk component data. In addition, the negative impact of data shuffling was examined. Together, these methods provide robust and practical tools for detecting suspect milk component records at the individual cow-day level.

Authors

Institutions

Publication Details

Journal
JDS Communications
Published
2025-12-13
DOI
https://doi.org/10.3168/jdsc.2025-0825
Primary Topic
Milk Quality and Mastitis in Dairy Cows
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Identifying data anomalies in milk component measurements from partial-day milking records

John B. Cole, João Dürr, Asha M. Miles, R.L. Baldwin et al.
JDS Communications
Milk Quality and Mastitis in Dairy Cows
article

Identifying data anomalies in milk component measurements from partial-day milking records

John B. Cole, João Dürr, Asha M. Miles, R.L. Baldwin, Javier Burchard, Malia J. Caputo, Xiao‐Lin Wu, Steven Sievert, Jay Mattison, Chip Donatone
article en

Abstract

High-quality milk and milk component data are crucial for accurate genetic evaluations and effective herd management. However, data recording errors can compromise the validity of downstream decisions. In a recent study, we proposed using intraclass correlation coefficients as a herd-level metric to assess the consistency of milk components from single milkings, thereby effectively identifying farms with potential data quality concerns. A key challenge, however, is whether potentially erroneous records can be detected at the cow-day level. In this study, we introduce a novel metric-individual-level intraclass correlations-to assess data consistency at the cow-day level and evaluate its performance against 3 commonly used anomaly-detection methods. We further introduce a 2-step approach to estimate percentile thresholds for flagging outliers. The results demonstrate the superior performance of this new metric over the conventional univariate and multivariate methods in identifying anomalies in correlated partial daily milk component data. In addition, the negative impact of data shuffling was examined. Together, these methods provide robust and practical tools for detecting suspect milk component records at the individual cow-day level.

JDS CommunicationsVol. 7(2)
Agricultural Research Service (US), University of Wisconsin–Madison (US), North Carolina State University (US), American Dairy Science Association (US), University of Florida (US), Dairy Management (US), Beltsville Agricultural Research Center (US), University of Guelph (CA)
Openalex Percentile: Top 50%
Milk Quality and Mastitis in Dairy Cows
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.