Integrating precision and power criteria in determining the number of raters needed for standard area diagram validation studies
Abstract Visual estimation of plant disease severity is widely used but inherently subjective, resulting in variability among raters. Standard area diagrams (SADs) are commonly employed to improve accuracy and precision; however, the number of raters required to validate SAD performance is often arbitrarily defined. This study aimed to establish an empirical framework to estimate the number of raters needed to achieve precise estimates of mean agreement and sufficient statistical power to detect improvements due to SAD use. Data from nine SAD validation studies across diverse plant disease pathosystems were analyzed. Agreement was quantified using Lin’s concordance correlation coefficient (LCCC). Two complementary criteria were applied: a precision criterion targeting a relative standard error of 0.10 for mean rater LCCC, and a power criterion evaluating absolute LCCC improvements of 0.05, 0.10, 0.15, and 0.20. Effects were assessed for feasibility relative to baseline agreement and converted exactly to the inverse hyperbolic tangent scale. The pooled precision estimate was seven raters, whereas study-specific precision estimates ranged from 2 to 16, indicating that the pooled value is not a universal recommendation. For an improvement of 0.10, combined sample-size estimates ranged from 5 to 32 raters across eight studies; this improvement was infeasible for pecan brown spot because baseline agreement was already 0.926. Required sample sizes depended on baseline agreement, variability in paired differences, and the specified improvement. These estimates provide context-specific guidance for SAD validation studies, although diagnostic and bootstrap analyses indicate that analytic power estimates require caution at small sample sizes. An interactive web application was developed to support adaptive estimation of rater numbers.
Authors
- Ricardo Gomes Tomáz (ORCID: https://orcid.org/0000-0001-7708-4263)
- Emerson M. Del Ponte (ORCID: https://orcid.org/0000-0003-4398-409X)
- Clive Howard Bock (ORCID: https://orcid.org/0000-0003-1163-268X)
- Mary Paz Romero-Benavides (ORCID: https://orcid.org/0009-0007-2412-6862)
- William C. Nunes
Institutions
- Universidade Estadual de Maringá (BR)
- Agricultural Research Service (US)
- United States Department of Agriculture (US)
- Universidade Federal de Viçosa (BR)
- U.S. Horticultural Research Laboratory (US)
Publication Details
- Journal
- European Journal of Plant Pathology
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1007/s10658-026-03339-7
- Primary Topic
- Reliability and Agreement in Measurement
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- U.S. Department of Agriculture
- Universidade Federal de Viçosa
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Agricultural Research Service