Length-scale-dependent evaluation of detection reliability for NDT image data

To aid the acceptance for and adoption of novel automatic defect recognition (ADR) algorithms in non-destructive testing (NDT) and bridge a gap in the perspectives of machine learning (ML) and NDT domains, a framework has been developed to enable reliability metrics to be computed from segmented NDT image data in a length-scale-dependent manner. The proposed algorithm consists of the definition of a target grid, an optional binary dilation, a remapping of arrays to the target grid and the computation of the selected metrics, followed by Monte Carlo sampling of the target grid to obtain a stochastic distribution of reliability metrics. Illustrative results demonstrate the operation and utility of the algorithm. The framework has the potential to improve the transparency of the performance of ADR implementations, enable more meaningful comparisons of ADR options and hence support more widespread deployment.