Automated corner grading of trading cards: Defect identification and confidence calibration through deep learning

This research focuses on trading card quality inspection, where defects have a significant effect on both the quality inspection and grading. The present inspection procedure is subjective which means the grading is sensitive to mistakes made by individuals. To address this, a deep neural network based on transfer learning for automated defect detection is proposed with a particular emphasis on corner grading which is a crucial factor in overall card grading. This paper presents an extension of our prior study, in which we achieved an accuracy of 78% by employing the VGG-net and InceptionV3 models. In this study, our emphasis is on the DenseNet model where convolutional layers are used to extract features and regularisation methods including batch normalisation and spatial dropout are incorporated for better defect classification. Our approach outperformed prior findings, as evidenced by experimental results based on a real dataset provided by our industry partner, achieving an 83% mean accuracy in defect classification. Additionally, this study investigates various calibration approaches to fine-tune the model confidence. To make the model more reliable, a rule-based approach is incorporated to classify defects based on confidence scores. Finally, a human-in-the-loop system is integrated to inspect the misclassified samples. Our results demonstrate that the model’s performance and confidence are expected to improve further when a large number of misclassified samples, along with human feedback, are used to retrain the network. • Proposed a transfer learning model to automate corner grading in trading cards. • Various calibration techniques are explored to enhance model reliability. • Proposed a rule-based method using confidence scores to improve corner grading accuracy. • A human-in-the-loop system is integrated to inspect instances of low machine confidence.