Skor Kepatuhan Berbobot Multi-Sumber untuk Verifikasi Label Kemasan Obat dan Suplemen Berbasis YOLO-OCR

Natural Medicine, Quasi-Drug, and Health Supplement products sold in Indonesia must display at least 17 mandatory elements on their packaging labels under BPOM Regulation No. 10 of 2024, yet compliance inspection still relies on slow and inconsistent manual review. This study designs an automated system combining YOLOv8 object detection, multi-engine OCR (PaddleOCR, EasyOCR, Tesseract), and a new confidence-weighted multi-source compliance-scoring method with a three-tier decision (compliant, needs manual review, non-compliant), compared against a binary rule-based baseline. The method fuses detection confidence, OCR confidence, format validity, and an official-registry cross-check through a hybrid mandatory-completeness hard gate and soft weighted score. On 897 annotated images (16 label-element classes; 630 training/134 validation/133 test), the trained YOLOv8s model achieves an average [email protected] of 0.267 and OCR CER of 2.3–4.8. Despite this limited base quality, the weighted method significantly outperforms the binary baseline: balanced accuracy improves from 52.9% to 59.6% and F1 from 0.361 to 0.397 (McNemar p=0.0001), robust to weight perturbation (at most 4.5% of decisions change) but sensitive to decision thresholds (up to 38.4%). Failure analysis shows pure detection misses on complete labels as the dominant error source, indicating base detection/OCR quality, not robustness to non-ideal conditions, is the main practical deployment barrier.