Multimodal brain lesion classification approach through MRI radiomics and OCR-recovered MRS metabolites with explainable AI
This work presents a multimodal machine learning pipeline for brain lesion classification, fusing structural radiomic features from multi-sequence MRI (T2-FLAIR, T1-contrast) with metabolic features recovered from Magnetic Resonance Spectroscopy (MRS) screenshots via a custom Optical Character Recognition (OCR) pipeline. The study draws on a clinical cohort of 55 patients (17 non-neoplastic, 38 neoplastic). MRI radiomics span first-order intensity statistics, Grey-Level Co-occurrence Matrix (GLCM) texture, morphological shape, and contrast-enhancement ratios, while MRS metabolites (Cho, Cr, NAA, and their ratios) are lifted from rendered spectra through Tesseract OCR with multi-region cropping and Lanczos upsampling. Features are evaluated under a leakage-controlled Leave-One-Out Cross-Validation (LOOCV) protocol with in-fold imputation, feature selection and normalization, using the classifier’s default decision threshold. The primary, bias-corrected performance estimate comes from nested LOOCV, in which classifier family and feature count are chosen using training data only: the multimodal pipeline reaches 82.0% accuracy. The best fixed configuration (Random Forest, k = 3) reaches 85.5% accuracy (47/55) and AUC = 0.82, but because it is selected on the same data it is scored on, it is reported as an exploratory, optimistic figure. The comparison model, which uses MRI-derived features without OCR-recovered metabolites, reaches 78.2% at its own best configuration, so the best-configuration gain is +7.3 percentage points and the nested estimate exceeds that baseline by +3.8 points. The two strongest discriminators are log NAA ( F = 16.01) and log Cho ( F = 12.89). The best-configuration result holds under 1,000-permutation testing (random-label mean 55.0%) and McNemar’s exact test ( p = 0.05, borderline). At N = 55, and with diagnostic labels that themselves drew on MRS, these results stand as proof-of-concept: OCR-recovered MRS metabolites are informative for reproducing expert diagnostic judgment beyond what MRI-derived features supply alone.
Authors
- N. Maheswari (ORCID: https://orcid.org/0000-0001-9467-4038)
- Azra Fathima
- Neha Khadeeja (ORCID: https://orcid.org/0009-0007-2099-2659)
- Manasa Balaji (ORCID: https://orcid.org/0009-0003-6125-4100)
Institutions
- Vellore Institute of Technology University (IN)
Publication Details
- Journal
- Frontiers in Neurology
- Published
- 2026-11-12
- DOI
- https://doi.org/10.3389/fneur.2026.1978957
- Primary Topic
- Radiomics and Machine Learning in Medical Imaging
- Type
- article
- Field-Weighted Citation Impact
- 0.00