Multimodal brain lesion classification approach through MRI radiomics and OCR-recovered MRS metabolites with explainable AI

This work presents a multimodal machine learning pipeline for brain lesion classification, fusing structural radiomic features from multi-sequence MRI (T2-FLAIR, T1-contrast) with metabolic features recovered from Magnetic Resonance Spectroscopy (MRS) screenshots via a custom Optical Character Recognition (OCR) pipeline. The study draws on a clinical cohort of 55 patients (17 non-neoplastic, 38 neoplastic). MRI radiomics span first-order intensity statistics, Grey-Level Co-occurrence Matrix (GLCM) texture, morphological shape, and contrast-enhancement ratios, while MRS metabolites (Cho, Cr, NAA, and their ratios) are lifted from rendered spectra through Tesseract OCR with multi-region cropping and Lanczos upsampling. Features are evaluated under a leakage-controlled Leave-One-Out Cross-Validation (LOOCV) protocol with in-fold imputation, feature selection and normalization, using the classifier’s default decision threshold. The primary, bias-corrected performance estimate comes from nested LOOCV, in which classifier family and feature count are chosen using training data only: the multimodal pipeline reaches 82.0% accuracy. The best fixed configuration (Random Forest, k = 3) reaches 85.5% accuracy (47/55) and AUC = 0.82, but because it is selected on the same data it is scored on, it is reported as an exploratory, optimistic figure. The comparison model, which uses MRI-derived features without OCR-recovered metabolites, reaches 78.2% at its own best configuration, so the best-configuration gain is +7.3 percentage points and the nested estimate exceeds that baseline by +3.8 points. The two strongest discriminators are log NAA ( F = 16.01) and log Cho ( F = 12.89). The best-configuration result holds under 1,000-permutation testing (random-label mean 55.0%) and McNemar’s exact test ( p = 0.05, borderline). At N = 55, and with diagnostic labels that themselves drew on MRS, these results stand as proof-of-concept: OCR-recovered MRS metabolites are informative for reproducing expert diagnostic judgment beyond what MRI-derived features supply alone.

Authors

Institutions

Publication Details

Journal
Frontiers in Neurology
Published
2026-11-12
DOI
https://doi.org/10.3389/fneur.2026.1978957
Primary Topic
Radiomics and Machine Learning in Medical Imaging
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Multimodal brain lesion classification approach through MRI radiomics and OCR-recovered MRS metabolites with explainable AI

N. Maheswari, Azra Fathima, Neha Khadeeja, Manasa Balaji
Frontiers in Neurology
Radiomics and Machine Learning in Medical Imaging
article

Multimodal brain lesion classification approach through MRI radiomics and OCR-recovered MRS metabolites with explainable AI

N. Maheswari, Azra Fathima, Neha Khadeeja, Manasa Balaji
article en

Abstract

This work presents a multimodal machine learning pipeline for brain lesion classification, fusing structural radiomic features from multi-sequence MRI (T2-FLAIR, T1-contrast) with metabolic features recovered from Magnetic Resonance Spectroscopy (MRS) screenshots via a custom Optical Character Recognition (OCR) pipeline. The study draws on a clinical cohort of 55 patients (17 non-neoplastic, 38 neoplastic). MRI radiomics span first-order intensity statistics, Grey-Level Co-occurrence Matrix (GLCM) texture, morphological shape, and contrast-enhancement ratios, while MRS metabolites (Cho, Cr, NAA, and their ratios) are lifted from rendered spectra through Tesseract OCR with multi-region cropping and Lanczos upsampling. Features are evaluated under a leakage-controlled Leave-One-Out Cross-Validation (LOOCV) protocol with in-fold imputation, feature selection and normalization, using the classifier’s default decision threshold. The primary, bias-corrected performance estimate comes from nested LOOCV, in which classifier family and feature count are chosen using training data only: the multimodal pipeline reaches 82.0% accuracy. The best fixed configuration (Random Forest, k = 3) reaches 85.5% accuracy (47/55) and AUC = 0.82, but because it is selected on the same data it is scored on, it is reported as an exploratory, optimistic figure. The comparison model, which uses MRI-derived features without OCR-recovered metabolites, reaches 78.2% at its own best configuration, so the best-configuration gain is +7.3 percentage points and the nested estimate exceeds that baseline by +3.8 points. The two strongest discriminators are log NAA ( F = 16.01) and log Cho ( F = 12.89). The best-configuration result holds under 1,000-permutation testing (random-label mean 55.0%) and McNemar’s exact test ( p = 0.05, borderline). At N = 55, and with diagnostic labels that themselves drew on MRS, these results stand as proof-of-concept: OCR-recovered MRS metabolites are informative for reproducing expert diagnostic judgment beyond what MRI-derived features supply alone.

Frontiers in NeurologyVol. 17
Vellore Institute of Technology University (IN)
Good health and well-being
Radiomics and Machine Learning in Medical Imaging
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon