Hybrid deep learning and machine learning pipeline for three-class kidney cancer classification from CT images using selective fine-tuning of DenseNet121

Automated classification of renal CT images into Normal, Benign, and Malignant categories is clinically challenging because of the visual overlap between tissue types and the inter-observer variability of manual radiological interpretation. This study evaluates a hybrid pipeline that combines selective two-phase fine-tuning of DenseNet121 with Random Forest (RF) classification of the resulting deep features. The dataset comprises 3,547 CT slices from 404 patients, collected retrospectively at King Abdullah University Hospital (KAUH) under IRB approval; labels were assigned using a composite reference standard of histopathology, where available, and expert radiological diagnosis. In Phase A, the classification head and batch-normalisation parameters are trained while the convolutional backbone remains frozen; in Phase B, the final dense block and normalisation layer are additionally unfrozen. The resulting 1,024-dimensional embeddings are classified by four machine learning models, of which RF performed best on the held-out patient-level test set, with an accuracy of 95.36% (95% CI 93.04%–97.39% at the slice level; 84.2%–98.8% under a conservative patient-level bound), an F1-score of 95.25%, an AUC of 98.72%, a Cohen's kappa of 0.929, and an MCC of 0.930. Under identical optimization settings, the complete pipeline outperformed end-to-end fine-tuned baselines, including MobileNetV2 (87.83%; +7.53 percentage points) and DenseNet121 with the same backbone (81.45%; +13.91 points), although, given the small number of independent test patients, these gains were not, or only borderline, statistically significant under the conservative patient-level bound. Because the pipeline differs from these baselines in both the fine-tuning strategy and the final classifier, these comparisons evaluate the pipeline as a whole rather than either component in isolation. Exact SHAP analysis showed that the RF relies predominantly on features generated within the fine-tuned dense block, while Grad-CAM++ provided a qualitative view of the image regions used by the backbone. Because the proportion of histologically confirmed cases is unknown, the model may partly reproduce routine radiological interpretation rather than predict confirmed pathology. These findings constitute single-center, proof-of-concept evidence and require external, multi-center, prospective validation against a histopathology-confirmed reference standard before any clinical application.

Authors

Institutions

Publication Details

Journal
Frontiers in Medicine
Published
2026-11-17
DOI
https://doi.org/10.3389/fmed.2026.1990505
Primary Topic
AI in cancer detection
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Hybrid deep learning and machine learning pipeline for three-class kidney cancer classification from CT images using selective fine-tuning of DenseNet121

Rabia Emhamed Al Mamlook, Omar Halalsheh, Hamad Yahia Abu Mhanna, Mohammad Amin et al.
2 links
Frontiers in Medicine
AI in cancer detection
article

Hybrid deep learning and machine learning pipeline for three-class kidney cancer classification from CT images using selective fine-tuning of DenseNet121

Rabia Emhamed Al Mamlook, Omar Halalsheh, Hamad Yahia Abu Mhanna, Mohammad Amin, Abdullah Mohammed Al Fatais, Salem Alhatamleh, Amal Alishwait, Hatem Malkawi, Baraa Awad, Khaled Freihat, Tarek Aloada
article en

Abstract

Automated classification of renal CT images into Normal, Benign, and Malignant categories is clinically challenging because of the visual overlap between tissue types and the inter-observer variability of manual radiological interpretation. This study evaluates a hybrid pipeline that combines selective two-phase fine-tuning of DenseNet121 with Random Forest (RF) classification of the resulting deep features. The dataset comprises 3,547 CT slices from 404 patients, collected retrospectively at King Abdullah University Hospital (KAUH) under IRB approval; labels were assigned using a composite reference standard of histopathology, where available, and expert radiological diagnosis. In Phase A, the classification head and batch-normalisation parameters are trained while the convolutional backbone remains frozen; in Phase B, the final dense block and normalisation layer are additionally unfrozen. The resulting 1,024-dimensional embeddings are classified by four machine learning models, of which RF performed best on the held-out patient-level test set, with an accuracy of 95.36% (95% CI 93.04%–97.39% at the slice level; 84.2%–98.8% under a conservative patient-level bound), an F1-score of 95.25%, an AUC of 98.72%, a Cohen's kappa of 0.929, and an MCC of 0.930. Under identical optimization settings, the complete pipeline outperformed end-to-end fine-tuned baselines, including MobileNetV2 (87.83%; +7.53 percentage points) and DenseNet121 with the same backbone (81.45%; +13.91 points), although, given the small number of independent test patients, these gains were not, or only borderline, statistically significant under the conservative patient-level bound. Because the pipeline differs from these baselines in both the fine-tuning strategy and the final classifier, these comparisons evaluate the pipeline as a whole rather than either component in isolation. Exact SHAP analysis showed that the RF relies predominantly on features generated within the fine-tuned dense block, while Grad-CAM++ provided a qualitative view of the image regions used by the backbone. Because the proportion of histologically confirmed cases is unknown, the model may partly reproduce routine radiological interpretation rather than predict confirmed pathology. These findings constitute single-center, proof-of-concept evidence and require external, multi-center, prospective validation against a histopathology-confirmed reference standard before any clinical application.

Frontiers in MedicineVol. 13
Jordan University of Science and Technology (JO), Trine University (US), Al Zawiya University (LY), King Khalid University (SA), Yarmouk University (JO), Isra University (JO)
Jordan University of Science and Technology, King Khalid University
Good health and well-being
AI in cancer detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon