Post-hoc Interpretability of LDA on Hyperspectral Imagery: SHAP Attributions, Counterfactual Topic Flips, and LLM-judge Alignment under Token-Mass-Dispersion Asymmetry
Programme paper P5 of the CAOS_LDA_HSI series. The interpretability claim of LDA-on-HSI rests on the assumption that topic-word distributions are human-readable. This study is restricted to the LDA backbone; the cross-backbone comparison (HDP, ProdLDA, ETM) is the subject of the companion paper P4. We test three orthogonal post-hoc interpretability axes on the V1-V15, V17-V20 wordification sweep (nineteen LDA-fitted recipes) from companion paper P3: F-13 SHAP attribution of pixel-to-topic decisions via a closed-form posterior, F-22 counterfactual L1 perturbation required to flip a document's argmax topic, and F-15 LLM-as-judge alignment between a document's top tokens and its argmax topic's top tokens. Three concrete results. (1) F-13 SHAP gives recipe-specific explanations consistent across scenes: V1 reduces to specific wavelength bands, V7 to absorption features, V12 to clusters of Gaussian-mixture components. (2) F-22 counterfactual L1 separates recipes into robust (V20=26.3, V12=24.5, V3=23.5), moderate (V1=6.1, V7=5.2) and fragile (V9=1.0, V10=1.2) regimes; the most-robust basis (V20) is ~26x harder to flip than V9's. (3) F-15 LLM-judge alignment is anti-correlated with F-2 coherence on large-vocabulary recipes, but cardinality cannot explain the gap: V20 matches V3/V12 alphabet sizes yet scores F-15 = 0.64 against their 0.12/0.16. We make the mechanism quantitative with the token-mass dispersion metric (effective vocabulary N_eff = exp(H(phi_k))): N_eff falls V3 (431) > V12 (400) > V20 (309) while F-15 rises in lock-step, so the artefact is driven by token-mass dispersion, not nominal vocabulary cardinality. We recommend reporting F-13 SHAP top-K attributions as the primary interpretability artefact, F-22 as the topic-stability artefact, and F-15 only after dispersion normalisation. Code and derived artefacts: https://github.com/fsantibanezleal/CAOS_LDA_HSI . Interactive web application: https://lda-hsi.fasl-work.com . Manuscript sources: https://github.com/fsantibanezleal/CAOS_LDA_HSI_Paper . Funding: The Advanced Mining Technology Center (AMTC) Basal project (ANID/PIA Project AFB220002) and ANID FONDECYT Postdoctorado 3220094.
Authors
- Felipe A. Santibáñez-Leal (ORCID: https://orcid.org/0000-0002-0150-3246)
Institutions
- Open University of Cyprus (CY)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-07-23
- DOI
- https://doi.org/10.5281/zenodo.21504352
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint