Low-Resource Topic Modeling for Domain Experts: A Pilot Study in CO2 Mineralization Literature

Bibliometric topic-modeling is essential for synthesizing scientific literature, yet its adoption by domain-expert teams is hindered by technical barriers. This pilot study benchmarks four topic modeling paradigms on 286 CO2 mineralization papers (2013–2025), with all methods implemented end-to-end on consumer hardware by researchers without prior natural language processing (NLP) training. Its principal methodological contribution is a four-level calibration protocol for zero-shot large language models (LLMs) used as innovation detectors. Naive zero-shot prompting flagged 71.0% of the corpus as innovative, over-attributing novelty by a factor of 22.7 relative to a calibrated statistical ensemble. A qualitative-strict prompt reduced this to 26.6%, while a structurally different unified prompt yielded 28.0%. Together, these results indicate a floor near 28% that prompt engineering alone does not overcome. A self-reflection pass, in which the model re-reads and revises its own output, instead reduced the flag rate to 3.5%, close to the ensemble’s 3.13%. The papers retained by self-reflection form a strict subset of those retained by the unified prompt, suggesting that the model applies a consistent discriminative criterion rather than simply adopting a stricter prompting frame. On this basis, we propose a transferable four-layer triangulation protocol combining statistical ensembles with complementary LLM regimes. The classical benchmark shows that no single pipeline dominates and we derive selection guidance accordingly.