Towards efficient large language models for scientific data: a review

Abstract The adaptation of large language models (LLMs) to scientific domains requires efficient strategies to mitigate high computational costs while preserving performance. We present a systematic survey of efficiency techniques for adapting LLMs to scientific applications. Our review analyzes 120 studies across seven scientific fields, including biomedicine, clinical medicine, biology, mathematics, chemistry, geo-environmental science, and cross-domain scientific research. Across the surveyed literature, we summarize 34 representative models and 30 scientific datasets for efficient scientific LLM research. Examining parameter-efficient fine-tuning, knowledge distillation, architectural optimization, and data-centric approaches, we introduce a dual-axis taxonomy that organizes efficiency techniques by both methodological strategy and scientific domain. We further provide practical guidance for selecting efficiency strategies under resource constraints and highlight open challenges including multimodal integration, catastrophic forgetting, and automated method selection. By synthesizing cross-domain evidence and outlining directions for sustainable model development, this survey supports the adoption of efficient scientific LLMs in resource-constrained research environments and promotes more accessible and responsible deployment across scientific disciplines.