Research on a Dynamic Multi-Dimensional Lightweight Evaluation Framework for Large Language Models

To address the limitations of strong static assumptions, single-dimensional evaluation, and insufficient engineering flexibility in current evaluation methods for large language models (LLMs), this paper proposes a dynamic multidimensional evaluation framework, which includes adversarial sample generation and temporal drift techniques to simulate real-world distribution shifts. To comprehensively measure model performance under resource constraints, we introduce a novel metric, PCR_lite, which integrates accuracy, inference latency, and VRAM usage into a unified efficiency score. Experiments were conducted on mainstream LLMs including GPT-4, Claude 3 Opus, LLaMA3-70B, and DeepSeek-70B, evaluated under both static benchmarks (HELM and Big-Bench) and the proposed dynamic evaluation scenarios. Results show that all models experience a 5–8% accuracy degradation under distributional shifts, revealing latent vulnerabilities not captured by static benchmarks. Moreover, PCR_lite demonstrates improved sensitivity to inference efficiency, ranking models more accurately for deployment readiness—DeepSeek-70B, for instance, achieves the highest score due to low latency and memory consumption. Compared to traditional evaluation methods, the proposed framework reduces evaluation time and memory usage by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\sim 30 \%$</tex>, while preserving comparable task coverage. These findings suggest that our framework offers a more realistic and resource-aware evaluation paradigm for the development and deployment of next-generation LLMs.