Financial News Analysis Capability Assessment Standard: An End-to-End System Implementation

This paper presents the design and implementation of a systematic framework for evaluating Large Language Models' capabilities in financial news analysis. Our framework introduces a five-layer architecture that integrates data preprocessing, model analysis, feature integration, strategy generation, and backtesting validation. The system implements four key technical components: quality assessment for news relevance and reliability, impact analysis for market factors, multi-period price predictions, and sentiment evaluation. Through this integrated approach, we establish a market-based validation methodology that moves beyond traditional manual annotations. We evaluate our framework using financial news data from 14 major US-listed companies, testing models across different parameter scales (1-10B to 600B). Experimental results demonstrate that larger models consistently outperform smaller models across various evaluation dimensions, including prediction accuracy, trading returns, and stability of performance across different market conditions. The framework demonstrates consistency between evaluation metrics and trading performance across different model scales. Our implementation advances existing methods by providing a comprehensive, automated approach to assessing language models in financial applications, offering practical guidance for model selection in production environments.