A Survey of Theory Foundation and Key Technology in Large Models

The training and reasoning technology of large models continues to go deeper and more practical, and the model performance continues to hit new highs, driving the emergence of intelligent native applications and laying a solid technical foundation for new applications, new services, and new formats. This paper focuses on the technical capabilities and development directions of large models, introduces the theory foundation research of large models in detail, including scaling laws, intrinsic task subspace, low-rank decomposition optimization theory, effective model complexity theory and alignment theory foundation; it conducts in-depth research on key technology of large models such as the neural network architecture, chain of thought(CoT), multimodal fusion technology, retrieval-augmented generation(RAG) and large language model (LLM) based agent, sorts out the iteration of technology and innovative research results; it discusses technical challenges such as the professionalism, authenticity, controllability, and credibility of large models, pointing out the key technical points of innovative research; finally, it summarizes and looks forward to the high-quality development of large models.