Neural architecture search from a natural language processing perspective: a survey

Neural architecture search has revolutionized automated machine learning and demonstrated transformative potential across natural language processing domains. This survey presents a comprehensive task-oriented analysis of NAS applications in NLP, systematically categorizing research into three core areas of text representation and classification, sequence modeling and generation, and information extraction. We trace the field’s evolution through three distinct developmental phases from foundational methodology transfer and task-specific architectural customization to deployment-oriented optimization. The analysis examines key methodological advances in search space construction, search strategy optimization, and efficient evaluation mechanisms, while highlighting architectural innovations including pre-trained model compression, multimodal fusion, and specialized designs for machine translation, speech recognition, and entity extraction. Building on this comprehensive synthesis of current progress, we identify critical future research directions in large language model optimization, zero-cost evaluation methods, and knowledge-aware architecture design for practical NLP systems.