A survey on deep learning fundamentals

Deep learning, driven by big data and graphic processing units, has garnered significant attention across various domains. The flexibility of network architectures, combined with their diverse components, has allowed deep learning techniques to be applied across a wide range of domains, expanding from low- and high-level computer vision tasks to encompass video processing, natural language processing (NLP), and 3D data processing. However, there has been relatively little effort to systematically summarise these works from principles to applications in terms of deep learning fundamentals. The present study aims to address this gap in the literature by presenting components of deep networks for image applications, and describing several classical deep networks for image applications. The study then introduces principles, relations, ranges, and applications of deep networks across an expanded scope, covering low-level vision tasks, high-level vision tasks, video processing, NLP, and 3D data processing. The study then compares the performance of different networks across these diverse tasks. Finally, it summarises potential focuses and challenges of deep learning research for these applications with concluding remarks.