Optical vortices in communication systems: mode (de)modulation, processing, and transmission

Optical vortices, characterized by their infinite orthogonal eigenmodes—such as orbital angular momentum (OAM) and cylindrical vector beam (CVB) modes—offer unprecedented opportunities for advancing optical communication systems. The core components of these systems—mode (de)modulation, mode processing, and mode transmission—are fundamental to the construction and networking of OAM/CVB mode-based communication networks. They significantly influence signal encoding, enhance channel capacity, and facilitate signal interconnection and transmission. We explore the historical development and recent advancements in optical vortex-based communication systems from these three critical perspectives. We systematically summarize the normative definitions and research progress related to key concepts such as mode multiplexing and routing. We also demonstrate the performance of these systems in terms of communication capacity, bit error rate, and more. Furthermore, we examine the substantial challenges and future prospects in this field, with the aim of offering cutting-edge insights that will facilitate the advancement and practical implementation of optical communication networks leveraging optical vortex modes.