INGAD: Asynchronous acknowledgment channel decoupling for high-throughput data archiving in radio astronomy — E8 Intelligence Research

The Qitai Radio Telescope (QTT), a 110 m fully steerable dish under construction in China, will produce extremely large volumes of observational data that must be transferred efficiently and reliably from the observatory site to remote data center. The Next Generation Archive System (NGAS) is a widely used archival system in astronomy, but it suffers from stop-and-wait flow control, Global Interpreter Lock (GIL) bottlenecks, and a lack of backpressure, which limit its throughput and scalability for next-generation facilities. We present INGAD (Improved NGAS based on Asynchronous Acknowledgment Channel Decoupling), a new data transfer architecture that fully decouples the data and acknowledgment paths. The data channel uses ZeroMQ PUSH-PULL sockets for high-speed asynchronous transfer with inherent backpressure, while the acknowledgment channel employs a persistent RabbitMQ message queue to deliver ACKs independently without blocking the sender. A bounded in-flight sliding window limits Author: Andrew Stewart Caldin, Independent Researcher, UK. Part of the E8 Intelligence Research series. Platform: e8intelligence.com