A New Design of High-Speed Data Collection Mechanism Based on Cloud Simulation Environment

Aiming at the demand of simulation data acquisition in a cloud simulation environment, a lightweight data acquisition system based on QEMU/KVM virtualization technology was designed to solve the problems of high resource occupation, low transmission efficiency, and difficult data management in the existing technology. To improve the efficiency of data acquisition, a shared memory communication architecture between multiple virtual machines and the host machine was established based on the Ivshmem virtual PCI device, avoiding excessive context switching and data copying; to solve the problem of data communication synchronization in different operating system environments, a synchronous communication algorithm based on interrupt signals was proposed, and a virtual machine driver algorithm was designed to receive and process interrupt signals; to solve the problem of data management, the system efficiently transmits the simulation data generated by multiple virtual machines to the host machine through a shared memory channel, simplifies the data management process, and enables the host machine to efficiently process simulation data with a unified strategy; to solve the problem of resource occupation of the acquisition system, the system adopts a single-threaded operation mode on both the host and virtual machine sides, providing more computing resources to the simulation task to ensure its smooth progress. Experimental results show that compared with traditional Virtio communication technology, the system has significantly improved data throughput when transmitting different data packet lengths. At the same time, in a multi-virtual machine environment, the system effectively reduces CPU resource usage by about 30%-40%, reduces the impact on simulation tasks, and achieves the design purpose.