Models and methods for optimizing data processing in a multi-server environment using operation queues and chunking data authentication

This work is motivated by the need to raise the rate of computation at present day industrial usage, including but not limited to dust flow simulation. Since these tasks remain as complex as they are and are getting more complex, the necessity of optimizing the usage of the multi-server system’s resources becomes evident, thus the need for new approaches towards the improvement of the data processing. The goal of this work is to develop an information technology which can be applied in more complicated systems where there can be more than one server and the focus is made on operation queues. The solution offered in the work is directed to the optimization of computational loads distribution over the servers, and to the growth of modeling accuracy in relation to the time necessary for calculations. This improves computation efficiency especially when working with large datasets. Another key focus is the development of a chunking model and method aimed at enhancing data authentication and integrity for distributed operations. This prevents the potential IP spoofing and man-in-the-middle (MITM) attacks when data is transmitted within the same local area network, ensuring the quality and consistency of distributed computational systems in a multi-server environment. Experimental studies have confirmed the effectiveness of the proposed solution: it cuts down to a very great degree the time required to do the computation and at the same time enhances the feasibility of the modeling projection. The management of the queue of operations, parallel data processing, and chunking authentication constitute the system architecture which was developed and shown high effectiveness in real conditions. A new solution could be used to enable the optimization of computer modeling processes, scientific calculations and audio/video conversion as well make solving many other problems that ensure scalability in conditions of data growth and complication tasks.