Efficient Distributed System Testing Using Lightweight Virtualization, Fault Injection, And YAML Scenarios

Performance, fault tolerance, and resilience testing of distributed systems are difficult as they add complexity and scalability concerns. Most of the current techniques use the cloud infrastructure along with automatic fault injection and XML-based scenarios, which are expensive and complicated to implement, requiring a lot of resources. The existing approaches are resource-intensive, dependent on cloud infrastructure, and complicated management of test scenarios. These problems ultimately prevent the cost-effectiveness and scalability in the testing of distributed systems. This paper describes a lightweight testing framework based on virtualization, containerization, fault injection, and YAML scenarios. This method is not cloud-dependent for reduced resources and simplified test scenario management using YAML, thereby achieving affordable, more efficient testing. This method is tested for throughput under normal conditions at 1200 requests per second and gets disturbed to 800 requests per second in the event of fault injections. The latency appears uniform at 150 Ms during normal conditions, it increases to 450 ms in the case of faults. Log analysis reveals important patterns in several types of failures, for example, network latency, node failures, etc., to improve precision in improvements. This framework obtains 98% of system availability and reduces the mean time to detect (MTTD) by 14 hours and the average time to repair (MTTR) to 8 hours. The proposed approach shows improved resource efficiency, cost-effectiveness, and ease of implementation compared with cloud-based techniques while providing similar performance metrics. This framework offers organizations with limited resources the opportunity to test distributed systems as highly scalable, lightweight, and cost-efficient solutions that will cut down dependence on cloud infrastructure. The plan includes integrating machine learning for adaptive fault injection and monitoring to achieve better scalability and performance.