L3: Latency-aware Load Balancing in Multi-Cluster Service Mesh

Microservice architectures and service meshes have become highly popular and face increasingly stringent scalability and dependability requirements. To achieve low-latency service execution and maximize performance, service providers of large-scale distributed systems deploy microservices geographically closer to their users in multi-cluster service mesh environments. However, inter-cluster service dependencies introduce additional latency, and effective load balancing across multiple replicas distributed across clusters is crucial. Addressing this challenge, we present L3, an adaptive latency-aware load-balancing mechanism for multi-cluster service meshes. We conduct extensive simulations on Amazon EC2, and our results of using the microservices of the DeathStarBench suite for three clusters show that L3 reduces the 99th percentile latency by 26% and 22% compared with round-robin and C3.