LEVERAGING ISTIO FOR ADVANCED TRAFFIC MANAGEMENT AND SECURITY IN GENERATIVE AI APPLICATIONS ON KUBERNETES CLUSTER

The number of domains have begun to include AI as a result of its fast development; one such domain is AI Generated Content (AIGC), where Large Language Models (LLMs) have greatly improved capabilities. On the other hand, AI systems' vulnerabilities have been brought to light by their complexity, therefore reliable and resilient systems require strong methods for failure analysis (FA) and fault injection (FI). There hasn't been a thorough evaluation of FA and FI procedures in AI systems, even though these techniques are important. The use of containers has the potential to standardize and fine-tune resource management as infrastructures move from monolithic to microservices. Thanks to microservices, we can now use a single machine as if it were numerous machines. By letting programs adjust the number of computers according to demand, this reduced resource waste. The best way to route traffic to services that are held in a hybrid cloud, on-premises, or on several cloud deployments is by employing a service mesh, particularly in microservices environments. To deal with these kinds of situations, the idea is to use a service mesh. Improving Kubernetes' performance and safety is the main goal of service mesh. Istio is one application that runs on this service mesh idea. Even though Kubernetes offers Ingress, we still require Istio, and we explain in present Research paper.