Predictive Batching for Generative AI: Forecasting Future Request Arrivals and Proactively Organizing Computational Workloads

The increasing deployment of generative artificial intelligence (AI) systems has created substantial demand for efficient GPU utilization, high-throughput inference, and low-latency request processing. Large language model (LLM) inference systems commonly employ dynamic batching, continuous batching, and iteration-level scheduling to combine multiple requests into computational batches. Although these techniques improve hardware utilization by coordinating requests that have already entered the inference system, their scheduling decisions remain substantially dependent on currently observable workloads. Consequently, inference systems may encounter periods in which requests arrive irregularly, compatible workloads are distributed across different arrival intervals, and available GPU capacity cannot be fully utilized by the requests immediately present in the scheduling queue. This paper introduces Predictive Batching for Generative AI (PB-GAI), a conceptual framework that investigates whether future request arrival patterns can be predicted and used to organize computational batches before the corresponding requests materialize. Rather than waiting exclusively for incoming requests to accumulate, PB-GAI estimates the arrival times, expected computational characteristics, and compatibility relationships of future workloads. It uses these predictions to construct provisional batch configurations, reserve appropriate execution opportunities, and dynamically organize arriving requests into computational groups that satisfy GPU resource constraints and service-level requirements. The proposed architecture integrates seven principal functions: (1) request arrival-time forecasting, (2) future workload characterization, (3) computational compatibility estimation, (4) predictive batch formation, (5) deadline-aware batch scheduling, (6) adaptive batch execution and reconstruction, and (7) prediction-error feedback and continuous optimization. A mathematical formulation is developed to characterize the expected benefits of predictive batch formation while accounting for arrival-time uncertainty, batch execution efficiency, GPU memory capacity, heterogeneous sequence lengths, waiting-time costs, deadline constraints, and the computational expenditure associated with incorrect predictions. The framework further distinguishes between predicting individual request arrivals, predicting the arrival of workload groups, and predicting the optimal execution time of a batch. It also establishes a complementary relationship with conventional continuous batching: predictive batching may prepare batch configurations and scheduling decisions in advance, while continuous batching remains responsible for dynamically incorporating actual requests during execution. Importantly, a predicted request is not treated as an actual request, and speculative batch formation does not require generating responses for requests that have not arrived. An experimental methodology is proposed to compare predictive batching with conventional request batching, continuous batching, fixed-window batching, and prediction-assisted scheduling. Evaluation metrics include batch utilization, GPU throughput, request waiting time, time to first token, tail latency, deadline satisfaction, prediction accuracy, GPU energy consumption, and total computational expenditure. The proposed framework does not assume that predictive batching is universally beneficial. Its effectiveness depends on the predictability of request arrivals, the compatibility of anticipated workloads, the availability of sufficient scheduling flexibility, and the ability to avoid unnecessary waiting or speculative resource reservation. By introducing arrival-time prediction as an additional input to batch formation, this research investigates a broader inference-serving paradigm in which generative AI systems attempt not only to optimize batches from requests that have already arrived, but also to anticipate and organize the computational structure of future workloads.