Trend-Aware Predictive Generative Inference Caching: The Third Experimental Study on Repetition-Aware Cost Optimization, Independent Quality Evaluation, and Economically Guided Predictive Computation

The increasing use of large language models (LLMs) creates substantial computational demand, particularly when similar or identical questions are submitted repeatedly. Conventional inference systems generally process incoming requests reactively, even when a substantial proportion of future requests may be predictable. Trend-Aware Predictive Generative Inference Caching (TPGIC) addresses this challenge by identifying potentially recurring questions, preparing reusable responses before subsequent requests arrive, and evaluating whether predictive preparation can reduce the computational cost of repeated inference without degrading response quality. This paper presents the third experimental study of TPGIC. The experiment evaluates three generation models—Gemma 3 1B, Gemma 3 4B, and Qwen3 4B—under two repetition scenarios involving three and ten requests per question. Twenty questions were used as the experimental benchmark. These questions were generated using ChatGPT-6 mini in response to a prompt requesting twenty specific, independently answerable questions that people frequently ask ChatGPT across general knowledge, everyday life, education, writing, technology, programming, health, career development, and personal productivity. ChatGPT-6 mini was subsequently prompted to estimate the monthly global frequency of these questions using semantic-equivalent matching. The resulting estimates were used as exploratory demand-prediction inputs rather than verified usage statistics. A cache capacity of twenty entries was employed, and Baseline, Demand_Cache, and TPGIC were compared using an independent evaluator, tpgic-qwen3-1.7b. The completed experiment recorded 2,340 user requests, 120 independent evaluator calls, 360 quality records, 144 statistical test records, and 18 cost records. The cost results show consistent patterns across all three generation models. In the three-repetition scenario, normalized computational cost decreased from 60 units under Baseline to 20 units under Demand_Cache and 6 units under TPGIC. This corresponds to a modeled cost reduction of 90% for TPGIC relative to Baseline. In the ten-repetition scenario, normalized cost decreased from 200 units to 6 units under TPGIC, corresponding to a modeled reduction of 97%. Importantly, TPGIC and Demand_Cache generated the same number of responses in each corresponding scenario. The additional reduction therefore arises from the economic cost coefficient assigned to predictive preparation, rather than from a further reduction in actual generation calls. Independent quality evaluation produced consistently high scores across all three methods. Overall mean scores ranged from 9.40 to 9.70, and the reported pairwise statistical tests did not identify significant differences after adjustment for multiple comparisons. These findings provide experimental evidence that the evaluated caching strategies can preserve the observed quality level in this controlled repeated-question setting while offering different modeled cost profiles. The study also clarifies the economic interpretation of the 0.3 coefficient and identifies a broader research direction in which future-question prediction is combined with GPU workload forecasting, off-peak precomputation, dynamic electricity-price awareness, low-cost cloud GPU selection, and predictive task scheduling.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-06
DOI
https://doi.org/10.5281/zenodo.24010882
Primary Topic
Natural Language Processing Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Trend-Aware Predictive Generative Inference Caching: The Third Experimental Study on Repetition-Aware Cost Optimization, Independent Quality Evaluation, and Economically Guided Predictive Computation

Jincheng Zhang
10 links
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
preprint

Trend-Aware Predictive Generative Inference Caching: The Third Experimental Study on Repetition-Aware Cost Optimization, Independent Quality Evaluation, and Economically Guided Predictive Computation

Jincheng Zhang
preprint en

Abstract

The increasing use of large language models (LLMs) creates substantial computational demand, particularly when similar or identical questions are submitted repeatedly. Conventional inference systems generally process incoming requests reactively, even when a substantial proportion of future requests may be predictable. Trend-Aware Predictive Generative Inference Caching (TPGIC) addresses this challenge by identifying potentially recurring questions, preparing reusable responses before subsequent requests arrive, and evaluating whether predictive preparation can reduce the computational cost of repeated inference without degrading response quality. This paper presents the third experimental study of TPGIC. The experiment evaluates three generation models—Gemma 3 1B, Gemma 3 4B, and Qwen3 4B—under two repetition scenarios involving three and ten requests per question. Twenty questions were used as the experimental benchmark. These questions were generated using ChatGPT-6 mini in response to a prompt requesting twenty specific, independently answerable questions that people frequently ask ChatGPT across general knowledge, everyday life, education, writing, technology, programming, health, career development, and personal productivity. ChatGPT-6 mini was subsequently prompted to estimate the monthly global frequency of these questions using semantic-equivalent matching. The resulting estimates were used as exploratory demand-prediction inputs rather than verified usage statistics. A cache capacity of twenty entries was employed, and Baseline, Demand_Cache, and TPGIC were compared using an independent evaluator, tpgic-qwen3-1.7b. The completed experiment recorded 2,340 user requests, 120 independent evaluator calls, 360 quality records, 144 statistical test records, and 18 cost records. The cost results show consistent patterns across all three generation models. In the three-repetition scenario, normalized computational cost decreased from 60 units under Baseline to 20 units under Demand_Cache and 6 units under TPGIC. This corresponds to a modeled cost reduction of 90% for TPGIC relative to Baseline. In the ten-repetition scenario, normalized cost decreased from 200 units to 6 units under TPGIC, corresponding to a modeled reduction of 97%. Importantly, TPGIC and Demand_Cache generated the same number of responses in each corresponding scenario. The additional reduction therefore arises from the economic cost coefficient assigned to predictive preparation, rather than from a further reduction in actual generation calls. Independent quality evaluation produced consistently high scores across all three methods. Overall mean scores ranged from 9.40 to 9.70, and the reported pairwise statistical tests did not identify significant differences after adjustment for multiple comparisons. These findings provide experimental evidence that the evaluated caching strategies can preserve the observed quality level in this controlled repeated-question setting while offering different modeled cost profiles. The study also clarifies the economic interpretation of the 0.3 coefficient and identifies a broader research direction in which future-question prediction is combined with GPU workload forecasting, off-peak precomputation, dynamic electricity-price awareness, low-cost cloud GPU selection, and predictive task scheduling.

Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon