Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation

We present Cloud Cost Intelligence, a production-ready autonomous cloud FinOps and telemetry system built around three engineering design lenses that are conventionally in tension: (1) high-frequency resource observability, (2) autonomous remediation without availability regressions, and (3) operation within a near-zero marginal monitoring cost envelope. Traditional cloud billing monitors rely on delayed daily billing exports and static global thresholds that fail under diurnal traffic shifts, leaving idle compute instances, unattached storage volumes, and runaway serverless functions unaddressed. Our central architectural decision is a decoupled telemetry and remediation spine: multi-service AWS resource utilization metrics and billing signals are ingested asynchronously via boto3 into a single PostgreSQL (TimescaleDB) instance partitioned as compressed hypertables; anomaly scoring runs in a stateless dual-stage machine learning worker combining Holt-Winters seasonal forecasting (on a 7-day rolling window) with multivariate Isolation Forest residual scoring; and infrastructure mutations are governed by a deterministic, safety-gated remediation state machine executing 40+ automated workflows across AWS EC2, CloudWatch, IAM, and Systems Manager (SSM). To prevent destructive false-positive terminations, adaptive per-instance dynamic thresholds (via Median Absolute Deviation) are paired with IAM tag-scoped blast-radius guards, pre-execution snapshot hooks, and real-time operator human-in-the-loop (HITL) overrides delivered via a live Dash/Plotly observability console and a low-latency WhatsApp webhook pipeline. We describe the system architecture, TimescaleDB data model, dual-stage anomaly detection mathematics, 6-state remediation lifecycle, least-privilege IAM posture, and zero-cost infrastructure envelope. Evaluated on multi-service cloud workloads within a rapid-prototyping and benchmarking cycle, the system achieves 94.2% anomaly detection accuracy, reduces manual FinOps oversight by 90%+, lowers net cloud compute overhead by 15%, and uncovers $500+/month in recurring infrastructure waste.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-24
DOI
https://doi.org/10.5281/zenodo.23622225
Primary Topic
Cloud Computing and Resource Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation

S. Goel, Samyukta Sreekanth, Poojitha Madan
Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Resource Management
preprint

Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation

S. Goel, Samyukta Sreekanth, Poojitha Madan
preprint en

Abstract

We present Cloud Cost Intelligence, a production-ready autonomous cloud FinOps and telemetry system built around three engineering design lenses that are conventionally in tension: (1) high-frequency resource observability, (2) autonomous remediation without availability regressions, and (3) operation within a near-zero marginal monitoring cost envelope. Traditional cloud billing monitors rely on delayed daily billing exports and static global thresholds that fail under diurnal traffic shifts, leaving idle compute instances, unattached storage volumes, and runaway serverless functions unaddressed. Our central architectural decision is a decoupled telemetry and remediation spine: multi-service AWS resource utilization metrics and billing signals are ingested asynchronously via boto3 into a single PostgreSQL (TimescaleDB) instance partitioned as compressed hypertables; anomaly scoring runs in a stateless dual-stage machine learning worker combining Holt-Winters seasonal forecasting (on a 7-day rolling window) with multivariate Isolation Forest residual scoring; and infrastructure mutations are governed by a deterministic, safety-gated remediation state machine executing 40+ automated workflows across AWS EC2, CloudWatch, IAM, and Systems Manager (SSM). To prevent destructive false-positive terminations, adaptive per-instance dynamic thresholds (via Median Absolute Deviation) are paired with IAM tag-scoped blast-radius guards, pre-execution snapshot hooks, and real-time operator human-in-the-loop (HITL) overrides delivered via a live Dash/Plotly observability console and a low-latency WhatsApp webhook pipeline. We describe the system architecture, TimescaleDB data model, dual-stage anomaly detection mathematics, 6-state remediation lifecycle, least-privilege IAM posture, and zero-cost infrastructure envelope. Evaluated on multi-service cloud workloads within a rapid-prototyping and benchmarking cycle, the system achieves 94.2% anomaly detection accuracy, reduces manual FinOps oversight by 90%+, lowers net cloud compute overhead by 15%, and uncovers $500+/month in recurring infrastructure waste.

Zenodo (CERN European Organization for Nuclear Research)
Manipal Academy of Higher Education (IN)
Cloud Computing and Resource Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon