Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation
We present Cloud Cost Intelligence, a production-ready autonomous cloud FinOps and telemetry system built around three engineering design lenses that are conventionally in tension: (1) high-frequency resource observability, (2) autonomous remediation without availability regressions, and (3) operation within a near-zero marginal monitoring cost envelope. Traditional cloud billing monitors rely on delayed daily billing exports and static global thresholds that fail under diurnal traffic shifts, leaving idle compute instances, unattached storage volumes, and runaway serverless functions unaddressed. Our central architectural decision is a decoupled telemetry and remediation spine: multi-service AWS resource utilization metrics and billing signals are ingested asynchronously via boto3 into a single PostgreSQL (TimescaleDB) instance partitioned as compressed hypertables; anomaly scoring runs in a stateless dual-stage machine learning worker combining Holt-Winters seasonal forecasting (on a 7-day rolling window) with multivariate Isolation Forest residual scoring; and infrastructure mutations are governed by a deterministic, safety-gated remediation state machine executing 40+ automated workflows across AWS EC2, CloudWatch, IAM, and Systems Manager (SSM). To prevent destructive false-positive terminations, adaptive per-instance dynamic thresholds (via Median Absolute Deviation) are paired with IAM tag-scoped blast-radius guards, pre-execution snapshot hooks, and real-time operator human-in-the-loop (HITL) overrides delivered via a live Dash/Plotly observability console and a low-latency WhatsApp webhook pipeline. We describe the system architecture, TimescaleDB data model, dual-stage anomaly detection mathematics, 6-state remediation lifecycle, least-privilege IAM posture, and zero-cost infrastructure envelope. Evaluated on multi-service cloud workloads within a rapid-prototyping and benchmarking cycle, the system achieves 94.2% anomaly detection accuracy, reduces manual FinOps oversight by 90%+, lowers net cloud compute overhead by 15%, and uncovers $500+/month in recurring infrastructure waste.
Authors
- S. Goel
- Samyukta Sreekanth
- Poojitha Madan
Institutions
- Manipal Academy of Higher Education (IN)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-24
- DOI
- https://doi.org/10.5281/zenodo.23622225
- Primary Topic
- Cloud Computing and Resource Management
- Type
- preprint