DataWatts: An Accounting Framework for Resource-Aware Efficiency Measurement in Distributed Data Processing

Efficiency measurements of distributed data processing depend on what work is held constant, which resources are counted, and how shared capacity is attributed. DataWatts proposes an operational accounting contract that makes these choices explicit. It separates elapsed time from allocated, available, and engaged resource-time; preserves native resource semantics; defines additive measurements invariant to telemetry partitioning; and distinguishes directly observed quantities from modeled allocations. Efficiency is represented primarily through latency, logical-work productivity, and an accompanying resource vector, rather than a mandatory scalar score. An optional dimensionless reference-relative score expresses a declared latency–resource trade-off. The framework includes adapter specifications for BigQuery, Apache Spark, Databricks, Snowflake, Teradata Vantage/VantageCloud, and Oracle Database on OCI/Autonomous Database. A reference implementation and conformance suite verify the contract’s invariants—accounting identities, split/merge partition invariance, result equality, and attribution conservation—on forty original executions and a public Spark event log, with distributed-engine validation as the defined next step. The contribution is the measurement contract and its operational conformance tests.