Fönsterbaserade CRDT:er för decentraliserad aggregering i distribuerad strömbehandling : Holon: Konfliktfri aggregation för streaming i skala

Performing global aggregations over large data streams introduce significant bottlenecks, limiting overall system throughput and increasing latency. Current systems typically rely on centralized processing or statically defined tree-like structures to distribute aggregation workloads among nodes. Both approaches suffer from limited parallelism and tightly coupled aggregation logic, resulting in reduced performance even for simple queries. Using a shared global state may help decouple aggregation responsibilities. But maintaining strong consistency and synchronization introduces blocking operations that are incompatible with real-time streaming requirements. This thesis presents a novel approach to maintain global shared state in stream processing systems using Conflict-Free Replicated Data Types (CRDTs). CRDTs provide more relaxed consistency guarantees, enabling decentralized, coordination-free state convergence. This project designs and implements a windowed CRDT state that efficiently processes diverse workloads and supports exactly-once processing semantics. The system also allows users to define custom queries with minimal templating, improving flexibility without sacrificing performance. The system is evaluated against Apache Flink using the Nexmark benchmark suite, measuring end-to-end latency and throughput across workloads and cluster sizes of up to 100 nodes. The results demonstrate sub-2-second latency and significantly higher throughput, even at scale. These findings highlight the potential of CRDT-based shared state for improving global aggregation in decentralized stream processing, offering robust consistency with minimal coordination overhead.