Improving Event Aggregation in Automation of Software Development Workflows

Modern CI/CD systems produce vast streams of events that reflect the state and progression of software development processes. Aggregating these events is essential for enabling automation and traceability. In this context, aggregation refers to the process of detecting and grouping sequences of events into higher-level constructs based on predefined rules. This thesis investigates alternative mechanisms for event aggregation, with a focus on graph-based approaches within the Eiffel framework. By representing events and their relationships as a property graph, our proposed system enables declarative specification of patterns and supports incremental detection using trigger-based mechanisms in graph databases. The work implements and evaluates multiple aggregation systems, including graph databases and a complex event processing engine, and compares them based on throughput and resource efficiency under high workloads, using historical Eiffel data. The evaluation includes three different potential event aggregation systems: Neo4j, Memgraph, and Apache Flink. The results show that both Memgraph and Flink exhibit significant limitations in terms of performance and functionality for the use case, and are ultimately deemed unsuitable. Neo4j, on the other hand, provide a more promising foundation for implementing graph-based pattern matching. The findings show that Neo4j enables flexible event aggregation whilst simplifying rule definition. However, performance degrades significantly when the number of active triggers increase. Addressing this scalability limitation remains an open challenge.