Achilles' SPEar: Using Metamorphic Testing to Find Bugs in Stream Processing Engines

Stream Processing Engines (SPEs) are critical for real-time data processing, relying on aggressive optimizations to meet performance demands. Ensuring the reliability of such systems requires robust testing, yet testing remains costly and challenging due to the oracle problem. This paper investigates adapting query partitioning, a metamorphic testing technique, to the domain of SPEs. We present Achilles, an automated testing framework for SPEs. Achilles utilizes query partitioning to automatically generate, execute, and evaluate diverse test cases, reducing manual effort and targeting stream operators like filters and windowed aggregations. Our evaluation highlights Achilles' ability to detect unique bugs, including division-by-zero errors and predicate evaluation flaws. We further analyze framework parameters and demonstrate the effectiveness of predicate-based query partitioning, finding that increasing predicate depth boosts bug detection by 10%.