Local recovery and partial snapshot in distributed stateful stream processing

Abstract In data stream applications, a query is created using a series of operators connected by unidirectional pipelines. Since those operators may be deployed in a shared-nothing distributed environment, it is necessary to have a valid restore point in case an operator fails. This ensures consistency of the states among operators after recovering from failures. Typical fault-tolerant approaches periodically capture global snapshots, encompassing all operator states. In the recovery process, all operators must be reinstated to the most recent global snapshot. This global dependency affects the recovery performance and the required computational costs for recovery. We propose fault-tolerant schemes for stateful stream processing that localizes recovery to a subset of operators. Additionally, we introduce a partial snapshot mechanism to capture the states required for local recovery efficiently. This snapshot records only a subset of operator states instead of global states. Our implemented recovery schemes in Apache Flink show improved recovery times by up to more than 50 percent.