Exactly-Once Delivered Zero Duplicates at 0.634 Times the Throughput: A Pre-registered Fault-Injection Study of Kafka Delivery Semantics on a Single Node

Kafka's delivery semantics are usually quoted from documentation: at-least-once may duplicate, exactly-once should not, and an acknowledged write on a healthy log should survive. This study counted them instead. Under a pre-registration locked before any measured run, a three-stage pipeline (source, processor, verifier) moved 20,000 sequenced 256-byte messages through a single-partition topic pair on a single-node Apache Kafka 3.9.0 broker, in two configurations: A (at-least-once: acks = all, no idempotence, auto-committing consumer) and B (exactly-once: idempotent source, read-committed consumer, transactional processor committing output writes and offsets together in batches of at most 100). Four scenarios (baseline, consumer kill, broker kill, rebalance kill) ran 3 times each, 24 runs, with correctness scored at the output topic from sequence numbers alone. All three frozen hypotheses were CONFIRMED. H1: configuration B produced zero duplicates and zero losses in all 9 of its fault runs, while configuration A produced duplicates in all 3 of its consumer-kill runs (5,141, 8,805 and 5,349 duplicate records out of 20,000, a quarter to nearly a half of the stream). H2: exactly-once costs throughput, median baseline pipeline throughput 3,337.8 messages per second for B against 5,264.5 for A, a ratio of 0.634, below the frozen 0.80 line. H3: no acknowledged message was lost in any of the 6 broker-kill runs, with median broker recovery of 22,406.5 ms; the claim's scope is one node and one disk, stated in the registration itself. Two cells are degraded and the paper prices them openly: the configuration A rebalance kill never landed (the stream finished inside the frozen 1.0 s window in all 3 runs), and the configuration B broker kill landed after its fast source had finished, so it stress-tested the transactional processor mid-stream rather than the source. The trade the documentation describes is real, and its measured terms on this machine are: all of the duplicates or none of them, at about 63 percent of the throughput.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24328164
Primary Topic
Distributed systems and fault tolerance
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Exactly-Once Delivered Zero Duplicates at 0.634 Times the Throughput: A Pre-registered Fault-Injection Study of Kafka Delivery Semantics on a Single Node

Jesun Ahmad Ushno
3 links
Zenodo (CERN European Organization for Nuclear Research)
Distributed systems and fault tolerance
preprint

Exactly-Once Delivered Zero Duplicates at 0.634 Times the Throughput: A Pre-registered Fault-Injection Study of Kafka Delivery Semantics on a Single Node

Jesun Ahmad Ushno
preprint en

Abstract

Kafka's delivery semantics are usually quoted from documentation: at-least-once may duplicate, exactly-once should not, and an acknowledged write on a healthy log should survive. This study counted them instead. Under a pre-registration locked before any measured run, a three-stage pipeline (source, processor, verifier) moved 20,000 sequenced 256-byte messages through a single-partition topic pair on a single-node Apache Kafka 3.9.0 broker, in two configurations: A (at-least-once: acks = all, no idempotence, auto-committing consumer) and B (exactly-once: idempotent source, read-committed consumer, transactional processor committing output writes and offsets together in batches of at most 100). Four scenarios (baseline, consumer kill, broker kill, rebalance kill) ran 3 times each, 24 runs, with correctness scored at the output topic from sequence numbers alone. All three frozen hypotheses were CONFIRMED. H1: configuration B produced zero duplicates and zero losses in all 9 of its fault runs, while configuration A produced duplicates in all 3 of its consumer-kill runs (5,141, 8,805 and 5,349 duplicate records out of 20,000, a quarter to nearly a half of the stream). H2: exactly-once costs throughput, median baseline pipeline throughput 3,337.8 messages per second for B against 5,264.5 for A, a ratio of 0.634, below the frozen 0.80 line. H3: no acknowledged message was lost in any of the 6 broker-kill runs, with median broker recovery of 22,406.5 ms; the claim's scope is one node and one disk, stated in the registration itself. Two cells are degraded and the paper prices them openly: the configuration A rebalance kill never landed (the stream finished inside the frozen 1.0 s window in all 3 runs), and the configuration B broker kill landed after its fast source had finished, so it stress-tested the transactional processor mid-stream rather than the source. The trade the documentation describes is real, and its measured terms on this machine are: all of the duplicates or none of them, at about 63 percent of the throughput.

Zenodo (CERN European Organization for Nuclear Research)
Distributed systems and fault tolerance
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon