CommCanary 0.4v: Fail Closed Evidence for Decision Fidelity Evaluation of Collective Trace Replay
CommCanary is a fail closed evidence workflow for testing whether collective communication trace replay preserves the configuration decisions of a measured workload. The workflow freezes campaign identity and policy before execution, retains immutable attempts, applies an explicit terminal selection and completeness verdict, binds archived evidence by hash, and generates publication artifacts deterministically from the verified selection. This preprint compares four replay representations with a full workload across eight NCCL configurations on one node with four NVIDIA A100 GPUs. The trusted join contains all 280 selected cells. Communication only W-canary agrees with the full workload on 16 of 28 pairwise decisions, or 57.1%, with Kendall tau b = 0.204. W-micro agrees on 18 of 28 pairs, or 64.3%, with tau b = 0.490. W-shared-overlap is the strongest measured proxy, reaching 20 of 28 pairs, or 71.4%, with tau b = 0.708, but it still disagrees on eight decisions. A separately predeclared exact work gate reaches 26 of 28 pairwise decisions, or 92.86%, with tau b = 0.857, one false negative, and one false positive. Its median absolute relative error is 1.55%, and its p95 absolute relative error is 4.05%. Although every point criterion passes, the persisted verdict is inconclusive because bootstrap uncertainty crosses frozen decision boundaries and one configuration violates the 20% relative IQR stability limit. The record contains the nine page preprint and a complete source archive with LaTeX, BibTeX references, vector figures, a figure generation script, verified metrics, provenance information, and working notes. The evidence is bound to repository commit c133ad3fb5883100cb8622b597d5b6233b5efca1. The results are limited to the measured single node, four A100 system and the evaluated configuration set. They do not establish production qualification, reduced canary cost savings, general decision preservation, or multi-node generalization.
Authors
- Anshuman Agrawal
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-14
- DOI
- https://doi.org/10.5281/zenodo.21939041
- Primary Topic
- Scientific Computing and Data Management
- Type
- preprint