RyBench: Runtime Verification Benchmarking Framework over Publish/Subscribe Networks
Autonomous systems built as distributed, node-based systems communicate via publish/subscribe middleware. Runtime Verification (RV) tools are used to monitor those systems against formal specifications, receiving monitored events through the pub/sub channel. An RV tool's practical value depends on its monitoring throughput capacity and verdict latency, besides its correctness, so a systematic benchmarking setup is needed to evaluate how efficiently these tools run. This paper presents RyBench, a pub/sub-based benchmarking framework that acts as a synthetic event publisher in place of a monitored system, providing a common benchmarking setup. RyBench separates benchmark generation, serialization, and transport into distinct components, enabling the combination of different event-generation strategies, data formats, and transport layers. We also evaluate the framework's throughput capacity across these configurations using Open Simulation Interface (OSI) schemas and Timescales properties, and illustrate the end-to-end benchmarking flow through an application to Reelay and an integration with MonPoly.
Authors
- Dogan Ulus (ORCID: https://orcid.org/0000-0002-5090-1769)
- Mehmet Hilmi Dündar
Institutions
- Boğaziçi University (TR)
Publication Details
- Journal
- Electronic Proceedings in Theoretical Computer Science
- Published
- 2026-11-17
- DOI
- https://doi.org/10.4204/eptcs.457.6
- Primary Topic
- Formal Methods in Verification
- Type
- article
- Field-Weighted Citation Impact
- 0.00