Synchronous fusion framework of real-time video analysis data stream and text stream for substation report generation

Abstract The asynchronous arrival and heterogeneous semantics of surveillance video, sensor measurements, alarm messages, and operation logs make it difficult to reconstruct substation incidents and generate verifiable reports. This paper proposes an evidence-constrained synchronous fusion framework for real-time video and text streams. A unified event representation maps modality-specific observations to equipment, time, state, confidence, and source-evidence fields. Candidate events are then aligned by a sliding time window and canonical asset-ledger matching, converted to a common state-hypothesis space, and fused with calibrated confidence. Equipment-centered event chains are finally transformed into structured reports by deterministic rules and templates, followed by chronology, unit, conflict, and evidence-anchor validation. Experiments on 48 expert-verified incidents from a 110 kV substation yield an F1-score of 94.8 %, an average validated-report generation time of 18.7 s, and 100.0 % evidence traceability. The framework therefore supports timely incident reconstruction while retaining auditable links to original video frames, sensor records, alarm messages, and rule identifiers.