A Leakage-Aware Evaluation Framework and Benchmark for Edge-Based Multi-Sensor Fire and Smoke Detection

Low-cost multi-sensor nodes are increasingly used to detect fire and smoke inside buildings, where separating flame from smouldering emission determines the response. Such detectors are usually validated on densely sampled logs whose consecutive readings are nearly identical. Earlier studies report near-perfect accuracy in this setting but seldom state whether the train–test split respected event boundaries; when it does not, the score measures memorisation rather than generalisation. This paper treats the evaluation itself as the object of study. A controlled single-room testbed with temperature, humidity, gas, and particulate sensing produced a labelled time-series dataset with exactly known event boundaries, so that leakage could be introduced or excluded deliberately. Six methods were evaluated on identical data under a random split and under an event-preserving split using the macro-averaged F1 score, which weights minority classes equally. The random split yielded markedly higher three-class scores for every supervised model, whereas hazard-versus-normal detection was largely unaffected. Holding out single events revealed strong between-event variability, while thinning records within events changed results only slightly: independent events, not records, are the binding constraint. The dataset, code, and a short leakage-aware protocol are released; the findings characterise the evaluation protocol under controlled conditions, not field performance.