Pi-transformer: A prior-informed dual-attention model for multivariate time-series anomaly detection

Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer (Prior-Informed Transformer), a transformer with two attention pathways: data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior provides an amplitude-insensitive temporal reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct. The prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding prior attention into transformer scoring yields a calibrated and robust approach to anomaly detection in complex multivariate systems. • Prior-informed dual attention : Pi-Transformer couples data-driven series attention with a prior attention built from phase synchrony and scale invariance. • Two-stream, interpretable scoring : Inference combines a prior-aligned reconstruction energy stream with a mismatch stream that emphasises timing and phase disruptions, fused point-wise. • Complementary detection behaviour : The energy and mismatch streams provide sensitivity to both amplitude or shape deviations and to timing or phase-breaking anomalies. • Broad benchmark evidence with ablations : Results on five multivariate benchmarks (SMD, MSL, SMAP, SWaT, PSM) are consistently competitive in F1, and ablations quantify the contribution of the prior components and design choices.