From fault logs to insights: TF–IDF vectorization and cluster-based behavioral analysis of railway vehicles

Railway vehicles generate extensive diagnostic messages through onboard monitoring systems, reflecting diverse technical and operational conditions. Their irregular structure and frequency complicate systematic analysis, and manual inspection or filtering is often insufficient for long-term or fleet-wide evaluation. This study presents a general framework for analyzing diagnostic logs across vehicles and over time. Weekly fault log data for each vehicle are represented using term frequency–inverse document frequency (TF–IDF) vectors, a technique adapted from text mining. This approach captures both local and fleet-wide patterns, producing a structured representation of vehicle behavior over time. Subsequent clustering of these representations identifies recurring operational states and enables comparison of vehicles and periods. Analysis of transitions between states provides insights into behavioral dynamics, recurring patterns, and deviations. Quantitative measures allow structured evaluation of state distributions and transitions across the fleet. The framework was applied to seven years of data from 28 vehicles, covering over a thousand unique fault types. Results facilitate systematic monitoring, comparison of operational behavior across vehicles, and identification of patterns relevant for maintenance planning. The approach offers interpretable, data-driven insights into vehicle diagnostics, supporting informed operational decisions and long-term maintenance strategies.