StableID: An Iterative Graph-Based Framework for Persistent Entity Identification in Evolving Industrial Data Systems

Graph-based entity deduplication has proven effective for resolving duplicate records when identifying information is sparse and heterogeneous, yet in continuously evolving industrial data systems, it remains insufficient on its own. As new records and relationships are added incrementally, previously separate entity components can merge, causing instability and inconsistency in entity identifiers that undermine downstream analytics, auditing, and system integration, requiring persistent, interpretable identifiers over time. This work introduces StableID, an iterative graph-based framework for persistent entity identification in dynamic environments. StableID treats identifiers as long-lived system assets rather than transient outputs, incorporating historical grouping results into subsequent graph constructions through a feedback mechanism to ensure previously resolved entities retain consistent identifiers. When components merge, a deterministic dominance rule assigns the identifier from the largest prior component to the unified entity, minimizing churn, while time-scoped identifier generation with execution-level prefixes prevents collisions and guarantees global uniqueness during incremental updates. Implemented with distributed graph processing, StableID was evaluated through iterative executions on large-scale, multi-state voter registration data lacking global identifiers and featuring heterogeneous schemas. Results demonstrate strong identifier stability, progressive convergence in the number of entity identifiers, and a clear trend toward a stable identity state as relational connectivity increases. Overall, StableID transforms graph-based deduplication into a production-ready identity management solution suitable for continuously updating industrial data.