From data to decision: a multi-stage framework for class imbalance mitigation in optical network failure analysis

Machine learning-based failure management in optical networks has gained significant attention in recent years, but severe class imbalance, where normal instances far outnumber failure cases, remains a considerable challenge. While pre- and in-processing techniques have been widely studied, post-processing methods are largely unexplored. We present a direct comparison of pre-, in-, and post-processing approaches for class imbalance mitigation in failure detection and identification using an experimental dataset. For failure detection, post-processing, particularly threshold adjustment, yields the highest F1 score improvement of up to 15.3%, while random under-sampling offers the fastest inference. In failure identification, generative AI methods deliver the most significant performance gains up to 24.2%, whereas post-processing has a limited impact in multi-class settings. When class overlap exists and latency is critical, over-sampling methods like synthetic minority over-sampling technique (SMOTE) are most effective; without latency constraints, meta-learning excels. In low-overlap scenarios, generative AI approaches provide the best performance with minimal inference time.