Collaborative Multitask Learning Framework Based on Constrained Bi-Level Gradient Optimization for Mechanical Fault Diagnosis

In recent years, multitask learning (MTL) has attracted the increasing focus in the field of mechanical fault diagnosis. Relevant research shows that the MTL-based fault diagnosis frameworks generally achieve more satisfactory performance than the single-task ones. The key to promoting the above MTL frameworks is how to assign the task weights to balance the model training among multiple tasks. However, this issue has not received sufficient attention. In these frameworks, the task weights are generally statically assigned by manual predefinition or simple grid search, which is thus unable to provide the real-time guidance for the model training. In this regard, this paper aims to integrate the multitask model parameters update and task weights assignment into a collaborative training procedure via a gradient-based approach. To this end, we propose a constrained bi-level gradient optimization (C-BLGO) algorithm. For each training epoch, C-BLGO can adaptively optimize the real-time task weights through the task-level and layer-level gradients calculated from the dynamically updated multitask model. In addition, C-BLGO can integrate prior task priorities to modify real-time task weights within constrained range, so as to make the task weights assignment more controllable and targeted. Subsequently, these assigned task weights further provide the real-time guidance for the update of the entire multitask model. The proposed framework makes the training dynamics well controlled and thus brings satisfactory performance gains. The experimental results on three public datasets and an electromechanical measurement system, demonstrate that our work can better improve the training stability and fault diagnosis accuracy compared to other mainstream MTL methods. Thanks to the model-independent characteristics, C-BLGO is scalable to various model architectures and thus is adapted to different fault diagnosis scenarios.