Online Adaptively Deep Learning via Correlation-Driven Framework

Recent years have witnessed an enormous increase in online learning. However, there are two key challenges in this setting. One challenge is how to learn a favorable feature representation. Current models usually extract underlying features based solely on the data itself and neglect the correlations between features, which can directly influence the performance of the model. Another significant challenge is concept drift; that is, the probability distribution of the data may change as instances arrive sequentially in a stream fashion, which can lead to catastrophic forgetting. In this paper, we attempt to address the above open challenges of online learning by proposing an Online Adaptively Deep Learning framework based on Correlation-Driven (OADLCDF) methods. In our OADLCDF algorithm, the correlational loss module, which considers both data features and their correlations, can obtain a more favorable feature representation compared to traditional autoencoders. The drift detection mechanism monitors changes in the data probability distribution based on the error rate of the classifier. If concept drift is detected, the learner can adaptively adjust the model parameters with a low learning rate through tiny experience replay. This means that the model can effectively balance new and old knowledge, encouraging positive transfer while mitigating catastrophic forgetting. Finally, extensive experiments conducted on various datasets demonstrate that our OADLCDF algorithm is a promising approach, as evidenced by competitive numerical results.