Improving Deep Optimisation for the Multi-dimensional Knapsack Problem using Elastic Weight Consolidation

Deep Optimisation is a recently developed Estimation of Distribution Algorithm that has shown notable efficiency in solving the Multi-Dimensional Knapsack Problem. However, since this optimiser relies on an auto-encoder that is sequentially trained, we posit it is susceptible to catastrophic forgetting, a phenomenon where previously learned knowledge is lost during successive trainings. This issue can diminish the optimiser's performance, motivating the present work. To address this challenge, we explore the application of Elastic Weight Consolidation (EWC), a deep learning technique that mitigates forgetting by associating weight variations during future training with their importance in predicting current tasks. EWC has previously demonstrated significant reductions in forgetting across various tasks and architectures. Our results show that applying EWC to Deep Optimisation significantly enhances performance on Multi-Dimensional Knapsack Problems. Furthermore, we analyse how these improvements are linked to the resilience of information retained within the auto-encoder, highlighting the critical role of preserving learned knowledge in sequential training scenarios.