Addressing Coupling in Restless Multi-Armed Bandits by Finetuning Whittle Index

Restless multi-armed bandit (RMAB) is a popular model for budget-constrained scheduling problems, where the Whittle index policy could effectively solve it. As the requirements of arm separability limit the application of RMAB in real-world scenarios involving coupling components, recent studies have further extended the RMAB formulation to enable the coupling of arms. However, existing methods in this field require searching algorithms or domain knowledge, which loses the effectiveness and the universality of the Whittle index policy. This paper presents an extension of the Whittle index policy to coupled RMAB problems. Noticing that the pre-trained Whittle index is an informative arm feature and induces a suboptimal policy, we leverage it as a base policy to minimize the joint training process and thus mitigate the complexity of joint optimization. The proposed algorithm does not rely on specific problem properties or domain knowledge and has the potential to be applied in various domains. Experimental results demonstrate that the proposed algorithm is effective and efficient in learning a satisfactory policy.