Distributed Frank-Wolfe Algorithm for Constrained Bilevel Optimization

Bilevel optimization has attracted substantial attentions in recent years due to its wide applications in machine learning. However, most existing algorithms either primarily developed under centralized setting or suffer expensive inner-loop updates for hypergradient estimation. What's worse, the projection operator in constrained scenario may demand prohibitively computational cost, which further necessitates efficient projection-free bilevel optimization algorithms over networks. To fill this gap, we propose a novel single-loop distributed Frank-Wolfe algorithm DBO-FW for constrained bilevel optimization problems by simultaneously leveraging a nested approximation technique and a gradient tracking mechanism to locally estimate the global hypergradient. Moreover, we provide the convergence guarantee for the proposed DBO-FW. Numerical results also validate the efficiency of our algorithm.