Learning to Balance Utility and Delay in Bipartite Queueing Networks With Sample Path Constraints

Bipartite queueing networks with unknown statistics, where jobs are routed to and queued at servers and yield job-type and server-dependent utilities upon completion, model a wide range of problems in communications and related research areas (e.g., call routing in call centers, task assignment in crowdsourcing, job dispatching to cloud servers). Additionally, many such problems have additional routing constraints, such as quality of service or budgeted server cost constraints. The utility maximization problem in a bipartite queueing network with unknown statistics and subject to additional routing constraints is a constrained bandit learning problem with delayed feedback that depends on the server queueing delay. In this paper, we propose an efficient algorithm that overcomes the technical shortcomings of the state-of-the-art and effectively balances utility regret and peak job completion delay, while achieving constant peak constraint violation for the additional sample path routing constraints. Empirically, our algorithm is shown to simultaneously achieve low regret, peak delay, and peak constraint violation compared to existing algorithms.