Learning payoffs while routing in skill-based queues

Skill-based queues are used in, for example, data centers, service systems, and healthcare applications. In such systems, customers must be handled by servers with the right skill set. An illustration is in Figure 1. Maximizing the quality of service is essential for system performance. Canonical routing policies including First–Come–First– Served (FCFS) Assign–the–Longest–Idle–Server (ALIS) and Serve–the–Longest–Queue are typically static by design. Although these policies are easy to implement, they do not account for long-term performance tradeoffs or stability constraints. Moreover, they offer limited robustness and adaptability against changes in environment. Other policies such as the c? rule and MaxWeight scheduling rely on system parameters which might a priori be unknown in real-world applications. This limits their applicability. To address these challenges, there is growing interest in data-driven control policies that can deal with unknown system parameters.