Load Adaptive Spline Grids for KAN MoE

Mixture-of-Experts architecture with Kolmogorov-Arnold Network (KAN) experts, where each expert's spline grid resolution dynamically adapts based on its load, rather than using a fixed resolution across all experts. We also propose Fractional KAN-MoE, which uses a continuous, fractional grid resolution instead of discrete integer values, enabling smoother and more gradual capacity adjustments. The grid resolution for each expert is updated using a Newton-step scheduler with backtracking line search and exploration decay, this allows the model to allocate more resolution to experts handling complex input regions and less to experts handling simpler ones. Experiments on California Housing, UCI Superconductivity, and the large-scale HIGGS dataset show that this dynamic adaptive approach consistently achieves lower loss than static KAN-MoE baselines, escapes loss plateaus that static resolution cannot reach, and produces interpretable, emergent specialisation across experts, all validated across multiple random seeds.