Semantic Unification Above Runtime Heterogeneity-A Lightweight Serverless Inference Hub forCommunity-Contributed Accelerators

Community-contributed accelerators are increasingly attractive for cost-efficient inference,but they violate the assumptions behind both cluster-centric serverless systems and open decentralized compute marketplaces. Nodes may be weakly managed, geographically distributed,intermittently reachable, and backed by heterogeneous runtimes such as CUDA, ROCm, oneAPI,CANN, or TPU-specific serving stacks. Cluster-first serverless platforms contribute strongscheduling and cold-start mechanisms, yet they assume stable orchestration substrates. Decentralized compute markets contribute open participation and economic coordination, yet theydo not directly solve inference-specific SLO routing, capability verification, or lease-aware servicesemantics.This paper presents Community Inference Hub (CIH), a design for a lightweight serverlessinference hub for community-contributed heterogeneous accelerators. The core thesis is thata practical platform for this setting should unify service-layer semantics, capability modeling,leasing, metering, and governance, while keeping device and runtime differences below a backend adapter boundary. We formalize the system model, define a permissioned consortium-chaingovernance layer for contributor admission, escrow, settlement, penalties, and disputes, andintegrate trust, verification evidence, and relay penalties into placement. The architecture combines a self-developed control plane with lightweight node agents, explicit capability declarations,lease-driven scheduling, and a split terminal model: Host-Direct nodes are the default execution mode, while K3s-backed nodes remain an optional node-local orchestration path. Finally,we provide representative execution flows, failure scenarios, decentralized-platform comparisons,and a visible prototype-oriented validation methodology. The resulting CIH design offers apractical boundary for community serverless inference: unify semantics and governance above,preserve runtime heterogeneity below.