SLMRouter: Reliable Tool Calling for Small Local Language Models

Small language models with seven to eight billion parameters can now run on ordinary laptops and consumer GPUs,which keeps data private and removes cloud costs. However, the agent frameworks that connect language models to program functions are designed for large cloud models, and they fail on small local models through long prompts, malformed JSON output and retry loops that never end. SLMRouter is a zero-dependency Python library that makes tool calling dependable on such models by treating the language model as a deterministic router rather than a reasoner. Each user query becomes exactly one bounded decision: call one registered Python function with arguments, or return a short final answer. The library builds a compact prompt of about one hundred tokens from Python type hints, queries a local model served by Ollama or llama.cpp at temperature zero, and parses a single-line reply with a quote-aware tolerant parser. Arguments are strictly validated in code, only allowlisted functionscan run, and at most two short repair prompts are allowed, so no query can use more than three model calls. On a 50-query live benchmark, the system selected the correct tool and arguments for 50 of 50 queries on both Llama 3 8B and Mistral 7B, and it produced zero false executions across 795 fault-injection test cases. The library is released as open-source software on the Python Package Index under the Apache 2.0 license.