Glim: an on-device assistant with offline knowledge, and what our first tests caught

We describe Glim, a private assistant that runs entirely on a consumer laptop (a fanless Apple MacBook Air with 16 GB of unified memory) using a 4-billion-parameter open model at 4-bit precision. Glim answers from offline sources: a Wikipedia subset, a curated emergency library (first aid, water, food, repair), a world gazetteer, and, with the user's permission, their own notes, mail and calendar on the device. We report latency and memory measurements, describe the retrieval and safety mechanisms, and present the results of a small qualitative evaluation of 36 probes across facts, honesty, privacy, safety, language, arithmetic, survival knowledge and tone. The evaluation exposed concrete failures (a fabricated memory of a previous conversation, an unverifiable helpline number, an incorrect chemical dosage, retrieval of irrelevant pages, and a model that blended a forum poster's situation into the user's answer). We describe the fixes, separating those that can be trusted to the model's instructions from those that must be enforced in code, and test a prompt-injection defence against an adversarial email. Our main lesson is that a small on-device model becomes useful through grounding and guardrails, and that rules about numbers and safety cannot be left to the model alone.