Auditable Private LLM Serving on Kubernetes: A Vendor-Neutral Architecture and Evidence Model for Regulated Environments

Regulated organisations increasingly want to run large language models on infrastructure they control. Self-hosting a model, however, is not the same as being able to prove how it was served. An auditor working under the EU AI Act, the Digital Operational Resilience Act (DORA), or the GDPR asks narrower questions than "is the model private": which identity called which model, over what data, with what provenance, and where is the record that says so. We present a vendor-neutral reference architecture for private LLM serving on Kubernetes, together with an evidence model that turns these regulatory expectations into controls a machine can check. The architecture composes an inference gateway, GitOps delivery, and policy-as-code so that three controls, namely request lineage, access and model governance, and model provenance, each emit evidence a machine can check, including a redacted, queryable, tamper-evident audit trail. The evidence model regenerates a complete evidence pack with a single command and fails closed when the proof is missing or stale. We evaluate the design in three ways: a threat-to-control-to-evidence analysis, conformance checks that confirm forbidden actions are blocked and that a single altered audit record is detected, and a measurement of the runtime cost of the governance layer. On commodity CPU hardware the full governance path adds about 28 microseconds of processing per request, a sub-percent fraction of end-to-end latency once inference is in the path. In this setting the cost of making compliance provable is small and bounded.