QuasiPersona: behavioral release testing of LLM assistants on known-truth material from a business domain — method and a worked example

Large language model (LLM) assistants are released into customer service as systems: a model wrapped in instructions, retrieval, tools and guardrails. Most behavioral evaluation, including work on LLM "personality", measures the model alone. We describe QuasiPersona, a release-testing method for the deployed system, built on the client's own reference material so that every test item has a known correct answer. It combines (i) a completeness matrix of 6 architectural layers x 10 deviation guide words borrowed from hazard and operability studies (HAZOP); (ii) a behavioral personality test with four validity scales and fourteen traits, scored on what the system does; (iii) named failure characters and a character verdict that enters the release judgment next to per-class error rates; (iv) a therapy derived from the coding criteria, which rewrites the instructions and plans interventions on the other layers, aiming to raise the treated traits without lowering others; and (v) a co-twin control, borrowed from developmental psychology, which gives the new post-therapy questionnaire also to an untreated copy of the system, separating the therapy effect from the questionnaire effect. The release dossier states what was tested and what was not, so that deployers and their auditors can see what the judgment covers. The method is iterative and serves as continuous maintenance after every change. In a worked example on a constructed insurance assistant with nine seeded defects and 1,213 executions, the method found seven defects and an eighth partially; the ninth was never activated. The co-twin control showed that most apparent post-therapy changes on six traits were questionnaire effects. Agreement among three automated coders reached Krippendorff's alpha = 0.86-1.0, and, in a retrospective count, a test-guided diagnostic path recovered eight of nine grave findings at 62-82% of the cost.