Self-Service BI Support with an Enterprise LLM Agent: Architecture, Copilot Studio Deployment and Ticket-Deflection Outcomes in a 3,000-User Municipal Platform
Self-service business intelligence moves report building to thousands of staff, and the support load moves with it: questions about access, refresh failures, DAX and permissions become tickets for a far smaller BI team. This article reports the design, deployment and outcomes of a large language model (LLM) support agent for the City of San José's Power BI platform (3,000+ users, 100+ production dashboards). Piloted as a custom GPT and rebuilt in Microsoft Copilot Studio, grounded in curated City content and connected to Teams and the IT service-management system, the agent is described through a four-layer architecture, a weekly routine that turns closed tickets into cited answers, and the governance controls a public agency required. A twelve-month interrupted time series shows median monthly Power BI tickets falling 45% (191 to 104) and 49% per 1,000 active users, a level change at release of −60 tickets a month (95% CI −84 to −35, p < 0.001) with no pre-release trend, while the agent handled 316 sessions a month with cited answers in under a minute. Controls are mapped to the NIST AI Risk Management Framework, a quality-and-effort protocol is specified, and the findings argue that content currency, not prompt design, is the operating cost of such an agent.
Authors
- Vamshi Vanguri (ORCID: https://orcid.org/0009-0004-0900-8540)
Publication Details
- Journal
- Preprints.org
- Published
- 2026-11-12
- DOI
- https://doi.org/10.20944/preprints202611.1331.v1
- Primary Topic
- Big Data and Business Intelligence
- Type
- preprint