Self-Service BI Support with an Enterprise LLM Agent: Architecture, Copilot Studio Deployment and Ticket-Deflection Outcomes in a 3,000-User Municipal Platform

Self-service business intelligence moves report building to thousands of staff, and the support load moves with it: questions about access, refresh failures, DAX and permissions become tickets for a far smaller BI team. This article reports the design, deployment and outcomes of a large language model (LLM) support agent for the City of San José's Power BI platform (3,000+ users, 100+ production dashboards). Piloted as a custom GPT and rebuilt in Microsoft Copilot Studio, grounded in curated City content and connected to Teams and the IT service-management system, the agent is described through a four-layer architecture, a weekly routine that turns closed tickets into cited answers, and the governance controls a public agency required. A twelve-month interrupted time series shows median monthly Power BI tickets falling 45% (191 to 104) and 49% per 1,000 active users, a level change at release of −60 tickets a month (95% CI −84 to −35, p < 0.001) with no pre-release trend, while the agent handled 316 sessions a month with cited answers in under a minute. Controls are mapped to the NIST AI Risk Management Framework, a quality-and-effort protocol is specified, and the findings argue that content currency, not prompt design, is the operating cost of such an agent.

Authors

Publication Details

Journal
Preprints.org
Published
2026-11-12
DOI
https://doi.org/10.20944/preprints202611.1331.v1
Primary Topic
Big Data and Business Intelligence
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Self-Service BI Support with an Enterprise LLM Agent: Architecture, Copilot Studio Deployment and Ticket-Deflection Outcomes in a 3,000-User Municipal Platform

Vamshi Vanguri
Preprints.org
Big Data and Business Intelligence
preprint

Self-Service BI Support with an Enterprise LLM Agent: Architecture, Copilot Studio Deployment and Ticket-Deflection Outcomes in a 3,000-User Municipal Platform

Vamshi Vanguri
preprint en

Abstract

Self-service business intelligence moves report building to thousands of staff, and the support load moves with it: questions about access, refresh failures, DAX and permissions become tickets for a far smaller BI team. This article reports the design, deployment and outcomes of a large language model (LLM) support agent for the City of San José's Power BI platform (3,000+ users, 100+ production dashboards). Piloted as a custom GPT and rebuilt in Microsoft Copilot Studio, grounded in curated City content and connected to Teams and the IT service-management system, the agent is described through a four-layer architecture, a weekly routine that turns closed tickets into cited answers, and the governance controls a public agency required. A twelve-month interrupted time series shows median monthly Power BI tickets falling 45% (191 to 104) and 49% per 1,000 active users, a level change at release of −60 tickets a month (95% CI −84 to −35, p < 0.001) with no pre-release trend, while the agent handled 316 sessions a month with cited answers in under a minute. Controls are mapped to the NIST AI Risk Management Framework, a quality-and-effort protocol is specified, and the findings argue that content currency, not prompt design, is the operating cost of such an agent.

Preprints.org
Big Data and Business Intelligence
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon