LLaMA-UTP: Knowledge-Guided Expert Mixture for Analyzing Uncertain Tax Positions

This paper introduces LLaMA-UTP, a novel framework for classifying and interpreting uncertain tax positions (UTPs) in technology companies. Tax professionals face significant challenges when analyzing UTPs: navigating vast bodies of intricate legal text, interpreting highly technical domain-specific language, maintaining accurate citations to authoritative sources, and producing justifications that comply with legal standards. We address these challenges with a knowledge-guided expert mixture system built upon a domain-adapted LLaMA architecture incorporating retrieval-augmented generation to ground analysis in relevant tax authorities, specialized expert subnetworks for different tax knowledge domains, and a tax citation knowledge graph that enhances interpretability through structured reasoning paths. Our comprehensive evaluation demonstrates that LLaMA-UTP achieves 90.5% accuracy in classifying UTP risk levels, significantly outperforming baseline models including LegalBERT (82.0%) and GPT-3.5 (78.4%). The system generates explanations with 93% citation accuracy and zero hallucinated references, as validated by tax professionals who rated its outputs as highly aligned with expert reasoning (4.2/5). Through ablation studies, we demonstrate that each architectural component contributes measurably to both accuracy and interpretability. This work makes significant contributions to both AI/ML methodology and tax law analysis by showcasing howknowledge-guided expert mixtures can enhance model performance in highly specialized professional domains while maintaining the explainability necessary for high-stakes legal applications.