Interaction-Aware Bias Correction for LLM-as-a-Judge Evaluation: BIC, SSIT, IASC and IA-PPI (code, pilot data, pre-registration)

Reference implementation, gold-free real-judge pilot data, and frozen analysis pipeline for the paper Toward Interaction-Aware Bias Correction in LLM-as-a-Judge Evaluation: The Bias Interaction Coefficient and Interaction-Aware Prediction-Powered Inference. Automated LLM-as-a-judge pipelines correct position, verbosity and self-preference bias sequentially, implicitly assuming the biases are additively independent. This artifact accompanies analytical results showing that a position × verbosity interaction leaves a structural residual of magnitude ½|β_PV| that no sequential composition of single-bias corrections removes, and that a marginal prediction-powered-inference rectifier under-covers under bias-cell composition shift. Contents ssit.py — Swap-Symmetry Interaction Test: a gold-free difference-in-differences over the four position × verbosity swap cells that identifies β_PV and tests additive independence with no human labels. iasc.py — Interaction-Aware Swap Correction: a label-free debiasing method that subtracts the swap-identified position-coupled bias terms, matching 2× swap-averaging at 1× evaluation cost. ia_ppi.py — Interaction-Aware Prediction-Powered Inference: a factorial-rectifier PPI estimator returning an asymptotically valid CI for the debiased win rate. sim_prop2_coverage.py, sim_ranking_impact.py — NumPy-only simulations that regenerate the coverage and leaderboard-impact tables. analyze_real.py, analyze_iasc.py — Frozen analysis of the pilot data. collect_free.py — Multi-provider free-tier judgment collector covering 13 model families across Groq, Google AI Studio, Cerebras, Mistral, OpenRouter, Cohere, SambaNova, NVIDIA NIM, GitHub Models, Hugging Face, and Together. data/ — Stimulus pairs and 555 valid pilot judgments from three judges (gpt-oss-20b, gpt-oss-120b, gemma-3-27b) across all eight (P,V,S) swap cells. paper_neurips_workshop.pdf, paper_neurips_workshop.tex — The manuscript. Dependencies: numpy, scipy for all estimators and simulations; matplotlib, pillow, pandas, scikit-learn, and statsmodels only for the legacy figure pipeline; and requests for the collector. Seed 2024 throughout.