From Disclosure to Substance: The Next Step for AI Transparency in Science

Disclosure + opinion figures — reproduction bundle================================================== Python scripts-------------- generate_opinion_figures.py — Figure 1 (temporal + forest) & Figure 2 (field-specific ORs), PDFs granularity_analysis.py — tier analysis: forest PNG, tier distribution PNG, modeling frame CSV granularity_bootstrap.py — bootstrap Data/Code OR violin PNGs + JSON Data under output/ (bundled)---------------------------- scorecard_2024_5000.csv, scorecard_2025_5000.csv papers_2024_5000.csv, papers_2025_5000.csv granularity_modeling_frame.csv (for bootstrap without re-fetching full text) opinion_figures/field_results.csv (optional snapshot from a prior run of the field models) Setup----- pip install -r requirements.txt Opinion figures (PDF, as in the manuscript)------------------------------------------- cd /path/to/disclosure_granularity_figures python3 generate_opinion_figures.py Writes: output/opinion_figures/figure1_combined.pdf output/opinion_figures/figure2_field_specific.pdf output/opinion_figures/field_results.csv (refit from scorecards + papers) Note: Figure 1 panels (A) bar heights and (B) pooled ORs are the values coded in generate_opinion_figures.py (main-text point estimates). Figure 2 is estimated from the bundled scorecard + papers tables (fields with n ≥ 250). Granularity + bootstrap figures (PNG)------------------------------------- python3 granularity_bootstrap.py # fast if granularity_modeling_frame.csv present python3 granularity_analysis.py # optional: refetch/cache full text, then PNGs Typical outputs: output/granularity_forest_tiers.png output/granularity_tier_distribution.png output/granularity_bootstrap_data_or.png output/granularity_bootstrap_code_or.png Reference PNGs from an earlier run: output/figures_reference/ Environment (optional)---------------------- GRANULARITY_WORKERS=8 GRANULARITY_MAX_N=0 GRANULARITY_BOOTSTRAP_B=2000 GRANULARITY_BOOTSTRAP_STRATIFIED=1