How often do San Francisco building permits that add homes reach a final inspection? A held-out test of permit-stage completion forecasts

Objective. Does a model that adds years open, size and permit type to a project's permit stage forecast housing completion better than a stage-only rate, and what share of units reach a final inspection? I set the shares beside San Francisco's Housing Element credits: 70% for approved projects that have filed for a building permit or await a first construction document, and 100% for projects authorized for construction (my reading of a 2018 Planning memo, not a City percentage). Data. DataSF building-permit records: projects with a unit-adding permit on file (Planning approval not checked), scored once under pre-registered rules on six held-out 1 January dates, 2017 to 2022. This is a retrospective test on today's records, not a real-time forecast. Findings. The model's Brier score was 10.1% lower than the stage rate (95% bootstrap interval 6.5% to 13.0%; coverage unverified, probably too narrow), lower on 6 of 6 dates. On a stricter outcome the gain is 5.4% (2.2% to 8.5%), inconclusive against the 5% margin, and 5.5% (-1.9% to 13.1%) over a stage-by-years-open average, so that edge is not established. Four of seven predictions held. Within 4 years, 11.1%, 23.3% and 68.5% of stage F, I and C units were in projects with a final inspection on a linked permit (capped at 200 units per project; uncapped 9.8%, 23.3%, 69.6%); within 8 years, 47.3%, 45.8% and 77.5% (stricter linking, uncapped; 49.7%, 47.0%, 79.5% with the 4-year test's linking): more than 10 points below the reference shares across the whole interval for F and C, inconclusive for I; the C reference is my own 100%, not a City figure. On the City's own measure for stage F (a permit issued within 8 years), the share was 62.7% (53.7% to 71.6%), which includes 70%. One permit's final does not show all counted units were built. The City's 70% is a permit-receipt rate, so this does not show the City overcounts; the 8-year plan's timing rests on my own record, not an independent timestamp. What is new. A held-out, stage-by-stage completion test with intervals for San Francisco; I found none earlier in a search. Use. A historical check for planners reviewing stage credits.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24280945
Primary Topic
Housing Market and Economics
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

How often do San Francisco building permits that add homes reach a final inspection? A held-out test of permit-stage completion forecasts

Jayden Ling
1 links
Zenodo (CERN European Organization for Nuclear Research)
Housing Market and Economics
preprint

How often do San Francisco building permits that add homes reach a final inspection? A held-out test of permit-stage completion forecasts

Jayden Ling
preprint en

Abstract

Objective. Does a model that adds years open, size and permit type to a project's permit stage forecast housing completion better than a stage-only rate, and what share of units reach a final inspection? I set the shares beside San Francisco's Housing Element credits: 70% for approved projects that have filed for a building permit or await a first construction document, and 100% for projects authorized for construction (my reading of a 2018 Planning memo, not a City percentage). Data. DataSF building-permit records: projects with a unit-adding permit on file (Planning approval not checked), scored once under pre-registered rules on six held-out 1 January dates, 2017 to 2022. This is a retrospective test on today's records, not a real-time forecast. Findings. The model's Brier score was 10.1% lower than the stage rate (95% bootstrap interval 6.5% to 13.0%; coverage unverified, probably too narrow), lower on 6 of 6 dates. On a stricter outcome the gain is 5.4% (2.2% to 8.5%), inconclusive against the 5% margin, and 5.5% (-1.9% to 13.1%) over a stage-by-years-open average, so that edge is not established. Four of seven predictions held. Within 4 years, 11.1%, 23.3% and 68.5% of stage F, I and C units were in projects with a final inspection on a linked permit (capped at 200 units per project; uncapped 9.8%, 23.3%, 69.6%); within 8 years, 47.3%, 45.8% and 77.5% (stricter linking, uncapped; 49.7%, 47.0%, 79.5% with the 4-year test's linking): more than 10 points below the reference shares across the whole interval for F and C, inconclusive for I; the C reference is my own 100%, not a City figure. On the City's own measure for stage F (a permit issued within 8 years), the share was 62.7% (53.7% to 71.6%), which includes 70%. One permit's final does not show all counted units were built. The City's 70% is a permit-receipt rate, so this does not show the City overcounts; the 8-year plan's timing rests on my own record, not an independent timestamp. What is new. A held-out, stage-by-stage completion test with intervals for San Francisco; I found none earlier in a search. Use. A historical check for planners reviewing stage credits.

Zenodo (CERN European Organization for Nuclear Research)
Sustainable cities and communities
Housing Market and Economics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon