Evidence-Grounded Retrieval of Historical BIM Coordination Knowledge: A Human-in-the-Loop Framework and Open Synthetic Benchmark

On repeat programs such as data-center campuses, the same coordination clashes recur across projects that share a basis of design, yet each project team resolves them again because resolution knowledge stays inside individual project records. Large language models make it easy to summarize old records, but an ungrounded answer to an engineering question is a liability. This article proposes a human-in-the-loop framework for retrieving historical coordination precedents and an open benchmark for testing it. The framework contributes four elements. First, an issue-record schema keeps the engineering context of a clash, such as grid zone, disciplines and elements, alongside its text. Second, a hybrid retrieval method fuses keyword, semantic and structured-feature rankings, so that two issues with similar wording but different root causes are separated, and demotes precedents that a later design revision has superseded. Third, an answer contract allows a generated summary to state only what cited records support and requires it to say when they support nothing. Fourth, a feedback loop turns each reviewer's acceptance or rejection into labelled data for calibration. On a synthetic benchmark of 200 issue records and 40 queries with deliberate hard negatives, keyword and semantic retrieval placed 6.7% to 9.2% hard negatives and 10.8% to 13.3% stale precedents in their top five results on the held-out split. The hybrid with superseded-record demotion raised nDCG@5 from 0.825 for keyword retrieval to 0.915, cut hard negatives to 2.5% and stale precedents to 5.8%, and abstained correctly on all three held-out queries without a precedent. Demotion gave the largest single gain; the remaining stale results were supersessions that no one had recorded. The answer contract kept 92% of cited statements supported, against 44% for an unconstrained summary. These are results on synthetic data, not evidence of field accuracy. The benchmark and code are released openly.

Authors

Publication Details

Journal
Engineering Archive (engrXiv)
Published
2026-11-17
DOI
https://doi.org/10.31224/8819
Primary Topic
Information Retrieval and Search Behavior
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Evidence-Grounded Retrieval of Historical BIM Coordination Knowledge: A Human-in-the-Loop Framework and Open Synthetic Benchmark

Roja Malligarjunan
3 links
Engineering Archive (engrXiv)
Information Retrieval and Search Behavior
preprint

Evidence-Grounded Retrieval of Historical BIM Coordination Knowledge: A Human-in-the-Loop Framework and Open Synthetic Benchmark

Roja Malligarjunan
preprint en

Abstract

On repeat programs such as data-center campuses, the same coordination clashes recur across projects that share a basis of design, yet each project team resolves them again because resolution knowledge stays inside individual project records. Large language models make it easy to summarize old records, but an ungrounded answer to an engineering question is a liability. This article proposes a human-in-the-loop framework for retrieving historical coordination precedents and an open benchmark for testing it. The framework contributes four elements. First, an issue-record schema keeps the engineering context of a clash, such as grid zone, disciplines and elements, alongside its text. Second, a hybrid retrieval method fuses keyword, semantic and structured-feature rankings, so that two issues with similar wording but different root causes are separated, and demotes precedents that a later design revision has superseded. Third, an answer contract allows a generated summary to state only what cited records support and requires it to say when they support nothing. Fourth, a feedback loop turns each reviewer's acceptance or rejection into labelled data for calibration. On a synthetic benchmark of 200 issue records and 40 queries with deliberate hard negatives, keyword and semantic retrieval placed 6.7% to 9.2% hard negatives and 10.8% to 13.3% stale precedents in their top five results on the held-out split. The hybrid with superseded-record demotion raised nDCG@5 from 0.825 for keyword retrieval to 0.915, cut hard negatives to 2.5% and stale precedents to 5.8%, and abstained correctly on all three held-out queries without a precedent. Demotion gave the largest single gain; the remaining stale results were supersessions that no one had recorded. The answer contract kept 92% of cited statements supported, against 44% for an unconstrained summary. These are results on synthetic data, not evidence of field accuracy. The benchmark and code are released openly.

Engineering Archive (engrXiv)
Information Retrieval and Search Behavior
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon