LLM Agent-Based Task Synthesis Framework with Environment-Grounded Scoring for Training Text-to-BIM Agents

Text-to-building information model (BIM), or TTB, tasks aim to automate the creation of BIMs from textual design instructions. Although recent large language model (LLM)-driven agents have demonstrated potential for TTB tasks, existing TTB approaches primarily employ closed-source LLMs, limiting local deployment and task-specific fine-tuning. Open-weight small language models (SLMs) can offer an effective alternative, but training them as TTB agents requires substantial instruction-trajectory data, which is costly to construct manually. This paper proposes BIMCraft, an LLM agent-based task synthesis framework for training SLMs as TTB agents. The BIMCraft framework employs LLM agents to generate diverse TTB tasks as instruction-metadata pairs and produce corresponding multiturn code-execution trajectories in a BIM environment. To obtain reliable training data, we use five environment-grounded scoring functions to evaluate requirement fulfillment and validity and a rejection fine-tuning strategy to select verified instruction-trajectory pairs for training. The experiments demonstrate that BIMCraft substantially improves the TTB performance of three open-weight SLMs, with the best-performing model (Qwen3-4B) achieving 68.8% Pass@1 and 54.2% Pass4 and reaching performance that is competitive with closed-source LLM agents. This work also analyzes the diversity and instruction-metadata consistency of the synthesized tasks and the effects of the scoring functions, score threshold, and training data scale.