Designing Student Rockets for the Worst-Case 1% of Flight Scenarios: An LLM-Agent-Built Pipeline Compared with Actual Flight Data and Re-implemented by a Second LLM Agent
From 2022 to 2026, 187 of 524 rockets launched in the International Rocket Engineering Competition scored no recovery points. Yet none of 15 student design documents verified every rule at the worst 1% of thousands of simulated flights with varied wind, thrust and mass, and the three rulebooks reviewed ask only for fixed cases. Such verification needs a large software pipeline, so the author had the LLM agent Claude Code build one that accepts a design only if all implemented flight and recovery rules (mostly the strictest of the three) are met at the worst 1% of 9,600 flights in archived winds. Designs were accepted in 6 of 13 missions, two with assumed stand-in motors, and on new seeds 5 of the 6 kept a 95% lower bound of at least 99% on the share of flights meeting all rules. To see whether this verification matters in the design search, the agent swapped its Monte Carlo flights for 5 fixed-input ones with equal margins, and the bound fell below 99% in all 9 runs, while the full method reached 99% in 6 of 9. Nearly all violations came from a closely binding rule, NASA Student Launch's 2.0-caliber pad stability margin. Fixed cases also mislead, as the 2024 NASA Student Launch drift calculation keeps the NASA design within 762 m, yet 27.9% of its simulated descents drift farther. As the bounds cover only sampling error, the agent compared the flight and load models with flight logs. With forecasts made at least 6 hours ahead, mean absolute apogee error was 3.60% on 8 archived flights (5.02% with a ninth, pre-registered one), but the one measured opening shock was at least 2.9 times the prediction. To audit the code, a second agent (Codex) reproduced all 13 missions file by file and re-implemented the recovery sizing and descent statistics without reference code or results, matching all 222 values. No design was built or flown. The contribution is the LLM-built method and its audit.
Authors
- David Castro (ORCID: https://orcid.org/0009-0002-7634-3600)
Institutions
- Ateneo de Manila University (PH)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-02
- DOI
- https://doi.org/10.5281/zenodo.23887825
- Primary Topic
- Aerospace and Aviation Technology
- Type
- preprint