Prompt Complexity, Overprompting, and the Honesty of Error in LLM-Based Mechanical Drawing Interpretation: A Structured Empirical Evaluation

Large language models (LLMs) are increasingly used to interpret mechanical engineering drawings, yet the main risk is a wrong answer that reads as correct. We report an exploratory empirical study of how prompt design and evaluation shape the reliability of this interpretation, in two connected stages over the same twenty drawings. Stage 1 runs one prose prompting protocol on two model families and finds that progressive enrichment does not improve accuracy; beyond a threshold, it raises confident fabrication. Stage 2 compares six structured prompt architectures at thirty repetitions per cell, producing 3540 generations scored under four judge conditions. Across both stages, identification accuracy does not improve, while the rate of claims the drawing does not support varies by nearly an order of magnitude with prompt architecture. Only a judge that sees the drawing can measure it; a text-only judge inflates confident fabrication three to four times, making evaluator modality a first-order variable. Without a ground-truth answer, an image-only check can test only whether claims are visible; it recovers about half of the reference flags, enough for human-in-the-loop triage. A judge from another model family reproduces the ranking. We distill five practical recommendations for this setting.