Cascaded Neurosymbolic Code Generation for Niche DSLs: Preserving Chain-of-Thought in Grammar-Constrained Decoding

Domain-Specific Languages (DSLs) are essential in software engineering for safely expressing complex domain logic. However, Large Language Models (LLMs) struggle to generate syntactically and semantically correct code for niche DSLs due to sparse representation in pre-training corpora. While Grammar-Constrained Decoding (GCD) resolves syntactic hallucinations by masking logits through a formal Context-Free Grammar (CFG), empirical evidence shows that strict GCD disrupts the autoregressive Chain-of-Thought (CoT) reasoning of modern models, frequently forcing them into irreversible semantic dead-ends. To overcome the friction between internal neural reasoning and external symbolic constraints, we propose a Dual-Phase Cascaded Neurosymbolic framework. In the first phase, the model is provided with dynamically injected grammar rules and is permitted to reason unconstrained, producing an optimistic code draft. If the draft fails native compiler checks, the system enters a second phase: it preserves the successful semantic reasoning from Phase 1 but re-generates the code under strict GCD enforcement. This cascaded architecture utilizes the formal FSM not as an adversarial constraint, but as a localized syntax repair engine guided by the model’s own prior reasoning. We construct a comprehensive benchmark of 100 MiniZinc constraint programming tasks and evaluate our approach using a pass@k metric with a strict semantic LLM judge. Our findings demonstrate that this dual-phase “think-then-constrain” approach significantly outperforms zero-shot, pure few-shot, and pure GCD baselines, achieving highly reliable, training-free code generation for unseen DSLs.