Generating Executable Code in a Visually Opaque Unicode Language: A CLI Case Study
Can a language model use a supplied specification to produce executable programs in a notation that offers an unaided reader almost no visual cues? We study emd, a stack language written in eight similar Unicode dashes, through command-line clients with tools disabled. Across 49 tasks, we collected 1150 Claude responses and an incomplete extension of 573 OpenAI responses. Exact copying is often successful, and in a complete Sol basic-prompt sample, 70 of 72 program answers produce the required behavior. Correct content does not always arrive in usable source form: output formatting rejects 33.3% of Haiku's correct answers under the basic prompt, reduced to 0.0% when the consumer's requirements are stated explicitly. We report exact-answer and normalized-content scores separately, together with token-cost estimates. These observations characterize the tested model-CLI-prompt configurations. They do not measure human readability, isolate the effect of visual opacity, or establish that the language was absent from training data.
Authors
- Daniele Scaratti
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-06
- DOI
- https://doi.org/10.5281/zenodo.24011465
- Primary Topic
- Software Engineering Research
- Type
- preprint