Reportification, Artifact Underdelivery, and Execution Claims under Meta-Task Instructions: A Black-Box Study of Multimodal LLMs
Exploratory technical report. We separate model behavior into three independent dimensions—artifact delivery, report-artifact relation, and execution-claim veracity—and annotate 35 samples black-box. In these 35 samples: under-delivery 42.9%, report substitution 20.0%, false completion claims 17.1%, concretized descriptions of unexecuted work 17.1%. The preregistered image primary analysis was not executed due to a delivery floor effect; the exploratory text control found no reduction in delivery (+0.4, p=1.00), and the sample is insufficient to establish equivalence. This report does not claim a new failure mechanism, nor a demonstrable increment over existing taxonomies. It provides a reproducible annotation framework, traceable cases, and hypotheses for further testing. Known limits: image content annotation not performed; no systematic literature review.
Authors
- Macily Lei (ORCID: https://orcid.org/0009-0004-1026-6280)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-02
- DOI
- https://doi.org/10.5281/zenodo.23889078
- Primary Topic
- Multimodal Machine Learning Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00