Reportification, Artifact Underdelivery, and Execution Claims under Meta-Task Instructions: A Black-Box Study of Multimodal LLMs

Exploratory technical report. We separate model behavior into three independent dimensions—artifact delivery, report-artifact relation, and execution-claim veracity—and annotate 35 samples black-box. In these 35 samples: under-delivery 42.9%, report substitution 20.0%, false completion claims 17.1%, concretized descriptions of unexecuted work 17.1%. The preregistered image primary analysis was not executed due to a delivery floor effect; the exploratory text control found no reduction in delivery (+0.4, p=1.00), and the sample is insufficient to establish equivalence. This report does not claim a new failure mechanism, nor a demonstrable increment over existing taxonomies. It provides a reproducible annotation framework, traceable cases, and hypotheses for further testing. Known limits: image content annotation not performed; no systematic literature review.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-02
DOI
https://doi.org/10.5281/zenodo.23889078
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Reportification, Artifact Underdelivery, and Execution Claims under Meta-Task Instructions: A Black-Box Study of Multimodal LLMs

Macily Lei
2 links
Zenodo (CERN European Organization for Nuclear Research)
Multimodal Machine Learning Applications
article

Reportification, Artifact Underdelivery, and Execution Claims under Meta-Task Instructions: A Black-Box Study of Multimodal LLMs

Macily Lei
article en

Abstract

Exploratory technical report. We separate model behavior into three independent dimensions—artifact delivery, report-artifact relation, and execution-claim veracity—and annotate 35 samples black-box. In these 35 samples: under-delivery 42.9%, report substitution 20.0%, false completion claims 17.1%, concretized descriptions of unexecuted work 17.1%. The preregistered image primary analysis was not executed due to a delivery floor effect; the exploratory text control found no reduction in delivery (+0.4, p=1.00), and the sample is insufficient to establish equivalence. This report does not claim a new failure mechanism, nor a demonstrable increment over existing taxonomies. It provides a reproducible annotation framework, traceable cases, and hypotheses for further testing. Known limits: image content annotation not performed; no systematic literature review.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 20%
Multimodal Machine Learning Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon