MorphoEval: Nanopublication-Backed Evaluation of Morphology Implementations
MorphoEval is a seven-dimension evaluation framework with model-implementation fidelity (30%), biological validation (40%) and reproducibility (30%) sub-axes. Checkpointed, evidence-linked ratings are represented as signed nanopublications with immutable supersession histories. Ratings are not probabilities. Framework documentation and generator revision: 11a5328e3b223608984fe13f7679647803f688fd. Graph queries at that revision identify 327 recorded evaluation passes across 118 implementation claims, with 2,616 ratings. Two legacy passes have inferred claim associations explicitly flagged. The 270 production nanopublication identifiers in local manifests are not 270 evaluation passes. Live head resolution is partial (85 successful queries; 185 rate-limited). Full graph-result tables accompany the report. A subject-by-subject catalogue lists all 118 evaluated claims with each recorded overall score linked to its pinned assessment report. The companion CSV maps all 327 assessments to subjects, assessors, checkpoint contexts and reports, preserving historical disagreement without selecting an unverified latest score. Worked examples cover leaf-venation remediation and Conus assessor disagreement. The image-fitness section distinguishes direct visual comparisons from heuristic fallbacks, describes the scoring components and threshold, and reports the 3,116-entry image index without treating its passing labels as uniformly measured biological agreement. Formal-verification examples explain Lean proofs of conditional vein-tree conservation, Rule 30 local invertibility and reflection-symmetric cellular rules, including their graph links and limits. Proof source was inspected; no fresh Lean compilation is claimed. A frozen-snapshot evidence audit of MorphoLib: 41,255 repository files, 4,313 audit YAML files and 3,100 campaign entries. Complete tree and campaign reference checks are combined with a small deterministic diagnostic source sample. These counts are not independently validated experiments. All audit YAML contents were not revalidated; no biological simulations, Lean builds or external fish-data analyses were rerun. Thompson results are source-reported only. Snapshot: 330ea96196e857aaf1bac75874e6aab1d6181243. This is not a census of subsequent repository additions. The author maintains the evaluated repository; AI assistance is disclosed in the report. The report and derived findings are CC BY 4.0. Tooling retains MIT and original source notices are preserved. The selected source-and-evidence archive excludes third-party datasets and imagery; it includes a hash-verifying input retrieval tool.
Authors
- Joel Dietz
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-26
- DOI
- https://doi.org/10.5281/zenodo.23680704
- Primary Topic
- Scientific Computing and Data Management
- Type
- preprint