Can an essay be marked without being understood?

IMark is a rules-based marker for Leaving Certificate History and English essays. It contains no AI model, gives the same mark every time, traces every mark to a stated reason, and declines to mark what it cannot judge. Its central claim is that an essay's analytical form (making a claim, backing it with something specific, and tying it back to the question) can be assessed without any knowledge of what the essay is about. This paper sets out that thesis and reports everything measured so far, negatives included. On 242 History answer sets written by the author, IMark agrees with the author's paragraph labels on 98% of body paragraphs. A removal test shows why that proves less than it seems: IMark noticed a missing claim in 0 of 641 paragraphs, a missing tie-back in 11% and missing evidence in 46%. A prototype check that reads only a paragraph's shape noticed the missing claim and got both directions right in 82% of paragraphs. Word count ranks the answers as well as IMark does, and shuffling sentences within paragraphs leaves 223 of 242 marks unchanged. Fresh answers from other writers are needed before either prototype check enters the marker. The paper also pre-registers a blind validation study against experienced markers, with six hypotheses and fixed endpoints. The marker and script counts will be fixed in version 1.1, dated before any data is collected.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-28
DOI
https://doi.org/10.5281/zenodo.23014077
Primary Topic
Authorship Attribution and Profiling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

Can an essay be marked without being understood?

Sebastian Vaderaa
Zenodo (CERN European Organization for Nuclear Research)
Authorship Attribution and Profiling
preprint

Can an essay be marked without being understood?

Sebastian Vaderaa
preprint en

Abstract

IMark is a rules-based marker for Leaving Certificate History and English essays. It contains no AI model, gives the same mark every time, traces every mark to a stated reason, and declines to mark what it cannot judge. Its central claim is that an essay's analytical form (making a claim, backing it with something specific, and tying it back to the question) can be assessed without any knowledge of what the essay is about. This paper sets out that thesis and reports everything measured so far, negatives included. On 242 History answer sets written by the author, IMark agrees with the author's paragraph labels on 98% of body paragraphs. A removal test shows why that proves less than it seems: IMark noticed a missing claim in 0 of 641 paragraphs, a missing tie-back in 11% and missing evidence in 46%. A prototype check that reads only a paragraph's shape noticed the missing claim and got both directions right in 82% of paragraphs. Word count ranks the answers as well as IMark does, and shuffling sentences within paragraphs leaves 223 of 242 marks unchanged. Fresh answers from other writers are needed before either prototype check enters the marker. The paper also pre-registers a blind validation study against experienced markers, with six hypotheses and fixed endpoints. The marker and script counts will be fixed in version 1.1, dated before any data is collected.

Zenodo (CERN European Organization for Nuclear Research)
Quality Education
Authorship Attribution and Profiling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.