Validation design governs reliability claims in well-log lithology prediction

Abstract Well-log lithology classification supports reservoir identification and sedimentary facies mapping in exploration wells where core data are unavailable. Published studies commonly report row-level validation scores, although field deployment requires generalization to wells absent from training. We evaluate this deployment mismatch on a public 118-well Norwegian Continental Shelf dataset while controlling the target mapping, leakage protocol, missing-curve handling, and evaluation metric. Row-level validation substantially overstates unseen-well reliability: pooled F1-macro is 0.81–0.93 under row-level splitting but 0.31–0.45 under full leave-one-well-out cross-validation (LOWOCV), and the more conservative per-well mean over the fixed 12-class taxonomy is 0.187–0.245 across five configurations. Evaluated context-aware tree configurations score higher on some strict surfaces, though they differ from the numeric baselines in more than their feature set, so the margin is not attributed to the context fields themselves. Penalty-weighted and F1-macro model rankings agree on both row-level surfaces and diverge on all four strict surfaces, so metric choice matters on the same boundary. Class-level, per-well, and single-well diagnostics reveal persistent minority-class, mixed-lithology, model-specific, and well-to-well failure patterns. Reliability claims for this domain should be anchored in well-level evidence rather than row-level scores.