Cross-Model Analysis with Human Judgment: Making AI-Assisted Analysis Defensible

AI models produce analyses that read well but do not reveal what they missed or which statements are supported by evidence. Agreement between two models can be informative, but it does not establish whether an interpretation is well founded, and models can share the same errors. This working paper presents cross-model analysis with human judgment, a method for making interpretive analyses, such as strategy assessments, project reviews, and risk evaluations, transparent and defensible. Two models from different families independently analyze the same frozen sources. A model then compares their outputs item by item and classifies each divergence into one of seven types. Where the sources or predefined decision rules settle a divergence, the model proposes a resolution for the analyst to confirm; where they do not, the analyst decides and records the reasons. Before the result is frozen, two models from different families check every statement against the sources, and the analyst reviews their results in full or by sample. In the final result, every statement carries a cited source or a label as inference or analyst judgment. In two applications to strategic analysis, both conducted by the method’s developer, contradictions with the sources or rules accounted for about 5% of registered divergences in the analysis stage, and about half of the divergences concerned items covered by only one model. In the second application, 9% of the items in the analysis and 26% in the recommendations required an analyst decision. The method was developed in the case of SMA Solar Technology AG, documented in Strategic Analysis in Practice: A Connected Toolbox and the ALIGN Approach, and applied a second time to Heidelberger Druckmaschinen AG. Its logic is not limited to strategic analysis.