An AI Literacy Checkpoint Framework for Critical Evaluation of AI-Generated Content in Higher Education

Abstract The rapid adoption of generative artificial intelligence (GenAI) in higher education has created a significant assessment gap, as students increasingly submit AI-assisted work without demonstrating competencies required to verify facts, authenticate sources, identify hallucinations, assess bias, uphold academic integrity, and critically reflect on AI use. Although several AI literacy frameworks have been proposed, including those by Fan et al. (2025), AI4K12 (Touretzky et al., 2019), and international policy reports (UNESCO, 2019, 2021, 2022), no automated and empirically validated system has existed to assess AI-assisted academic submissions systematically. This study introduces the AI Literacy Checkpoint Framework (ALCF), a six-checkpoint automated sequential pipeline that evaluates Source Verification (CP1), Fact Verification (CP2), Hallucination Detection (CP3), Bias and Fairness Assessment (CP4), Academic Integrity (CP5), and Reflection and Critical Thinking (CP6). The framework was evaluated through three progressively larger sets of constructed submissions to examine feasibility, robustness, and scalability. Phase 1 (\(\:N=10\) submissions) established proof of concept; Phase 2 (\(\:N=20\) submissions) introduced more complex error scenarios and structured reporting; and Phase 3 (\(\:N=50\) submissions) provided the primary evaluation with comparative baseline systems, approval strategies, and inter-rater reliability analysis. Phase 3 achieved an end-to-end accuracy of 0.980, with precision of 0.964, recall of 1.000, F1-score of 0.982, and MCC of 0.960. Deterministic checkpoints (CP1–CP4) achieved strong performance (0.92–0.96 accuracy), whereas subjective checkpoints (CP5–CP6) remained limited at 0.72 accuracy, highlighting challenges in rubric-based assessment. Comparative evaluations showed that the complete ALCF outperformed citation-only, retrieval-only, and majority-vote alternatives. These findings establish ALCF as a flexible and extensible framework that can be adapted as AI capabilities, hallucination profiles, and educational assessment practices evolve.