Predictive capability of foundational concepts tests for problem-solving using machine learning concepts: Evaluating project-based learning courses in artificial intelligence literacy education

In the artificial intelligence (AI) era, secondary and university students should be able to apply AI for problem-solving. This study designed and evaluated an AI literacy programme to enhance understanding of machine learning concepts. It also examined how the conceptual understanding from two foundational courses (Courses 1 and 2) affected students' application of these concepts in the subsequent two project-based learning courses (Courses 3 and 4). The regression analysis of data from 566, 566, 470, and 196 student participants enrolled on Courses 1, 2, 3, and 4, respectively, revealed that the post-course concept tests for Courses 1 and 2 accounted for 19.9 % of the variance in the students' problem-solving ability test before they took Course 3. This result indicates that teaching students' foundational concepts can develop their ability to solve machine learning-related problems. The post-course concept tests for Courses 1 and 2, together with the pre-course problem-solving ability test for Course 3, collectively explained 27.4 % of the variance in the students’ problem-solving ability after completing Course 3. Together with the significant improvement in the paired-samples t -test statistics for the pre- and post-course problem-solving test of Course 3, this indicates the importance of providing opportunities for students to solve machine learning-related problems. These findings provide empirical evidence to inform the design of curricula for AI literacy programmes. Project-based learning (PBL) is an approach that can provide opportunities for participants to develop problem-solving skills using foundational AI knowledge. • We evaluated the impact of conceptual understanding of machine learning on PBL courses for solving related problems. • Foundational machine learning concepts can develop students' ability to solve machine learning-related problems. • It is equally important to provide opportunities for students to use these concepts for problem-solving. • Project-based learning provides opportunities for students to use their concepts for solving problems. • AI literacy programmes should include both foundational concepts and opportunities for using them for problem-solving.