Evaluation of teachers' explaining skills: An approach based on artificial intelligence and AI causal analysis

This study develops an innovative assessment system based on Artificial Intelligence (AI) and causal inference to objectively evaluate teachers' explaining skills. Traditional assessments often suffer from subjectivity and inconsistency, limiting their effectiveness in capturing the complex and dynamic nature of teaching skills. To overcome these limitations, this research integrates a black box AI model (Google Gemini 1.5 Flash LLM) and a glass box approach (multivariate linear regression) to provide transparent and interpretable assessments. Using 200 video transcripts from various teaching contexts, the results indicate a high correlation ( r = 0.91) between the AI-generated scores and regression model predictions, confirming the validity and reliability of the approach. Causal analysis through Directed Acyclic Graphs (DAG) and Propensity Score Matching (PSM) identifies critical teaching indicators, such as “Adaptation to Student Understanding” (ATE = 0.41) and “Wait Time” (ATE = 0.48), which significantly impact teaching effectiveness. Counterfactual simulations further reveal potential score improvements by up to 0.6 points when enhancing these key areas. The proposed method provides systematic, transparent, and actionable insights, contributing significantly to improving educational quality through precise and evidence-based teacher evaluations. This research also aligns with the Sustainable Development Goals (SDGs), particularly SDG 4 (Quality Education), SDG 9 (Industry, Innovation, and Infrastructure), and SDG 10 (Reduced Inequalities), by promoting equitable, innovative, and high-quality education practices.

Authors

Institutions

Publication Details

Journal
Science Talks
Published
2025-09-23
DOI
https://doi.org/10.1016/j.sctalk.2025.100491
Primary Topic
Online Learning and Analytics
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Evaluation of teachers' explaining skills: An approach based on artificial intelligence and AI causal analysis

Harjono Harjono, Ella Kusumastuti, Dimas Gilang Ramadhani, Sri Susilogati Sumarti et al.
Science Talks
Online Learning and Analytics
article

Evaluation of teachers' explaining skills: An approach based on artificial intelligence and AI causal analysis

Harjono Harjono, Ella Kusumastuti, Dimas Gilang Ramadhani, Sri Susilogati Sumarti, Nafi'’atun Amaliya
article en

Abstract

This study develops an innovative assessment system based on Artificial Intelligence (AI) and causal inference to objectively evaluate teachers' explaining skills. Traditional assessments often suffer from subjectivity and inconsistency, limiting their effectiveness in capturing the complex and dynamic nature of teaching skills. To overcome these limitations, this research integrates a black box AI model (Google Gemini 1.5 Flash LLM) and a glass box approach (multivariate linear regression) to provide transparent and interpretable assessments. Using 200 video transcripts from various teaching contexts, the results indicate a high correlation ( r = 0.91) between the AI-generated scores and regression model predictions, confirming the validity and reliability of the approach. Causal analysis through Directed Acyclic Graphs (DAG) and Propensity Score Matching (PSM) identifies critical teaching indicators, such as “Adaptation to Student Understanding” (ATE = 0.41) and “Wait Time” (ATE = 0.48), which significantly impact teaching effectiveness. Counterfactual simulations further reveal potential score improvements by up to 0.6 points when enhancing these key areas. The proposed method provides systematic, transparent, and actionable insights, contributing significantly to improving educational quality through precise and evidence-based teacher evaluations. This research also aligns with the Sustainable Development Goals (SDGs), particularly SDG 4 (Quality Education), SDG 9 (Industry, Innovation, and Infrastructure), and SDG 10 (Reduced Inequalities), by promoting equitable, innovative, and high-quality education practices.

Science TalksVol. 16
State University of Semarang (ID)
Openalex Percentile: Top 71%
Online Learning and Analytics
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.