From Sentiment to Pedagogical Insight: Comparing SVM, Random Forest, and Multi-Task IndoBERT for Joint Sentiment–Theme Classification of Indonesian Student Feedback

Purpose – Student feedback in Indonesian higher education is a rich pedagogical resource that stays underused. Sentiment analysis under controlled conditions has rarely reached the Indonesian academic domain. This study compares three classification approaches on student feedback: Support Vector Machine, Random Forest, and IndoBERT. Design/methods/approach – Unlike earlier work that splits sentiment and theme into two independent pipelines, we design a multi-task IndoBERT with one shared encoder and two classification heads, so that sentiment and instructional dimension are predicted jointly for the same response. We also build a preprocessing pipeline tuned to Indonesian academic discourse, with slang normalization and domain-specific stop-word filtering. Experiments run on a corpus of 800 annotated responses drawn from four study programs across three disciplinary clusters. Findings – Under repeated stratified 5-fold cross-validation, multi-task IndoBERT reached a mean sentiment accuracy of 77.9%, against SVM (72.8%) and Random Forest (70.1%); on the held-out test set its thematic macro F1 was 0.757. On the same test set, McNemar's test placed IndoBERT significantly ahead of both baselines at the 0.05 level (p = 0.008 against SVM and p = 0.012 against Random Forest); only the margin over SVM also clears the stricter 0.01 threshold. Three thematic dimensions emerged: Instructional Clarity, Assessment Design, and Technological Access. Research implications/limitations – The joint model provides actionable insights for NLP research and targeted faculty development. However, the single-institution scope constrains cross-institutional generalization. Originality/value – The pedagogical contribution lies in that joint output: instead of reporting that feedback is negative, the model identifies which dimension of teaching the negativity concerns, so faculty-development programs can target where dissatisfaction actually concentrates rather than act on an aggregate satisfaction score.

Authors

Publication Details

Journal
Information Technology Education Journal
Published
2026-11-01
DOI
https://doi.org/10.59562/intec.v5i4.14666
Primary Topic
Sentiment Analysis and Opinion Mining
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

From Sentiment to Pedagogical Insight: Comparing SVM, Random Forest, and Multi-Task IndoBERT for Joint Sentiment–Theme Classification of Indonesian Student Feedback

Sri Mujiyono, Ucta Pradema Sanjaya
3 links
Information Technology Education Journal
Sentiment Analysis and Opinion Mining
article

From Sentiment to Pedagogical Insight: Comparing SVM, Random Forest, and Multi-Task IndoBERT for Joint Sentiment–Theme Classification of Indonesian Student Feedback

Sri Mujiyono, Ucta Pradema Sanjaya
article en

Abstract

Purpose – Student feedback in Indonesian higher education is a rich pedagogical resource that stays underused. Sentiment analysis under controlled conditions has rarely reached the Indonesian academic domain. This study compares three classification approaches on student feedback: Support Vector Machine, Random Forest, and IndoBERT. Design/methods/approach – Unlike earlier work that splits sentiment and theme into two independent pipelines, we design a multi-task IndoBERT with one shared encoder and two classification heads, so that sentiment and instructional dimension are predicted jointly for the same response. We also build a preprocessing pipeline tuned to Indonesian academic discourse, with slang normalization and domain-specific stop-word filtering. Experiments run on a corpus of 800 annotated responses drawn from four study programs across three disciplinary clusters. Findings – Under repeated stratified 5-fold cross-validation, multi-task IndoBERT reached a mean sentiment accuracy of 77.9%, against SVM (72.8%) and Random Forest (70.1%); on the held-out test set its thematic macro F1 was 0.757. On the same test set, McNemar's test placed IndoBERT significantly ahead of both baselines at the 0.05 level (p = 0.008 against SVM and p = 0.012 against Random Forest); only the margin over SVM also clears the stricter 0.01 threshold. Three thematic dimensions emerged: Instructional Clarity, Assessment Design, and Technological Access. Research implications/limitations – The joint model provides actionable insights for NLP research and targeted faculty development. However, the single-institution scope constrains cross-institutional generalization. Originality/value – The pedagogical contribution lies in that joint output: instead of reporting that feedback is negative, the model identifies which dimension of teaching the negativity concerns, so faculty-development programs can target where dissatisfaction actually concentrates rather than act on an aggregate satisfaction score.

Information Technology Education Journal
Quality education
Openalex Percentile: Top 14%
Sentiment Analysis and Opinion Mining
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon