Automated Text Coherence Analysis through Semantic Subspace Modelling and Ensemble Learning

Coherence is fundamental to producing clear and logical academic writing. Despite its importance, most automated evaluation systems primarily target grammar and style, often overlooking this critical discourse-level quality. This study presents a comprehensive comparative analysis of multiple computational approaches like classical machine learning, recurrent and convolutional neural networks (RNNs and CNNs), transformer-based architectures (BERT, RoBERTa, SBERT), and graph-based and hybrid models, specifically for coherence assessment in academic texts. Using a labelled academic writing dataset, model performance was benchmarked using accuracy, precision, recall and F1-scores to evaluate both training and test generalisation. Among classical models trained on TF-IDF and topic-based features, ensemble methods, such as Random Forest and XGBoost, demonstrated stronger robustness than linear and probabilistic baselines, while the nonlinear Multi-Layer Perceptron (MLP) achieved a test F1-score of 76.97%. Deep neural models exhibited stable performance, with the BiLSTM achieving a 76% test F1-score, outperforming LSTM, GRU and CNN models. Graph-based approaches integrating knowledge graph features improved generalisation, reaching a 76.64% test F1-score. The best overall performance was achieved by a hybrid model combining RoBERTa embeddings with Random Forest, attaining an 81.64% test F1-score. These findings indicate that integrating deep semantic embeddings with well-regularised ensemble classifiers offers the most effective and interpretable strategy for automated coherence evaluation.

Authors

Institutions

Publication Details

Journal
Journal of Information & Knowledge Management
Published
2026-11-13
DOI
https://doi.org/10.1142/s0219649226500784
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Automated Text Coherence Analysis through Semantic Subspace Modelling and Ensemble Learning

Peeta Basa Pati, Samrudhi S. Kamble, Sanjivani P. Shendre, M. Yagnasri Priya et al.
1 links
Journal of Information & Knowledge Management
Topic Modeling
article

Automated Text Coherence Analysis through Semantic Subspace Modelling and Ensemble Learning

Peeta Basa Pati, Samrudhi S. Kamble, Sanjivani P. Shendre, M. Yagnasri Priya, Swetha Sridhar
article en

Abstract

Coherence is fundamental to producing clear and logical academic writing. Despite its importance, most automated evaluation systems primarily target grammar and style, often overlooking this critical discourse-level quality. This study presents a comprehensive comparative analysis of multiple computational approaches like classical machine learning, recurrent and convolutional neural networks (RNNs and CNNs), transformer-based architectures (BERT, RoBERTa, SBERT), and graph-based and hybrid models, specifically for coherence assessment in academic texts. Using a labelled academic writing dataset, model performance was benchmarked using accuracy, precision, recall and F1-scores to evaluate both training and test generalisation. Among classical models trained on TF-IDF and topic-based features, ensemble methods, such as Random Forest and XGBoost, demonstrated stronger robustness than linear and probabilistic baselines, while the nonlinear Multi-Layer Perceptron (MLP) achieved a test F1-score of 76.97%. Deep neural models exhibited stable performance, with the BiLSTM achieving a 76% test F1-score, outperforming LSTM, GRU and CNN models. Graph-based approaches integrating knowledge graph features improved generalisation, reaching a 76.64% test F1-score. The best overall performance was achieved by a hybrid model combining RoBERTa embeddings with Random Forest, attaining an 81.64% test F1-score. These findings indicate that integrating deep semantic embeddings with well-regularised ensemble classifiers offers the most effective and interpretable strategy for automated coherence evaluation.

Journal of Information & Knowledge Management
Amrita Vishwa Vidyapeetham (IN)
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon