When Six Respondents Make the Model: Extreme-Response Clusters and the Fragility of PLS-SEM Results in Tourism Technology Adoption Research

Tourism technology adoption studies rely heavily on Likert-scale surveys analyzed with partial least squares structural equation modelling (PLS-SEM). Such data are often strongly skewed towards agreement. This study shows how a very small cluster of uniformly negative respondents can manufacture the reliability, validity, structural, predictive and configurational results that researchers report, and how the standard diagnostics fail to detect it. Survey data from 304 Malaysian respondents on the adoption of a solar-powered tourist boat (the ePenambang, Kuala Terengganu) were analyzed with an extended Technology Acceptance Model. Every analysis was run with and without six respondents who disagreed with all 35 items. A dose-response analysis, bootstrap composition analysis, permutation null models and a Monte Carlo simulation (2,000 replications) were used to separate substantive signal from the influence of the extreme cluster. With all 304 cases, the model met every conventional criterion: α = 0.81–0.91, all ten paths significant and R² = 0.85 for intention. Without the six cases (2% of the sample), α fell to between −0.60 and 0.61, nine of ten paths became non-significant and R² fell to 0.24. Not one of 1,000 bootstrap resamples excluded all six cases. Necessary condition effect sizes fell from 0.46–0.72 to 0–0.11, a multigroup difference by education vanished, and fsQCA solutions were reproduced by a randomly permuted outcome. In simulation, adding six all-disagree cases to pure noise produced R² = 0.83, α = 0.84 and HTMT above 0.90 for every construct pair. Standard careless-response indices did not flag the six cases; a multivariate outlier screen did. Researchers, reviewers and editors in tourism should require sensitivity analyses that report results with and without uniformly extreme respondents, report within-cluster correlations, and treat very high R² and reliability in ceiling-affected data as a warning rather than a strength. A ten-point reporting checklist is provided. To our knowledge, this is the first study to document, with real tourism data and simulation, how a small extreme-response cluster propagates through the full modern PLS-SEM toolkit, including NCA, fsQCA, MICOM/MGA, IPMA and PLSpredict.

Authors

Institutions

Publication Details

Journal
International Journal of Research and Innovation in Social Science
Published
2026-11-16
DOI
https://doi.org/10.47772/ijriss.2026.1017psy0073
Primary Topic
Psychometric Methodologies and Testing
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

When Six Respondents Make the Model: Extreme-Response Clusters and the Fragility of PLS-SEM Results in Tourism Technology Adoption Research

Nur Zafirah A. Khadar, Fadhlur Rahim Azmi, Haslinda Musa, NURULIZWA ABDUL RASHID et al.
International Journal of Research and Innovation in Social Science
Psychometric Methodologies and Testing
article

When Six Respondents Make the Model: Extreme-Response Clusters and the Fragility of PLS-SEM Results in Tourism Technology Adoption Research

Nur Zafirah A. Khadar, Fadhlur Rahim Azmi, Haslinda Musa, NURULIZWA ABDUL RASHID, Samer Ali Hussein Al-Shami
article en

Abstract

Tourism technology adoption studies rely heavily on Likert-scale surveys analyzed with partial least squares structural equation modelling (PLS-SEM). Such data are often strongly skewed towards agreement. This study shows how a very small cluster of uniformly negative respondents can manufacture the reliability, validity, structural, predictive and configurational results that researchers report, and how the standard diagnostics fail to detect it. Survey data from 304 Malaysian respondents on the adoption of a solar-powered tourist boat (the ePenambang, Kuala Terengganu) were analyzed with an extended Technology Acceptance Model. Every analysis was run with and without six respondents who disagreed with all 35 items. A dose-response analysis, bootstrap composition analysis, permutation null models and a Monte Carlo simulation (2,000 replications) were used to separate substantive signal from the influence of the extreme cluster. With all 304 cases, the model met every conventional criterion: α = 0.81–0.91, all ten paths significant and R² = 0.85 for intention. Without the six cases (2% of the sample), α fell to between −0.60 and 0.61, nine of ten paths became non-significant and R² fell to 0.24. Not one of 1,000 bootstrap resamples excluded all six cases. Necessary condition effect sizes fell from 0.46–0.72 to 0–0.11, a multigroup difference by education vanished, and fsQCA solutions were reproduced by a randomly permuted outcome. In simulation, adding six all-disagree cases to pure noise produced R² = 0.83, α = 0.84 and HTMT above 0.90 for every construct pair. Standard careless-response indices did not flag the six cases; a multivariate outlier screen did. Researchers, reviewers and editors in tourism should require sensitivity analyses that report results with and without uniformly extreme respondents, report within-cluster correlations, and treat very high R² and reliability in ceiling-affected data as a warning rather than a strength. A ten-point reporting checklist is provided. To our knowledge, this is the first study to document, with real tourism data and simulation, how a small extreme-response cluster propagates through the full modern PLS-SEM toolkit, including NCA, fsQCA, MICOM/MGA, IPMA and PLSpredict.

International Journal of Research and Innovation in Social ScienceVol. 10(17)
Technical University of Malaysia Malacca (MY)
Universiti Teknikal Malaysia Melaka
Psychometric Methodologies and Testing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon