Benchmarking the Double-Entry Intraclass Correlation as an Index of the Similarity between Correlation Profiles

To infer whether the test scores subjected to analyses tap into the same construct (or opposite ones), measured as the similarity between sets of correlations with other variables, researchers often use the Double-Entry Intraclass Correlation (ICCDE). Importantly, the ICCDE has been developed as an index of the similarity between profiles consisting of unbounded coefficients (e.g., profiles of personality features), whereas the correlation metric conveys a series of challenges to the applicability of the ICCDE. Using simulation studies, this research benchmarked the applicability of the ICCDE to correlation profiles under various conditions regarding characteristics of the correlation profiles subjected to analysis. More specifically, equality versus inequality of the patterns of correlation trajectories (shape), levels of variance of the compared correlation profiles (scatter), and mean levels of the profiles (elevation) were manipulated along with the level of variance within the correlation profiles. Simulation results indicate that ICCDE values can be strongly influenced by profile shape and restricted within-profile variance. These findings suggest that conclusions about similarity between correlation profiles may depend on characteristics of the compared profiles and have important implications for evaluating jingle and jangle fallacies.

Authors

Publication Details

Journal
PsyArXiv (OSF Preprints)
Published
2026-11-16
DOI
https://doi.org/10.31234/osf.io/cq2ma
Primary Topic
Psychometric Methodologies and Testing
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Benchmarking the Double-Entry Intraclass Correlation as an Index of the Similarity between Correlation Profiles

Christian Blötner
PsyArXiv (OSF Preprints)
Psychometric Methodologies and Testing
preprint

Benchmarking the Double-Entry Intraclass Correlation as an Index of the Similarity between Correlation Profiles

Christian Blötner
preprint en

Abstract

To infer whether the test scores subjected to analyses tap into the same construct (or opposite ones), measured as the similarity between sets of correlations with other variables, researchers often use the Double-Entry Intraclass Correlation (ICCDE). Importantly, the ICCDE has been developed as an index of the similarity between profiles consisting of unbounded coefficients (e.g., profiles of personality features), whereas the correlation metric conveys a series of challenges to the applicability of the ICCDE. Using simulation studies, this research benchmarked the applicability of the ICCDE to correlation profiles under various conditions regarding characteristics of the correlation profiles subjected to analysis. More specifically, equality versus inequality of the patterns of correlation trajectories (shape), levels of variance of the compared correlation profiles (scatter), and mean levels of the profiles (elevation) were manipulated along with the level of variance within the correlation profiles. Simulation results indicate that ICCDE values can be strongly influenced by profile shape and restricted within-profile variance. These findings suggest that conclusions about similarity between correlation profiles may depend on characteristics of the compared profiles and have important implications for evaluating jingle and jangle fallacies.

PsyArXiv (OSF Preprints)
Psychometric Methodologies and Testing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon