Trends in sociology research methods in the 21st century: Automated journal article coding using large language models

Evidence on the methodological evolution of sociology in the 21st century is limited. Useful signals exist in the literature, but we lack a temporally extended description of how the discipline’s practice has changed across the period in which the data and method environment was most transformed. The constraint is the cost of hand-coding journal articles, capping feasible corpus sizes well below what disciplinary description requires. We use locally hosted large language models (LLMs) to break this constraint, validating their performance before applying the pipeline to nearly 20,000 full-text articles from 28 sociology journals over 2004–2024. We document a steady empirical turn, near-parity between quantitative and qualitative work, and a wholesale shift of survey administration online. These patterns bear on Savage and Burrows’ diagnosis of a ‘coming crisis of empirical sociology’: contrary to their expectations, the survey has held its share and the interview has grown, while big data has remained marginal.

Authors

Institutions

Publication Details

Journal
Sociology
Published
2026-11-20
DOI
https://doi.org/10.1177/00380385261484085
Primary Topic
Computational and Text Analysis Methods
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Trends in sociology research methods in the 21st century: Automated journal article coding using large language models

Ian Brunton‐Smith, Patrick Sturgis
Sociology
Computational and Text Analysis Methods
article

Trends in sociology research methods in the 21st century: Automated journal article coding using large language models

Ian Brunton‐Smith, Patrick Sturgis
article en

Abstract

Evidence on the methodological evolution of sociology in the 21st century is limited. Useful signals exist in the literature, but we lack a temporally extended description of how the discipline’s practice has changed across the period in which the data and method environment was most transformed. The constraint is the cost of hand-coding journal articles, capping feasible corpus sizes well below what disciplinary description requires. We use locally hosted large language models (LLMs) to break this constraint, validating their performance before applying the pipeline to nearly 20,000 full-text articles from 28 sociology journals over 2004–2024. We document a steady empirical turn, near-parity between quantitative and qualitative work, and a wholesale shift of survey administration online. These patterns bear on Savage and Burrows’ diagnosis of a ‘coming crisis of empirical sociology’: contrary to their expectations, the survey has held its share and the interview has grown, while big data has remained marginal.

Sociology
University of Surrey (GB), London School of Economics and Political Science (GB)
Computational and Text Analysis Methods
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon
Trends in sociology research methods in the 21st century: Automated journal article coding using large language models — Ian Brunton‐Smith, Patrick Sturgis · Sociology (2026) | TGRS Research Map | TGRS