Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era

Is AI taking over software engineering research? By 2025, AI appeared in more than half of the research-track papers at ICSE, FSE, and ASE. Yet ubiquity alone cannot show whether AI expanded the publication portfolio, displaced other work, or redirected the intellectual agenda. We investigate this shift through a census of 4,790 research papers (three SE conferences; 2018--2026) and free-form responses from 244 SE researchers discussing the future of the SE research community (ICSE 2026 FoSE survey). We found that while AI is taking over, it is not simply replacing non-AI research; it is unevenly reorganizing SE research around its capabilities. AI accounted for 93.1% of publication growth, but expansion in some venues and problem areas concealed contraction in others. The LLM wave also changed the field's tools more than its problem boundaries: 92.6% of LLM-wave AI papers revisit established SE problems, while most new problems arise from the introduction of LLMs in the SE process themselves. Engagement with AI remains predominantly downstream: 95.7% of AI papers apply AI or evaluate and support its use, while only 4.3% interrogate the paradigm or develop alternatives. The FoSE survey reveals a similar imbalance between recognition and response. AI-driven change was the second-most frequent concern about the field's future 20.2%, yet only 7.0% of responses mentioned AI when researchers were asked to identify one change the community should make. Together, our findings reveal a field reorganizing an inherited agenda around AI faster than it is questioning that reorganization. We discuss how SE can respond by building cumulatively across technological waves, strengthening evaluation and assurance, developing SE-native foundations, and studying software engineering in an AI-transformed world.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-03
DOI
https://doi.org/10.48550/arxiv.2611.04206
Primary Topic
Software Engineering Research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era

Souti Chattopadhyay, Sadra Sabouri, Zeinadsadat Saghi
arXiv (Cornell University)
Software Engineering Research
preprint

Are We Just Applying LLMs? What SE Research Studies, Underexplores, and Could Do Next in the LLM Era

Souti Chattopadhyay, Sadra Sabouri, Zeinadsadat Saghi
preprint en

Abstract

Is AI taking over software engineering research? By 2025, AI appeared in more than half of the research-track papers at ICSE, FSE, and ASE. Yet ubiquity alone cannot show whether AI expanded the publication portfolio, displaced other work, or redirected the intellectual agenda. We investigate this shift through a census of 4,790 research papers (three SE conferences; 2018--2026) and free-form responses from 244 SE researchers discussing the future of the SE research community (ICSE 2026 FoSE survey). We found that while AI is taking over, it is not simply replacing non-AI research; it is unevenly reorganizing SE research around its capabilities. AI accounted for 93.1% of publication growth, but expansion in some venues and problem areas concealed contraction in others. The LLM wave also changed the field's tools more than its problem boundaries: 92.6% of LLM-wave AI papers revisit established SE problems, while most new problems arise from the introduction of LLMs in the SE process themselves. Engagement with AI remains predominantly downstream: 95.7% of AI papers apply AI or evaluate and support its use, while only 4.3% interrogate the paradigm or develop alternatives. The FoSE survey reveals a similar imbalance between recognition and response. AI-driven change was the second-most frequent concern about the field's future 20.2%, yet only 7.0% of responses mentioned AI when researchers were asked to identify one change the community should make. Together, our findings reveal a field reorganizing an inherited agenda around AI faster than it is questioning that reorganization. We discuss how SE can respond by building cumulatively across technological waves, strengthening evaluation and assurance, developing SE-native foundations, and studying software engineering in an AI-transformed world.

arXiv (Cornell University)
Software Engineering Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon