Examining bias perpetuation in academic search engines: An algorithm audit of Google and Semantic Scholar

Researchers rely on academic Web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in queries. This study examines whether confirmation biased queries prompted into Google Scholar and Semantic Scholar will yield results aligned with a query’s bias. Six queries (topics across health and technology domains such as ‘vaccines’, ‘Internet use’) were analyzed for disparities in search results. We confirm that biased queries (targeting ‘benefits’ or ‘risks’) affect search results in line with bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more disparate results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.

Authors

Institutions

Publication Details

Journal
First Monday
Published
2024-11-03
DOI
https://doi.org/10.5210/fm.v29i11.13730
Citations
4
Primary Topic
Advanced Text Analysis Techniques
Type
article
Field-Weighted Citation Impact
1.39
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Examining bias perpetuation in academic search engines: An algorithm audit of Google and Semantic Scholar

Mona Bielig, Roberto Ulloa, Celina Kacperski, Maryna Sydorova et al.
4 citations
First Monday
Advanced Text Analysis Techniques
1.39
article

Examining bias perpetuation in academic search engines: An algorithm audit of Google and Semantic Scholar

Mona Bielig, Roberto Ulloa, Celina Kacperski, Maryna Sydorova, Mykola Makhortykh
article en
4 citations

Abstract

Researchers rely on academic Web search engines to find scientific sources, but search engine mechanisms may selectively present content that aligns with biases embedded in queries. This study examines whether confirmation biased queries prompted into Google Scholar and Semantic Scholar will yield results aligned with a query’s bias. Six queries (topics across health and technology domains such as ‘vaccines’, ‘Internet use’) were analyzed for disparities in search results. We confirm that biased queries (targeting ‘benefits’ or ‘risks’) affect search results in line with bias, with technology-related queries displaying more significant disparities. Overall, Semantic Scholar exhibited fewer disparities than Google Scholar. Topics rated as more polarizing did not consistently show more disparate results. Academic search results that perpetuate confirmation bias have strong implications for both researchers and citizens searching for evidence. More research is needed to explore how scientific inquiry and academic search engines interact.

First Monday
University of Bern (CH), University of Konstanz (DE), Seeburg Castle University (AT), GESIS - Leibniz Institute for the Social Sciences (DE)
Openalex Percentile: Top 15%
Advanced Text Analysis Techniques
1.39
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.