Can AI Scale Human Fact-Checking? Evidence from Community Notes

Community Notes---a crowd-based fact-checking program through which social media users can flag a post with a note explaining why the post is misleading, and the note is displayed if a diverse set of users rate the note as helpful---has become central to online content moderation. The program, however, faces substantial scalability constraints, as the generation of notes takes considerable time and effort, and the number of user participants is limited. Here, we investigate whether allowing AI models to write Community Notes helps to alleviate these constraints by analyzing the full corpus of AI-generated Community Notes on X through May 2026. Encouragingly, AI-generated notes are submitted more quickly, are longer, and cite more and higher-quality sources than human-written notes, while being rated helpful at a similar rate and being associated with similar decreases in reposts following the publication of a rated-helpful note. Furthermore, a pre-registered survey experiment finds no statistically detectable difference between AI- and human-generated fact-checks in their effect on belief updating, regardless of whether the author was revealed to participants. On the other hand, AI-generated note writing is highly concentrated, as a handful of accounts produce the large majority of notes, and are significantly less likely to engage with posts involving controversial and polarized topics such as politics and elections. While those findings suggest that AI may be a powerful tool for scaling crowd-based fact-checking systems such as Community Notes, this evolution also entails important risks: shifting the primary bottleneck from note production to note evaluation, moving the system from a decentralized toward a more centralized model, and creating incentives to prioritize claims that are more likely to achieve consensus over those of greater societal importance.

Authors

Institutions

Publication Details

Published
2026-11-02
DOI
https://doi.org/10.31234/osf.io/n23zb_v1
Primary Topic
Misinformation and Its Impacts
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Can AI Scale Human Fact-Checking? Evidence from Community Notes

Mohsen Mosleh, David Gertler Rand, Thomas Renault
1 links
Misinformation and Its Impacts
preprint

Can AI Scale Human Fact-Checking? Evidence from Community Notes

Mohsen Mosleh, David Gertler Rand, Thomas Renault
preprint en

Abstract

Community Notes---a crowd-based fact-checking program through which social media users can flag a post with a note explaining why the post is misleading, and the note is displayed if a diverse set of users rate the note as helpful---has become central to online content moderation. The program, however, faces substantial scalability constraints, as the generation of notes takes considerable time and effort, and the number of user participants is limited. Here, we investigate whether allowing AI models to write Community Notes helps to alleviate these constraints by analyzing the full corpus of AI-generated Community Notes on X through May 2026. Encouragingly, AI-generated notes are submitted more quickly, are longer, and cite more and higher-quality sources than human-written notes, while being rated helpful at a similar rate and being associated with similar decreases in reposts following the publication of a rated-helpful note. Furthermore, a pre-registered survey experiment finds no statistically detectable difference between AI- and human-generated fact-checks in their effect on belief updating, regardless of whether the author was revealed to participants. On the other hand, AI-generated note writing is highly concentrated, as a handful of accounts produce the large majority of notes, and are significantly less likely to engage with posts involving controversial and polarized topics such as politics and elections. While those findings suggest that AI may be a powerful tool for scaling crowd-based fact-checking systems such as Community Notes, this evolution also entails important risks: shifting the primary bottleneck from note production to note evaluation, moving the system from a decentralized toward a more centralized model, and creating incentives to prioritize claims that are more likely to achieve consensus over those of greater societal importance.

Massachusetts Institute of Technology (US)
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon