Using artificial intelligence (AI) to spread misinformation and fake content within social media posts

The purpose of this study was to explore how Artificial Intelligence (AI) can be used to generate and automate social media content for spreading misinformation and fake content.While prior research has examined the ethical and societal impact of misinformation, few studies have tested the real-world effectiveness of fully automated AI generated posts compared to those reviewed with human oversight.To address this gap, two experimental models were developed: Model 1 used a fully automated AI workflow, while Model 2 combined AI generated content with human interaction.Data was collected from Instagram and Meta between February 12th, 2025, and ending April 20th, 2025.Both models posted inspirational content with images, but Model 2, which featured human refinement and engagement, saw significantly higher user interaction, including friend requests, messages, and promotional offers.In contrast, Model 1 experienced low engagement and technical issues.The results highlight the importance of human oversight in boosting credibility and interaction with AI generated content.This study contributes to the understanding of misinformation by demonstrating how different levels of automation influence the content reach, user behavior, and potential ethical risks, emphasizing the need for platform regulation, and responsible AI usage.

Publication Details

Journal
Issues in Information Systems
Published
2025-01-01
DOI
https://doi.org/10.48009/4_iis_2025_126
Primary Topic
Misinformation and Its Impacts
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Using artificial intelligence (AI) to spread misinformation and fake content within social media posts

Issues in Information Systems
Misinformation and Its Impacts
article

Using artificial intelligence (AI) to spread misinformation and fake content within social media posts

article en

Abstract

The purpose of this study was to explore how Artificial Intelligence (AI) can be used to generate and automate social media content for spreading misinformation and fake content.While prior research has examined the ethical and societal impact of misinformation, few studies have tested the real-world effectiveness of fully automated AI generated posts compared to those reviewed with human oversight.To address this gap, two experimental models were developed: Model 1 used a fully automated AI workflow, while Model 2 combined AI generated content with human interaction.Data was collected from Instagram and Meta between February 12th, 2025, and ending April 20th, 2025.Both models posted inspirational content with images, but Model 2, which featured human refinement and engagement, saw significantly higher user interaction, including friend requests, messages, and promotional offers.In contrast, Model 1 experienced low engagement and technical issues.The results highlight the importance of human oversight in boosting credibility and interaction with AI generated content.This study contributes to the understanding of misinformation by demonstrating how different levels of automation influence the content reach, user behavior, and potential ethical risks, emphasizing the need for platform regulation, and responsible AI usage.

Issues in Information Systems
Openalex Percentile: Top 67%
Misinformation and Its Impacts
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Using artificial intelligence (AI) to spread misinformation and fake content within social media posts · Issues in Information Systems (2025) | TGRS Research Map | TGRS