LLM Assisted Annotation Framework for Cross Platform Analysis of Autism Online Communities: Implications for Parent Education and Digital Support (Preprint)

<sec> <title>BACKGROUND</title> Online health communities (OHCs) are key channels for families of children with autism spectrum disorder (ASD) to obtain information and psychosocial support. Platform heterogeneity, as reflected in differences between open peer forums and physician and patient portals, may shape parental decisions, yet comparable evidence from China remains limited. Large language models (LLMs) can accelerate transparent, scalable topic coding and help surface caregiver-relevant content at scale. </sec> <sec> <title>OBJECTIVE</title> To characterize health information needs, topic distributions, and poster roles across an open forum (Baidu Tieba) and a physician and patient platform (Chunyu Doctor) under a unified schema, and to implement and validate an LLM assisted, human in the loop annotation workflow oriented to caregiver education and service navigation. </sec> <sec> <title>METHODS</title> We implemented an innovative LLM assisted annotation framework for large scale content analysis. We analyzed an open community (8,097 threads; 68,576 replies; 76,284 nested replies; 11,118 users) and a physician and patient platform (197 consultations; 4,639 dialogue turns). Using human open coding, we first developed a unified taxonomy for topics and poster identities. An LLM assisted annotation framework was implemented using DeepSeek to generate candidate topic and identity labels under the unified schema; all outputs were verified and corrected by human coders to produce the final dataset. </sec> <sec> <title>RESULTS</title> In the open forum, family caregivers were the primary posters (68.24% , 2,432/3,564), with visible commercial rehabilitation actors (13.05%, 465/3,564) and few self-reported autistic individuals (2.50%, 89/3,564). Dominant topics were “Sharing/Case sharing” (27.47%, 2,171/7,904) and “Science popularization” (14.25%, 1,126/7,904); help-seeking concentrated on “Autism diagnosis” (13.09%, 1,035/7,904). On the physician–patient platform, medical topics focused on symptoms, treatment, diagnosis, and examinations, while “psychological support” remained salient (11.53%, 253/2,194). </sec> <sec> <title>CONCLUSIONS</title> The analysis identifies (1) caregiver dominated participation in open forums, (2) focused clinical questioning in physician and patient consultations, and (3) noticeable activity from commercial rehabilitation providers in open spaces. In light of these findings, we recommend establishing a transparent evaluation mechanism for commercial rehabilitation services and strengthening linkages between doctor and patient exchange platforms and open forums to improve topic quality and caregiver access to reliable, plain language materials. The LLM assisted, human verified workflow proved feasible and transparent for large scale OHC analysis in this context and can support subsequent content curation and moderation; all reported results derive from human confirmed labels rather than automated decisions. </sec>

Authors

Publication Details

Published
2025-10-04
DOI
https://doi.org/10.2196/preprints.85290
Primary Topic
Literature Analysis and Criticism
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

LLM Assisted Annotation Framework for Cross Platform Analysis of Autism Online Communities: Implications for Parent Education and Digital Support (Preprint)

Wei Feng, Yu Chen, Lei Zhang, Changwei Zhang et al.
Literature Analysis and Criticism
article

LLM Assisted Annotation Framework for Cross Platform Analysis of Autism Online Communities: Implications for Parent Education and Digital Support (Preprint)

Wei Feng, Yu Chen, Lei Zhang, Changwei Zhang, Xuening Zhang, Yifan Xu, Yixue Liu, Ruochen Huang
article en

Abstract

<sec> <title>BACKGROUND</title> Online health communities (OHCs) are key channels for families of children with autism spectrum disorder (ASD) to obtain information and psychosocial support. Platform heterogeneity, as reflected in differences between open peer forums and physician and patient portals, may shape parental decisions, yet comparable evidence from China remains limited. Large language models (LLMs) can accelerate transparent, scalable topic coding and help surface caregiver-relevant content at scale. </sec> <sec> <title>OBJECTIVE</title> To characterize health information needs, topic distributions, and poster roles across an open forum (Baidu Tieba) and a physician and patient platform (Chunyu Doctor) under a unified schema, and to implement and validate an LLM assisted, human in the loop annotation workflow oriented to caregiver education and service navigation. </sec> <sec> <title>METHODS</title> We implemented an innovative LLM assisted annotation framework for large scale content analysis. We analyzed an open community (8,097 threads; 68,576 replies; 76,284 nested replies; 11,118 users) and a physician and patient platform (197 consultations; 4,639 dialogue turns). Using human open coding, we first developed a unified taxonomy for topics and poster identities. An LLM assisted annotation framework was implemented using DeepSeek to generate candidate topic and identity labels under the unified schema; all outputs were verified and corrected by human coders to produce the final dataset. </sec> <sec> <title>RESULTS</title> In the open forum, family caregivers were the primary posters (68.24% , 2,432/3,564), with visible commercial rehabilitation actors (13.05%, 465/3,564) and few self-reported autistic individuals (2.50%, 89/3,564). Dominant topics were “Sharing/Case sharing” (27.47%, 2,171/7,904) and “Science popularization” (14.25%, 1,126/7,904); help-seeking concentrated on “Autism diagnosis” (13.09%, 1,035/7,904). On the physician–patient platform, medical topics focused on symptoms, treatment, diagnosis, and examinations, while “psychological support” remained salient (11.53%, 253/2,194). </sec> <sec> <title>CONCLUSIONS</title> The analysis identifies (1) caregiver dominated participation in open forums, (2) focused clinical questioning in physician and patient consultations, and (3) noticeable activity from commercial rehabilitation providers in open spaces. In light of these findings, we recommend establishing a transparent evaluation mechanism for commercial rehabilitation services and strengthening linkages between doctor and patient exchange platforms and open forums to improve topic quality and caregiver access to reliable, plain language materials. The LLM assisted, human verified workflow proved feasible and transparent for large scale OHC analysis in this context and can support subsequent content curation and moderation; all reported results derive from human confirmed labels rather than automated decisions. </sec>

Openalex Percentile: Top 61%
Literature Analysis and Criticism
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.