Bilibili/TikTok videos as sources of health information about atrial septal defect: a content and quality analysis

BACKGROUND: The proliferation of short-form educational video platforms has facilitated the public's access to health information; however, no research has assessed the characteristics and quality of videos related to atrial septal defect. ASD was selected because it is one of the most common congenital heart defects encountered across the lifespan, often requiring repeated explanations regarding diagnosis, follow-up, timing of intervention, and long-term prognosis for patients and families. In addition, although short-form video studies have been conducted for several other diseases, ASD-related content on major Chinese short-video platforms has not been systematically evaluated. This study aimed to evaluate the quality and reliability of short videos related to atrial septal defect on TikTok and Bilibili. METHODS: The Chinese term "atrial septal defect" was used to search for related videos on TikTok and Bilibili, and a predefined sampling strategy was used to screen the first 100 algorithm-ranked videos from each platform on October 21, 2025. This sample size was determined a priori to provide a feasible and standardized cross-sectional sample for manual content evaluation and to maintain comparability between platforms, rather than being based on a formal sample size calculation. Duplicate, irrelevant videos, and videos published within seven days were excluded to reduce the instability of early engagement indicators. As a result, 155 videos were included for analysis. The overall quality of these videos was assessed using the Global Quality Score (GQS), VIQI, PEMAT, JAMA Benchmark, and the DISCERN tool. These instruments were used to evaluate educational quality, reliability, transparency, understandability, and actionability, but they did not constitute a direct assessment of factual accuracy or potentially harmful medical content. Interobserver reliability was assessed for the independently retained duplicate GQS ratings using quadratic weighted kappa. Because social media search results are algorithm-ranked and may be affected by platform personalization, the included sample should be interpreted as a snapshot of highly visible ASD-related videos on the sampling day rather than an exhaustive representation of all available videos. RESULTS: Among the 155 videos (Bilibili: n = 70; TikTok: n = 85), significant differences were observed across all engagement and quality metrics. TikTok videos demonstrated significantly higher values for likes (median 415.0 [IQR 186.0 to 961.0] vs. 11 [IQR 5.0 to 52.75]), collections (131.0 [IQR 50.0 to 332.0] vs. 26.5 [IQR 8.0 to 106.75]), shares (124.0 [IQR 35.0 to 583.0] vs. 7.5 [IQR 2.0 to 29.0]), and comments (61.0 [IQR 17.0 to 492.0] vs. 1.0 [IQR 0.0 to 9.5]) (all P < 0.05). In contrast, the Bilibili group had longer video durations (in seconds) (639.0 [IQR 327.0 to 1,106.0] vs. 326.0 [IQR 216.9 to 429.0]; P < 0.001) and longer times since upload (in days) (356.0 [IQR 71.25 to 770.25] vs. 60.0 [IQR 37.0 to 102.0]; P < 0.001). Content quality assessments also differed, with TikTok videos having higher median DISCERN scores (23.788 vs. 22.786; P = 0.001), JAMA_Benchmark (1.918 vs. 1.714; P = 0.001) and GQS scores (3.635 vs. 3.314]; P = 0.006). However, the proportion of professional uploaders differed markedly between platforms (TikTok: 96.47% vs. Bilibili: 44.29%), and uploader-level analyses suggested that part of the observed quality advantage on TikTok may be attributable to uploader composition rather than platform characteristics alone. Importantly, this conclusion was based on the absolute position of the observed scores on their respective validated scales rather than on an arbitrary composite cutoff. Specifically, median GQS values of 3.314 and 3.635 on a 1-5 scale indicate moderate rather than high educational quality, whereas median JAMA Benchmark values of 1.714 and 1.918 on a 0-4 scale indicate that, on average, fewer than half of the transparency/reliability criteria were met. Because no universally accepted single "target quality score" exists across GQS, JAMA, DISCERN, PEMAT, and VIQI, each instrument was interpreted according to its own published scale direction and anchors. CONCLUSIONS: TikTok videos related to atrial septal defect are more engaging and of higher content quality than those on Bilibili. Overall, health-related videos on both platforms showed only moderate educational quality, with notable limitations in transparency, reliability, source attribution, and actionable guidance. Professionally produced content tended to perform better, although between-platform differences should be interpreted cautiously because of differences in uploader composition. These findings suggest that greater participation by health professionals may help improve the quality and reliability of online health information. As this study assessed informational quality rather than factual accuracy and was limited to Chinese-language videos from two platforms at a single time point, the results should be interpreted as a platform- and time-specific snapshot.