Six Methodological Pitfalls in YouTube Comment Extraction: What a rendering decision does to a research corpus.

Research on online comment interaction increasingly relies on reply structure: who answers whom, how deep a thread runs, which turns follow which. That structure is not observed. It is reconstructed twice — once by the platform when it stores a reply, and once by the extraction tool when it reads it back. This paper documents six points at which that reconstruction misleads, using four independently collected YouTube corpora (29 focal videos, 196,525 comments) and the author's own prior measurement work. The central finding concerns nesting depth. Interface-level extraction returns four levels of comment structure and no more, while the official API returns two. Neither is a defect: the four-level limit is a rendering decision, and the platform behaves as designed. But the limit becomes a property of the extracted corpus, where its origin is invisible. Where the limit does not bind, at depth 2, the platform-inserted address marker and the recorded parent agree in 13,439 of 13,443 cases (99.97%). One level down, where it binds, they disagree in 45.8% of cases. The divergence follows from where a comment sits in the structure, not from what its author did. The paper further documents coarse timestamp reconstruction, handle-migration effects on name resolution, orphan re-parenting after deletion, and collectors that silently return only top-level comments. The sixth pitfall is methodological: a pre-registered kill test on 100 randomly drawn divergent cases correctly classified 78 of them as genuine, and the conclusion drawn from that coding was nevertheless wrong. Case-level coders with full thread context cannot see what the platform did to the tree. This deposit contains the manuscript and a companion archive with the result tables behind every figure, the analysis scripts (Python standard library only), and the figures. No comment text is redistributed. The paper explicitly invites refutation and names three falsifiable targets; see the deposit README.