Adaptive Transformer Architectures for Robust Clickbait Detection and Semantic Understanding Across Varying Input Lengths

Sensational or misrepresentative content-based clickbait headlines aimed at capturing the user's attention and producing clicks have made their way ubiquitously through online media, and detection becomes imperative to make the content better quality and provide an authentic user experience. This contribution presents a new practical method for detecting clickbait through a hybrid transformer-based framework utilizing Distilled Bidirectional Encoder Representations from Transformers (DistilBERT) and Big Transformer (BigBird), which was optimized for lengths of input with varying lengths. Unlike conventional single-model approaches, this study dynamically selects the optimal model based on sequence length—DistilBERT for shorter texts (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\leq 256$</tex> tokens) and BigBird for longer texts (>256 tokens)—ensuring computational efficiency without sacrificing accuracy. The models were fine-tuned using a dataset of 32,000 headlines balanced between clickbait and non-clickbait, employing cross-entropy loss and the Adam with Weight Decay optimizer to prevent overfitting. The hybrid model demonstrated noticeable convergence through three training epochs, with training loss decreasing from 0.2051 to 0.0003 and test accuracy consistently at 98.78 %. The adaptive method suggested in this study facilitates a scalable and effective clickbait detection solution, better than conventional transformer-based models, and resolves the problems associated with handling various text lengths. The results affirm the efficiency of the hybrid strategy in dealing with varied text lengths.