Explainable Job Category Detection Using DistilBERT: A Three-Class Transformer Model for Classifying Real, Fake, and Invalid Online Job Offers

Abstract Online job portals have widened access to employment, but they have also become a primary channel for Online Recruitment Fraud (ORF), in which fake job postings are used to extract money, harvest personal data, or recruit victims into illicit schemes. Most fake job detection systems treat the task as binary Real/Fake classification, which forces incomplete, nonsensical, or spam-like postings into the wrong class and makes them less reliable in practice. This study develops and evaluates an explainable, lightweight DistilBERT model that classifies online job offers as Real, Fake, or Invalid, and deploys it as a real-time web application. The distilbert-base-uncased classifier was fine-tuned on a corpus that combines the Employment Scam Aegean Dataset (EMSCAD), the job-scam domain of the DIFrauD benchmark, and a researcher-curated Invalid Token dataset. The combined corpus of 34,325 records was highly imbalanced (21.8:1), so it was rebalanced to 3,000 records per class through controlled undersampling and textual augmentation instead of SMOTE, and balanced class weighting was kept during training. The model was trained for four epochs with AdamW (learning rate 2 × 10⁻⁵) and a 256-token sequence length on a stratified 70:15:15 train–validation–test split, with the best checkpoint chosen by validation loss, and was then served through a FastAPI backend with SHAP-based token-level explanations. On a held-out test set of 1,350 samples, the model achieved 98.74% accuracy and 98.74% balanced accuracy and misclassified only 17 samples. It reached F1-scores of 0.98, 1.00, and 0.98 and one-versus-rest ROC-AUC values of 0.998, 1.000, and 0.998 for the Fake, Invalid, and Real classes, respectively, and the deployed system returned predictions in about 1.0 to 2.9 seconds. These results show that a distilled transformer can deliver accurate, efficient, and interpretable three-class fake job detection, and that the explicit Invalid category and SHAP explanations take automated job offer screening beyond conventional binary approaches.