Phishibl: A Systematic Evaluation Of Instance-Based Learning Model For Predicting Phishing Susceptibility

Despite enormous efforts to develop defenses against phishing attacks, humans still struggle to detect phishing emails given the constantly evolving attacker strategies. This thesis aims to test the predictive capabilities of a cognitive model that represents the individual susceptibility to phishing emails. While training programs aim to raise awareness, most remain outdated and ineffective against evolving attack strategies. Recent advances in Machine Learning, Artificial Intelligence, and Large Language Models (LLMs) offer new defenses, yet understanding human decision processes remains crucial, as effective systems must emulate how people evaluate unfamiliar emails based on prior experience. This research introduces a cognitive model grounded in Instance-Based Learning (IBL) Theory, which captures how individuals compare new emails to past encounters. We evaluated five cognitive model variants against human email classification data. To examine the role of experience representation, we tested three approaches: IBL-Forecaster, which predicts decisions without prior experience to assess whether a model can learn from scratch; IBL-Human-Assisted, which begins with human experiences to evaluate how prior knowledge improves prediction; and IBL-Foregone, where model can learns not only from its actual experiences but also from the counterfactual outcomes. On the other hand, to develop more personalized model, we tested two additional variants: IBL-Tracing, which updates memory based on human decisions, and IBL-Tracing Calibration, which further tunes the model by optimizing the decay parameter after tracing. IBL-Tracing Calibration achieved the highest alignment (81\%), outperforming IBL-Tracing (74\%), IBL-Foregone (72\%), and all other tested models. Using IBL-Tracing and IBL-Tracing Calibration, we further test Latent Semantic Analysis (LSA) and embeddings from open-source LLMs: BERT, Llama-8B, Wizard, and Mistral. LLM embeddings reduce alignment compared to LSA, though applying Principal Component Analysis (PCA) yields modest improvements without surpassing LSA. These results suggest the need to refine how cognitive models represent human context and similarity judgments in phishing detection, and they reveal current limitations in open-source LLM embeddings for modeling human decision processes. Moreover, this work represents an early step toward adaptive, cognitively informed personalized training modules that learn with the user rather than merely assessing or correcting them.