Widening the Gap: Benchmarking Polish LLMs Against Human Baselines in Deception Detection

Detecting deceptive statements is a complex challenge, particularly in non-English contexts where resources are often limited. This study addresses this problem by evaluating the performance of the Polish Large Language Model (LLM), Bielik-11B-v2.3-Instruct. We utilized a dataset of nearly 1500 true and false statements in Polish. In this data, labels reflect agreement with the author’s views. We assessed the model’s performance across diverse prompting variants, adapter fine-tuning, and their combination. The results demonstrate that, while adapter fine-tuning outperforms zero-shot prompting across both data modalities, the combined approach of prompting and fine-tuning is far superior. It achieves an accuracy of 0.82 on typed utterances, exceeding the previously best machine-learning result of 0.69 and human-level accuracy of 0.54. This highlights a growing disparity between humans’ limited detection capabilities and LLMs’ increasing ability to identify deceit.