LLM-Based Compliance Analysis of Declared Policies and Actual Cookie Usage on Websites

This paper proposes a solution to analyze the compliance between the declared website policies and their observed cookie data and behaviors. The process focuses on five key cookie attributes: name, purpose, retention period, third-party involvement, and permitted behaviors. First, samples are constructed from 12,167 cookie policy URLs via extraction of text and tables, translation to English, and attribute annotation. Second, a LLaMA-3.1-8B-Instruct model is fine-tuned with LoRA and guided by a prompt to extract cookie attributes in a structured format. In parallel, the model classifies each attribute as specific, general, or undefined based on the level of specificity in cookie declarations. Third, three matching strategies, each dedicated to one level of specificity, are defined to compare the extracted information against the observed cookie data and behaviors. The model achieves 87.34% specificity classification accuracy and 91.46% average F1-score across extracted attributes on a test set of 3,042 samples. The solution is deployed as a browser extension to enable automated cookie auditing. Evaluation on 200 popular e-commerce websites shows that 47% use undeclared cookies, 80% deploy cookies before obtaining user consent, and 91% include third-party cookies not mentioned in their policies.