The impact of AI-generated recommendations on human decision-making in rule-based everyday scenarios: A user study with secondary school students

Abstract Background. AI-generated recommendations are increasingly embedded in the everyday digital environments used by adolescents, including educational platforms. Yet how young users weigh such recommendations against their own judgement in rule-based decisions - and how vulnerable they are when the AI is wrong - remains underexplored. Objective. We investigated whether AI-generated recommendations improve, degrade, or simply shift secondary school students' decisions on simple, rule-based scenarios drawn from a familiar context (school regulations), and whether students follow the AI even when it is demonstrably wrong. Methods. A between-subjects, individually randomised user study was conducted with 264 secondary school students (ages 16–18) at a single school complex (Zespół Szkół im. Narodów Zjednoczonej Europy in Polkowice, Lower Silesia, Poland). Participants completed an online questionnaire comprising 10 short rule-based scenarios with four answer options each. The treatment group (n = 140) received, alongside each scenario, an "AI recommendation" pointing to one option; 7 of the 10 recommendations were correct and 3 were intentionally incorrect. The control group (n = 124) received the same scenarios without any recommendation. Both groups were given a brief generic warning that AI systems can make mistakes and should not replace independent thinking. Differences between groups were analysed using Pearson's chi-square tests with Cramér's V as effect size, both per task and pooled. Results. Overall accuracy did not differ between groups (43.4% vs 42.5%, χ²(1) = 0.17, p = .68). However, the pattern depended strongly on whether the AI was correct. On the seven tasks where the AI was correct, the treatment group answered correctly slightly more often than the control group (44.2% vs 40.1%, χ²(1) = 3.01, p = .083); the effect was not uniform across items, and on Task 1 the control group was in fact more accurate. On the three tasks where the AI was wrong, the treatment group was less accurate (41.4% vs 48.1%, χ²(1) = 3.31, p = .069) and chose the AI-recommended (incorrect) option significantly more often than the control group chose the same option unaided (38.8% vs 27.7%, χ²(1) = 10.45, p = .001). The most pronounced single-task effect occurred on Task 5, where exposure to a misleading AI recommendation nearly doubled the rate at which students selected the wrong, recommended option (47.9% vs 24.2%, χ²(3) = 17.01, p < .001, Cramér's V = 0.254). Conclusions. AI recommendations exerted a measurable, asymmetric influence on adolescent decision-making in this sample: they helped modestly when correct and harmed substantially when wrong, with the harmful effect more reliably detectable than the helpful one. A generic warning that "AI can make mistakes" was insufficient to prevent over-reliance. The findings argue for explicit, scenario-grounded AI literacy in secondary education that goes beyond abstract caveats.