Simulated Users Can Reproduce an Algorithm Audit Without Sharing the Mechanism That Produced the Finding

Abstract Simulated users built from large language models are increasingly used to evaluate recommender systems, yet their validity is argued from agreement with a target distribution, which does not establish that the simulation behaves as the target does for the same reason. This study specifies a validation procedure that needs no data on real users. A published algorithm audit supplies the criterion: its reported direction is fixed in advance, its own meaningless-preference control is transplanted so that the test cannot pass indiscriminately, and baselines holding only the agents’ observed behaviour attribute whatever is reproduced. The procedure is instantiated on an audit of affective recommendation and applied to four generative models across 1,340 simulation runs under a fixed content pool, recommender, pass criteria and seed sequence. With one hundred seeds, two models reproduce the audited asymmetry, by +0.025 at p = 0.0007 and by +0.017 at p = 0.0130, yet a baseline given only the observed adherence and choice distribution recreates one third of the first and two thirds of the second. The verdict also depended on the number of runs, since one model passed at ten seeds, none at thirty and two at one hundred. Conveying the preference through interaction history instead of language reverses the difference to −0.029 at an adherence of 0.419 against a chance level of 0.429. Three pre-expenditure tests, a stability check and a four-valued verdict are proposed for evaluations of this kind.