Preregistration: the exchange rate between a log and randomized users (Yahoo! R3 and Open Bandit Dataset)
Preregistration of six hypotheses (H1 to H6) on the exchange rate between a logged-feedback evaluation and randomized users, on two platforms never scored by the project: Yahoo! R3 (real random ratings for 5,400 users) and the Open Bandit Dataset (the same Thompson-sampling log read with and without its logged propensities, against uniform-random traffic). Part of the Validity-Envelope Learning project (RecSys 2027); earlier registrations 10.5281/zenodo.24051368 and 10.5281/zenodo.24103191. Files: the protocol, the log-only predictions written before any gold was read (gold_free.csv, lens_values.csv, models.json, predict_manifest.json), and the four frozen scripts whose SHA-256 are listed in the protocol. Disclosure (Section 2.1): the two gold stages ran on 2026-11-18 at 10:04 to 10:05 UTC, before publication; they only wrote the gold arrays, which were hashed and made read-only without analysis. The confirmatory analysis (confirm_xr.py) runs once, after publication.
Authors
- Polignano Marco
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-18
- DOI
- https://doi.org/10.5281/zenodo.24338295
- Primary Topic
- Recommender Systems and Techniques
- Type
- preprint