Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition
Dense retrievers utilize pre-trained backbone language models (e.g., BERT, LLaMA) that are fine-tuned via contrastive learning to perform the task of encoding text into sense representations that can be then compared via a shallow similarity operation, e.g. inner product.
Authors
- Guido Zuccon (ORCID: https://orcid.org/0000-0003-0271-5563)
- Shuai Wang (ORCID: https://orcid.org/0000-0002-0726-5250)
- Zheng Yao
Institutions
- The University of Queensland (AU)
Publication Details
- Published
- 2025-07-13
- DOI
- https://doi.org/10.1145/3726302.3730332
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 0.00