In Situ Answer Sentence Selection at Web-scale

Current answer sentence selection (AS2) applied in open-domain question answering (ODQA) selects answers by ranking a large set of candidates, i.e., sentences, extracted from the retrieved text. In this paper, we present Passage-based Extracting Answer Sentence In-place (PEASI), a novel answer selection model optimized for Web-scale setting. This is a Transformer-based network that can jointly (i) rerank passages retrieved for a question and (ii) identify a probable answer from the top passages. We train PEASI with multi-task learning for sharing representations between the passage reranker and answer sentence extractor. We construct a new large-scale QA dataset (WQA) consisting of 800,000+ labeled passages/sentences for 60,000+ questions. The experiment results show that PEASI outperforms AS2 state of the art by 6.51% in accuracy on WQA, from 48.86% to 55.37%.