S5T1 Table 1: Comparison of the different Duet variants and other state-of-the-art approaches from the public MS MARCO leaderboard. The update Duet model—referred to as Duet v2—benefits significantly from the modifications proposed in this paper. Table S5T1 presents the MRR@10 corresponding to all the Duet variants we evaluated on the dev set. The updated Duet model with all the modifications described in Section S3—referred hereafter as Duet v2—achieves an MRR@10 of 0.243. We include the MRR numbers for other non-Duet based approaches that are available on the public leaderboard in Table S5T1. As of writing this paper, BERT (Devlin et al., 6) based approaches—e.g., (Nogueira and Cho, 5)—are outperforming other approaches by a significant margin. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268, 2016. The ranking model is evaluated on this passage re-ranking task using the mean reciprocal rank (MRR) metric (Craswell, 4). Participants are required to submit the ranked list of passages per query for a development (dev) set and a heldout (eval) set. The MS MARCO task provides a pre-processed training dataset—called “triples.train.full.tsv”—where each training sample consists of a triple ⟨q,p+,p−⟩, where q is a query and p+ and p− are a pair of passages, with p+ being more relevant to q than p−. Similar to the original Duet model, we employ the cross-entropy with softmax loss to learn the parameters of our model M:

An Updated Duet Model for Passage Re-ranking

Reader Guidelines

Editor Guidelines