Retriever-Augmented Generation, or RAG, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. For query $x$, Maximum Inner Product Search (MIPS) is used to find the top-K documents $z_{i}$. For final prediction $y$, we treat $z$ as a latent variable and marginalize over seq2seq predictions given different documents.
Source: Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Retrieval | 122 | 33.15% |
Question Answering | 46 | 12.50% |
Language Modelling | 26 | 7.07% |
Information Retrieval | 19 | 5.16% |
Large Language Model | 18 | 4.89% |
Text Generation | 10 | 2.72% |
Open-Domain Question Answering | 9 | 2.45% |
Benchmarking | 8 | 2.17% |
Sentence | 5 | 1.36% |