Search Results for author:
Found 14 papers, 12 papers with code
NoMIRACL: Knowing When You Don't Know for Robust Multilingual Retrieval-Augmented Generation
We measure LLM robustness using two metrics: (i) hallucination rate, measuring model tendency to hallucinate an answer, when the answer is not present in passages in the non-relevant subset, and (ii) error rate, measuring model inaccuracy to recognize relevant passages in the relevant subset.
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution
In this paper, we introduce a new dataset, HAGRID (Human-in-the-loop Attributable Generative Retrieval for Information-seeking Dataset) for building end-to-end generative information-seeking models that are capable of retrieving candidate quotes and generating attributed explanations.
Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard
BEIR is a benchmark dataset for zero-shot evaluation of information retrieval models across 18 different domain/task combinations.
Evaluating Open-Domain Question Answering in the Era of Large Language Models
The recent success of large language models (LLMs) for QA aggravates lexical matching failures since candidate answers become longer, thereby making matching with the gold answers even more challenging.
Evaluating Embedding APIs for Information Retrieval
The ever-increasing size of language models curtails their widespread availability to the community, thereby galvanizing many companies into offering access to large language models through APIs.
Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval
The advent of multilingual language models has generated a resurgence of interest in cross-lingual information retrieval (CLIR), which is the task of searching documents in one language with queries from another.
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages
MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval across 18 different languages, which collectively encompass over three billion native speakers around the world.
Probing the Robustness of Pre-trained Language Models for Entity Matching
The paradigm of fine-tuning Pre-trained Language Models (PLMs) has been successful in Entity Matching (EM).
Ranked #3 on
Entity Resolution
on Amazon-Google
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue
The goal of information-seeking dialogue is to respond to seeker queries with natural language utterances that are grounded on knowledge sources.
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation
From a pre-generated pool of augmented samples, Glitter adaptively selects a subset of worst-case samples with maximal loss, analogous to adversarial DA.
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax
We exploit a semi-supervised approach based on KD to train a model on augmented data.
Evaluating Coherence in Dialogue Systems using Entailment
Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers.
Augmenting Neural Response Generation with Context-Aware Topical Attention
Our model is built upon the basic Seq2Seq model by augmenting it with a hierarchical joint attention mechanism that incorporates topical concepts and previous interactions into the response generation.
A Coherent Unsupervised Model for Toponym Resolution
The evaluation shows that our method outperforms the unsupervised technique as well as Reuters OpenCalais and Google Cloud Natural Language API on all three corpora; also, our method shows a performance close to that of the state-of-the-art supervised method and outperforms it when the test data has 40% or more toponyms that are not seen in the training data.
