Search Results for author:
Found 17 papers, 7 papers with code
WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models
Watermarking generative-AI systems, such as LLMs, has gained considerable interest, driven by their enhanced capabilities across a wide range of tasks.
Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models
Large Language Models (LLMs) have demonstrated impressive zero-shot capabilities and versatility in NLP tasks, however they sometimes fail to maintain crucial invariances for specific tasks.
Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment
Large Language Models (LLMs) are powerful zero-shot assessors and are increasingly used in real-world situations such as for written exams or benchmarking systems.
Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM
In conversational AI research, there's a noticeable trend towards developing models with a larger number of parameters, exemplified by models like ChatGPT.
Investigating the Emergent Audio Classification Ability of ASR Foundation Models
Text and vision foundation models can perform many tasks in a zero-shot setting, a desirable property that enables these systems to be applied in general and low-resource settings.
Assessing Distractors in Multiple-Choice Tests
Specifically, we define quality in terms of the incorrectness, plausibility and diversity of the distractor options.
Zero-shot Audio Topic Reranking using Large Language Models
A key element for this process is highly rapid, flexible, search to support large archives, which in MVSE is facilitated by representing video attributes by embeddings.
Mitigating Word Bias in Zero-shot Prompt-based Classifiers
To address this problem, it is possible to optimise classification thresholds on a labelled data set, however, this mitigates some of the advantages of prompt-based classifiers.
LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models
Current developments in large language models (LLMs) have enabled impressive zero-shot capabilities across various natural language tasks.
Analyzing Multiple-Choice Reading and Listening Comprehension Tests
Multiple-choice reading and listening comprehension tests are an important part of language assessment.
Analysis of the Cambridge Multiple-Choice Questions Reading Dataset with a Focus on Candidate Response Distribution
Multiple choice exams are widely used to assess candidates across a diverse range of domains and tasks.
CUED at ProbSum 2023: Hierarchical Ensemble of Summarization Models
In this paper, we consider the challenge of summarizing patients' medical progress notes in a limited data setting.
Who Needs Decoders? Efficient Estimation of Sequence-level Attributes
In these scenarios, where for example knowing the quality of a system's output to predict poor performance prevails over knowing the output itself, is it possible to bypass the autoregressive decoding?
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
In this work, we propose "SelfCheckGPT", a simple sampling-based approach that can be used to fact-check the responses of black-box models in a zero-resource fashion, i. e. without an external database.
Rewarding Chatbots for Real-World Engagement with Millions of Users
The proposed approach uses automatic pseudo-labels collected from user interactions to train a reward model that can be used to reject low-scoring sample responses generated by the chatbot model at inference time.
MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarization
In this work, we introduce an alternative scheme based on standard information-theoretic measures in which the information present in the source and summary is directly compared.
World Knowledge in Multiple Choice Reading Comprehension
Two metrics are described: the expected number of options, which measures whether a passage-free system can identify the answer a question using world knowledge; and the contextual mutual information, which measures the importance of context for a given question.
