Search Results for author: Jonathan Herzig

Found 30 papers, 17 papers with code

MiMiC: Minimally Modified Counterfactuals in the Representation Space

no code implementations15 Feb 2024 Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, Ponnurangam Kumaraguru

We demonstrate the effectiveness of the proposed approaches in mitigating bias in multiclass classification and in reducing the generation of toxic language, outperforming strong baselines.

A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains

no code implementations1 Feb 2024 Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva

REVEAL includes comprehensive labels for the relevance, attribution to evidence passages, and logical correctness of each reasoning step in a language model's answer, across a variety of datasets and state-of-the-art language models.

Open-Domain Question Answering

Multilingual Instruction Tuning With Just a Pinch of Multilinguality

no code implementations3 Jan 2024 Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, Matan Eyal

As instruction-tuned large language models (LLMs) gain global adoption, their ability to follow instructions in multiple languages becomes increasingly crucial.

Cross-Lingual Transfer Instruction Following

A Comprehensive Evaluation of Tool-Assisted Generation Strategies

no code implementations16 Oct 2023 Alon Jacovi, Avi Caciularu, Jonathan Herzig, Roee Aharoni, Bernd Bohnet, Mor Geva

A growing area of research investigates augmenting language models with tools (e. g., search engines, calculators) to overcome their shortcomings (e. g., missing or incorrect knowledge, incorrect logical inferences).

Retrieval

TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models

1 code implementation18 May 2023 Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, Idan Szpektor

Factual consistency evaluation is often conducted using Natural Language Inference (NLI) models, yet these models exhibit limited success in evaluating summaries.

Natural Language Inference Synthetic Data Generation

What You See is What You Read? Improving Text-Image Alignment Evaluation

1 code implementation NeurIPS 2023 Michal Yarom, Yonatan Bitton, Soravit Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, Idan Szpektor

Automatically determining whether a text and a corresponding image are semantically aligned is a significant challenge for vision-language models, with applications in generative text-to-image and image-to-text tasks.

Question Answering Question Generation +5

mFACE: Multilingual Summarization with Factual Consistency Evaluation

no code implementations20 Dec 2022 Roee Aharoni, Shashi Narayan, Joshua Maynez, Jonathan Herzig, Elizabeth Clark, Mirella Lapata

Abstractive summarization has enjoyed renewed interest in recent years, thanks to pre-trained language models and the availability of large-scale datasets.

Abstractive Text Summarization

Learning To Retrieve Prompts for In-Context Learning

2 code implementations NAACL 2022 Ohad Rubin, Jonathan Herzig, Jonathan Berant

In-context learning is a recent paradigm in natural language understanding, where a large pre-trained language model (LM) observes a test instance and a few training examples as its input, and directly decodes the output without any update to its parameters.

In-Context Learning Language Modelling +1

Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization

1 code implementation EMNLP 2021 Inbar Oren, Jonathan Herzig, Jonathan Berant

We evaluate our approach on a new split of the schema2QA dataset, and show that it leads to dramatic improvements in compositional generalization as well as moderate improvements in the traditional i. i. d setup.

Semantic Parsing

Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations

2 code implementations15 Apr 2021 Jonathan Herzig, Peter Shaw, Ming-Wei Chang, Kelvin Guu, Panupong Pasupat, Yuan Zhang

Sequence-to-sequence (seq2seq) models are prevalent in semantic parsing, but have been found to struggle at out-of-distribution compositional generalization.

Semantic Parsing Text-To-SQL

Span-based Semantic Parsing for Compositional Generalization

1 code implementation ACL 2021 Jonathan Herzig, Jonathan Berant

Despite the success of sequence-to-sequence (seq2seq) models in semantic parsing, recent work has shown that they fail in compositional generalization, i. e., the ability to generalize to new structures built of components observed during training.

Semantic Parsing

Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing

1 code implementation IJCNLP 2019 Jonathan Herzig, Jonathan Berant

Assuming access to unlabeled utterances from the true distribution, we combine crowdsourcing with a paraphrase model to detect correct logical forms for the unlabeled utterances.

Semantic Parsing

Bot2Vec: Learning Representations of Chatbots

no code implementations SEMEVAL 2019 Jonathan Herzig, S, Tommy bank, Michal Shmueli-Scheuer, David Konopnicki

Chatbots (i. e., bots) are becoming widely used in multiple domains, along with supporting bot programming platforms.

CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

3 code implementations NAACL 2019 Alon Talmor, Jonathan Herzig, Nicholas Lourie, Jonathan Berant

To investigate question answering with prior knowledge, we present CommonsenseQA: a challenging new dataset for commonsense question answering.

Ranked #30 on Common Sense Reasoning on CommonsenseQA (using extra training data)

Common Sense Reasoning Multiple-choice +2

Value-based Search in Execution Space for Mapping Instructions to Programs

1 code implementation NAACL 2019 Dor Muhlgay, Jonathan Herzig, Jonathan Berant

Training models to map natural language instructions to programs given target world supervision only requires searching for good programs at training time.

Decoupling Structure and Lexicon for Zero-Shot Semantic Parsing

1 code implementation EMNLP 2018 Jonathan Herzig, Jonathan Berant

Building a semantic parser quickly in a new domain is a fundamental challenge for conversational interfaces, as current semantic parsers require expensive supervision and lack the ability to generalize to new domains.

Semantic Parsing

Detecting Egregious Conversations between Customers and Virtual Agents

no code implementations NAACL 2018 Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki, John Richards, David Piorkowski

In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction.

Neural Semantic Parsing over Multiple Knowledge-bases

1 code implementation ACL 2017 Jonathan Herzig, Jonathan Berant

A fundamental challenge in developing semantic parsers is the paucity of strong supervision in the form of language utterances annotated with logical form.

Semantic Parsing

Cannot find the paper you are looking for? You can Submit a new open access paper.