Search Results for author: Shauli Ravfogel

Found 31 papers, 18 papers with code

It’s not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT

1 code implementation EMNLP (BlackboxNLP) 2020 Hila Gonen, Shauli Ravfogel, Yanai Elazar, Yoav Goldberg

Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages.

Translation

What Changed? Converting Representational Interventions to Natural Language

no code implementations17 Feb 2024 Matan Avitan, Ryan Cotterell, Yoav Goldberg, Shauli Ravfogel

Interventions targeting the representation space of language models (LMs) have emerged as effective means to influence model behavior.

counterfactual

MiMiC: Minimally Modified Counterfactuals in the Representation Space

no code implementations15 Feb 2024 Shashwat Singh, Shauli Ravfogel, Jonathan Herzig, Roee Aharoni, Ryan Cotterell, Ponnurangam Kumaraguru

We demonstrate the effectiveness of the proposed approaches in mitigating bias in multiclass classification and in reducing the generation of toxic language, outperforming strong baselines.

Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment

1 code implementation NeurIPS 2023 Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, Gal Chechik

This reflects an impaired mapping between linguistic binding of entities and modifiers in the prompt and visual binding of the corresponding elements in the generated image.

Attribute Sentence +1

Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation

1 code implementation26 May 2023 Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, Yanai Elazar

In this paper, we compare the generalization of few-shot fine-tuning and in-context learning to challenge datasets, while controlling for the models used, the number of examples, and the number of parameters, ranging from 125M to 30B.

Domain Generalization In-Context Learning

All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations

1 code implementation23 May 2023 Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf, Ryan Cotterell

Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.

Retrieving Texts based on Abstract Descriptions

no code implementations21 May 2023 Shauli Ravfogel, Valentina Pyatkin, Amir DN Cohen, Avshalom Manevich, Yoav Goldberg

While instruction-tuned Large Language Models (LLMs) excel at extracting information from text, they are not suitable for locating texts conforming to a given description in a large document collection (semantic retrieval).

Language Modelling Large Language Model +2

Conformal Nucleus Sampling

no code implementations4 May 2023 Shauli Ravfogel, Yoav Goldberg, Jacob Goldberger

Language models generate text based on successively sampling the next word.

Conformal Prediction

DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models

no code implementations19 Oct 2022 Royi Rassin, Shauli Ravfogel, Yoav Goldberg

We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image).

Log-linear Guardedness and its Implications

no code implementations18 Oct 2022 Shauli Ravfogel, Yoav Goldberg, Ryan Cotterell

Methods for erasing human-interpretable concepts from neural representations that assume linearity have been found to be tractable and useful.

Visual Comparison of Language Model Adaptation

no code implementations17 Aug 2022 Rita Sevastjanova, Eren Cakmak, Shauli Ravfogel, Ryan Cotterell, Mennatallah El-Assady

The simplicity of adapter training and composition comes along with new challenges, such as maintaining an overview of adapter properties and effectively comparing their produced embedding spaces.

Language Modelling

Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions

no code implementations28 Jul 2022 Yanai Elazar, Nora Kassner, Shauli Ravfogel, Amir Feder, Abhilasha Ravichander, Marius Mosbach, Yonatan Belinkov, Hinrich Schütze, Yoav Goldberg

Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.

Analyzing Gender Representation in Multilingual Models

1 code implementation RepL4NLP (ACL) 2022 Hila Gonen, Shauli Ravfogel, Yoav Goldberg

Multilingual language models were shown to allow for nontrivial transfer across scripts and languages.

Gender Classification

Kernelized Concept Erasure

1 code implementation28 Jan 2022 Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, Ryan Cotterell

One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure prevents the prediction of the concept from the representations.

Linear Adversarial Concept Erasure

2 code implementations28 Jan 2022 Shauli Ravfogel, Michael Twiton, Yoav Goldberg, Ryan Cotterell

Modern neural models trained on textual data rely on pre-trained representations that emerge without direct supervision.

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

5 code implementations ACL 2022 Elad Ben Zaken, Shauli Ravfogel, Yoav Goldberg

We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified.

Language Modelling

Neural Extractive Search

no code implementations ACL 2021 Shauli Ravfogel, Hillel Taub-Tabib, Yoav Goldberg

We advocate for a search paradigm called ``extractive search'', in which a search query is enriched with capture-slots, to allow for such rapid extraction.

Retrieval

Contrastive Explanations for Model Interpretability

1 code implementation EMNLP 2021 Alon Jacovi, Swabha Swayamdipta, Shauli Ravfogel, Yanai Elazar, Yejin Choi, Yoav Goldberg

Our method is based on projecting model representation to a latent space that captures only the features that are useful (to the model) to differentiate two potential decisions.

text-classification Text Classification

Measuring and Improving Consistency in Pretrained Language Models

1 code implementation1 Feb 2021 Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg

In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge?

It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT

1 code implementation16 Oct 2020 Hila Gonen, Shauli Ravfogel, Yanai Elazar, Yoav Goldberg

Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages.

Translation

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

no code implementations1 Jun 2020 Yanai Elazar, Shauli Ravfogel, Alon Jacovi, Yoav Goldberg

In this work, we point out the inability to infer behavioral conclusions from probing results and offer an alternative method that focuses on how the information is being used, rather than on what information is encoded.

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection

2 code implementations ACL 2020 Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, Yoav Goldberg

The ability to control for the kinds of information encoded in neural representation has a variety of use cases, especially in light of the challenge of interpreting these models.

Fairness Multi-class Classification +1

Ab Antiquo: Neural Proto-language Reconstruction

2 code implementations NAACL 2021 Carlo Meloni, Shauli Ravfogel, Yoav Goldberg

Historical linguists have identified regularities in the process of historic sound change.

Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages

2 code implementations NAACL 2019 Shauli Ravfogel, Yoav Goldberg, Tal Linzen

How do typological properties such as word order and morphological case marking affect the ability of neural sequence models to acquire the syntax of a language?

Object

Can LSTM Learn to Capture Agreement? The Case of Basque

no code implementations WS 2018 Shauli Ravfogel, Francis M. Tyers, Yoav Goldberg

We propose the Basque agreement prediction task as challenging benchmark for models that attempt to learn regularities in human language.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.