2 code implementations • 13 Jul 2023 • Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham
FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements.
1 code implementation • 21 Dec 2022 • Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training.
1 code implementation • NAACL (GeBNLP) 2022 • Yarden Tal, Inbal Magar, Roy Schwartz
We find that while larger models outperform smaller ones, the probability that their mistakes are caused by gender bias is higher.
1 code implementation • ACL 2022 • Inbal Magar, Roy Schwartz
Experiments with two models and three downstream tasks show that exploitation exists in some cases, but in others the models memorize the contaminated data, but do not exploit it.