no code implementations • 29 Jan 2024 • Yotam Wolf, Noam Wies, Dorin Shteyman, Binyamin Rothberg, Yoav Levine, Amnon Shashua
Representation engineering yields gains in alignment oriented tasks such as resistance to adversarial attacks and reduction of social biases, but was also shown to cause a decrease in the ability of the model to perform basic tasks.
2 code implementations • 13 Jul 2023 • Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, Yoav Shoham
FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements.
no code implementations • 4 Jul 2023 • Eliya Segev, Maya Alroy, Ronen Katsir, Noam Wies, Ayana Shenhav, Yael Ben-Oren, David Zar, Oren Tadmor, Jacob Bitterman, Amnon Shashua, Tal Rosenwein
Here we propose $\textit{Align With Purpose}$, a $\textbf{general Plug-and-Play framework}$ for enhancing a desired property in models trained with the CTC criterion.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +1
no code implementations • 19 Apr 2023 • Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, Amnon Shashua
An important aspect in developing language models that interact with humans is aligning their behavior to be useful and unharmful for their human users.
1 code implementation • 31 Jan 2023 • Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
Retrieval-Augmented Language Modeling (RALM) methods, which condition a language model (LM) on relevant documents from a grounding corpus during generation, were shown to significantly improve language modeling performance.
1 code implementation • 21 Dec 2022 • Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
We present Parallel Context Windows (PCW), a method that alleviates the context window restriction for any off-the-shelf LLM without further training.
no code implementations • 1 May 2022 • Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, Dor Muhlgay, Noam Rozen, Erez Schwartz, Gal Shachaf, Shai Shalev-Shwartz, Amnon Shashua, Moshe Tenenholtz
Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks.
no code implementations • 21 Apr 2022 • Yoav Levine, Itay Dalmedigos, Ori Ram, Yoel Zeldes, Daniel Jannai, Dor Muhlgay, Yoni Osin, Opher Lieber, Barak Lenz, Shai Shalev-Shwartz, Amnon Shashua, Kevin Leyton-Brown, Yoav Shoham
To demonstrate this, we introduce three novel methods for leveraging frozen models: input-dependent prompt tuning, frozen readers, and recursive LMs, each of which vastly improves on current frozen-model approaches.
1 code implementation • 6 Apr 2022 • Noam Wies, Yoav Levine, Amnon Shashua
Recently, several works have demonstrated high gains by taking a straightforward approach for incorporating intermediate supervision in compounded natural language problems: the sequence-to-sequence LM is fed with an augmented input, in which the decomposed tasks' labels are simply concatenated to the original input.
no code implementations • ICLR 2022 • Yoav Levine, Noam Wies, Daniel Jannai, Dan Navon, Yedid Hoshen, Amnon Shashua
We highlight a bias introduced by this common practice: we prove that the pretrained NLM can model much stronger dependencies between text segments that appeared in the same training example, than it can between text segments that appeared in different training examples.
no code implementations • 9 May 2021 • Noam Wies, Yoav Levine, Daniel Jannai, Amnon Shashua
After their successful debut in natural language processing, Transformer architectures are now becoming the de-facto standard in many domains.
no code implementations • 18 Mar 2021 • Or Sharir, Amnon Shashua, Giuseppe Carleo
We establish a direct connection between general tensor networks and deep feed-forward artificial neural networks.
1 code implementation • NeurIPS 2020 • Yoav Levine, Noam Wies, Or Sharir, Hofit Bata, Amnon Shashua
Our guidelines elucidate the depth-to-width trade-off in self-attention networks of sizes up to the scale of GPT3 (which we project to be too deep for its size), and beyond, marking an unprecedented width of 30K as optimal for a 1-Trillion parameter network.
no code implementations • 30 Mar 2020 • Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua
The AI-alignment problem arises when there is a discrepancy between the goals that a human designer specifies to an AI learner and a potential catastrophic outcome that does not reflect what the human designer really wants.
no code implementations • ACL 2020 • Yoav Levine, Barak Lenz, Or Dagan, Ori Ram, Dan Padnos, Or Sharir, Shai Shalev-Shwartz, Amnon Shashua, Yoav Shoham
The ability to learn from large unlabeled corpora has allowed neural language models to advance the frontier in natural language understanding.
Ranked #11 on Word Sense Disambiguation on Words in Context
2 code implementations • 11 Feb 2019 • Or Sharir, Yoav Levine, Noam Wies, Giuseppe Carleo, Amnon Shashua
Artificial Neural Networks were recently shown to be an efficient representation of highly-entangled many-body quantum states.
no code implementations • 26 Mar 2018 • Yoav Levine, Or Sharir, Nadav Cohen, Amnon Shashua
Modern deep learning has enabled unprecedented achievements in various domains.
no code implementations • ICLR 2018 • Yoav Levine, Or Sharir, Amnon Shashua
We prove that deep recurrent networks support Start-End separation ranks which are exponentially higher than those supported by their shallow counterparts.
1 code implementation • 25 Oct 2017 • Yoav Levine, Or Sharir, Alon Ziv, Amnon Shashua
A key attribute that drives the unprecedented success of modern Recurrent Neural Networks (RNNs) on learning tasks which involve sequential data, is their ability to model intricate long-term temporal dependencies.
no code implementations • 12 Oct 2017 • Or Sharir, Amnon Shashua
We present a novel tractable generative model that extends Sum-Product Networks (SPNs) and significantly boosts their power.
3 code implementations • 21 Aug 2017 • Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua
In the second part we describe a design of a system that adheres to our safety assurance requirements and is scalable to millions of cars.
no code implementations • 5 May 2017 • Nadav Cohen, Or Sharir, Yoav Levine, Ronen Tamari, David Yakira, Amnon Shashua
Expressive efficiency refers to the ability of a network architecture to realize functions that require an alternative architecture to be much larger.
no code implementations • ICLR 2018 • Yoav Levine, David Yakira, Nadav Cohen, Amnon Shashua
This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph.
no code implementations • ICLR 2018 • Nadav Cohen, Ronen Tamari, Amnon Shashua
By introducing and analyzing the concept of mixed tensor decompositions, we prove that interconnecting dilated convolutional networks can lead to expressive efficiency.
1 code implementation • ICLR 2018 • Or Sharir, Amnon Shashua
Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger.
2 code implementations • 13 Oct 2016 • Or Sharir, Ronen Tamari, Nadav Cohen, Amnon Shashua
Other methods, based on arithmetic circuits and sum-product networks, do allow tractable marginalization, but their performance is challenged by the need to learn the structure of a circuit.
no code implementations • 11 Oct 2016 • Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua
Second, the Markov Decision Process model often used in robotics is problematic in our case because of unpredictable behavior of other agents in this multi-agent scenario.
no code implementations • NeurIPS 2016 • Oren Tadmor, Yonatan Wexler, Tal Rosenwein, Shai Shalev-Shwartz, Amnon Shashua
This work is motivated by the engineering task of achieving a near state-of-the-art face recognition on a minimal computing budget running on an embedded system.
1 code implementation • 22 May 2016 • Nadav Cohen, Amnon Shashua
In addition to analyzing deep networks, we show that shallow ones support only linear separation ranks, and by this gain insight into the benefit of functions brought forth by depth - they are able to efficiently model strong correlation under favored partitions of the input.
no code implementations • 23 Apr 2016 • Shai Shalev-Shwartz, Amnon Shashua
We compare the end-to-end training approach to a modular approach in which a system is decomposed into semantically meaningful components.
no code implementations • 1 Mar 2016 • Nadav Cohen, Amnon Shashua
Second, and more importantly, we show that depth efficiency is weaker with convolutional rectifier networks than it is with convolutional arithmetic circuits.
no code implementations • 4 Feb 2016 • Shai Shalev-Shwartz, Nir Ben-Zrihem, Aviad Cohen, Amnon Shashua
We argue that dual versions of the MDP framework (that depend on the value function and the $Q$ function) are problematic for autonomous driving applications due to the non Markovian of the natural state space representation, and due to the continuous state and action spaces.
no code implementations • 16 Sep 2015 • Nadav Cohen, Or Sharir, Amnon Shashua
In this work we derive a deep network architecture based on arithmetic circuits that inherently employs locality, sharing and pooling.
no code implementations • CVPR 2016 • Nadav Cohen, Or Sharir, Amnon Shashua
We present a deep layered architecture that generalizes convolutional neural networks (ConvNets).
1 code implementation • 3 Oct 2014 • Nadav Cohen, Amnon Shashua
We present a deep layered architecture that generalizes classical convolutional neural networks (ConvNets).
no code implementations • NeurIPS 2011 • Shai Shalev-Shwartz, Yonatan Wexler, Amnon Shashua
We consider the problem of learning a multiclass predictor that uses only few features, and in particular, the number of used features should increase sub-linearly with the number of possible classes.
1 code implementation • 23 Apr 2009 • Amnon Shashua
Introduction to Machine learning covering Statistical Inference (Bayes, EM, ML/MaxEnt duality), algebraic and spectral methods (PCA, LDA, CCA, Clustering), and PAC learning (the Formal model, VC dimension, Double Sampling theorem).