no code implementations • 29 Dec 2022 • Stas Tiomkin, Ilya Nemenman, Daniel Polani, Naftali Tishby
Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation.
no code implementations • 8 Jun 2021 • Hagai Rappeport, Irit Levin Reisman, Naftali Tishby, Nathalie Q. Balaban
Estimating the largest Lyapunov exponent from observations of a process is especially challenging in systems affected by dynamical noise, which is the case for many models of real-world processes, in particular models of biological systems.
1 code implementation • 8 Jun 2020 • Zoe Piran, Ravid Shwartz-Ziv, Naftali Tishby
The Information Bottleneck (IB) framework is a general characterization of optimal representations obtained using a principled approach for balancing accuracy and complexity.
no code implementations • SCiL 2020 • Noga Zaslavsky, Terry Regier, Naftali Tishby, Charles Kemp
Recently, this idea has been cast in terms of a general information-theoretic principle of efficiency, the Information Bottleneck (IB) principle, and it has been shown that this principle accounts for the emergence and evolution of named color categories across languages, including soft structure and patterns of inconsistent naming.
no code implementations • ICLR 2019 • Ravid Shwartz-Ziv, Amichai Painsky, Naftali Tishby
Specifically, we show that the training of the network is characterized by a rapid increase in the mutual information (MI) between the layers and the target label, followed by a longer decrease in the MI between the layers and the input variable.
1 code implementation • 25 Mar 2019 • Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, Lenka Zdeborová
Machine learning encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent years.
Computational Physics Cosmology and Nongalactic Astrophysics Disordered Systems and Neural Networks High Energy Physics - Theory Quantum Physics
no code implementations • 31 Oct 2018 • Amichai Painsky, Meir Feder, Naftali Tishby
In this work we introduce an information-theoretic compressed representation framework for the non-linear CCA problem (CRCCA), which extends the classical ACE approach.
no code implementations • 9 Aug 2018 • Noga Zaslavsky, Charles Kemp, Terry Regier, Naftali Tishby
This work thus identifies a computational principle that characterizes human semantic systems, and that could usefully inform semantic representations in machines.
no code implementations • 16 May 2018 • Noga Zaslavsky, Charles Kemp, Naftali Tishby, Terry Regier
We show that greater communicative precision for warm than for cool colors, and greater communicative need, may both be explained by perceptual structure.
no code implementations • 10 Dec 2017 • Michal Moshkovitz, Naftali Tishby
Designing bounded-memory algorithms is becoming increasingly important nowadays.
no code implementations • 7 Nov 2017 • Amichai Painsky, Naftali Tishby
In this work we introduce a Gaussian lower bound to the IB curve; we find an embedding of the data which maximizes its "Gaussian part", on which we apply the GIB.
13 code implementations • 2 Mar 2017 • Ravid Shwartz-Ziv, Naftali Tishby
Previous work proposed to analyze DNNs in the \textit{Information Plane}; i. e., the plane of the Mutual Information values that each layer preserves on the input and output variables.
no code implementations • 2 Mar 2017 • Michal Moshkovitz, Naftali Tishby
We suggest analyzing neural networks through the prism of space constraints.
no code implementations • 18 Sep 2016 • Roy Fox, Michal Moshkovitz, Naftali Tishby
It is well known that options can make planning more efficient, among their many benefits.
no code implementations • 18 Apr 2016 • Pedro A. Ortega, Naftali Tishby
There is a consensus that human and non-human subjects experience temporal distortions in many stages of their perceptual and decision-making systems.
no code implementations • 29 Dec 2015 • Roy Fox, Naftali Tishby
One attempt to deal with this is to focus on reactive policies, that only base their actions on the most recent observation.
3 code implementations • 28 Dec 2015 • Roy Fox, Ari Pakman, Naftali Tishby
We propose G-learning, a new off-policy learning algorithm that regularizes the value estimates by penalizing deterministic policies in the beginning of the learning process.
no code implementations • 21 Dec 2015 • Pedro A. Ortega, Daniel A. Braun, Justin Dyer, Kee-Eung Kim, Naftali Tishby
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics.
1 code implementation • 9 Mar 2015 • Naftali Tishby, Noga Zaslavsky
Deep Neural Networks (DNNs) are analyzed via the theoretical framework of the information bottleneck (IB) principle.
no code implementations • 5 Apr 2012 • Sivan Sabato, Nathan Srebro, Naftali Tishby
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the margin-adapted dimension of the data distribution.
no code implementations • NeurIPS 2010 • Sivan Sabato, Nathan Srebro, Naftali Tishby
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the gamma-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the gamma-adapted-dimension of the source distribution.
no code implementations • NeurIPS 2008 • Ohad Shamir, Naftali Tishby
In this paper, we provide a set of general sufficient conditions, which ensure the reliability of clustering stability estimators in the large sample regime.
no code implementations • 28 Dec 2007 • William Bialek, Rob R. de Ruyter van Steveninck, Naftali Tishby
Does the brain construct an efficient representation of the sensory world?
3 code implementations • 24 Apr 2000 • Naftali Tishby, Fernando C. Pereira, William Bialek
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$.