Search Results for author: Geoffrey E. Hinton

Found 34 papers, 16 papers with code

Testing GLOM's ability to infer wholes from ambiguous parts

no code implementations29 Nov 2022 Laura Culp, Sara Sabour, Geoffrey E. Hinton

The GLOM architecture proposed by Hinton [2021] is a recurrent neural network for parsing an image into a hierarchy of wholes and parts.

Stacked Capsule Autoencoders

12 code implementations NeurIPS 2019 Adam R. Kosiorek, Sara Sabour, Yee Whye Teh, Geoffrey E. Hinton

In the second stage, SCAE predicts parameters of a few object capsules, which are then used to reconstruct part poses.

Cross-Modal Retrieval Object +1

Learning Sparse Networks Using Targeted Dropout

2 code implementations31 May 2019 Aidan N. Gomez, Ivan Zhang, Siddhartha Rao Kamalakara, Divyam Madaan, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.

Network Pruning Neural Network Compression

Targeted Dropout

1 code implementation NIPS Workshop CDNNRIA 2018 Aidan N. Gomez, Ivan Zhang, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Neural networks are extremely flexible models due to their large number of parameters, which is beneficial for learning, but also highly redundant.

Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures

1 code implementation NeurIPS 2018 Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, Timothy Lillicrap

Here we present results on scaling up biologically motivated models of deep learning on datasets which need deep networks with appropriate architectures to achieve good performance.

Large scale distributed neural network training through online distillation

no code implementations ICLR 2018 Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. Hinton

Two neural networks trained on disjoint subsets of the data can share knowledge by encouraging each model to agree with the predictions the other model would have made.

Language Modelling

Matrix capsules with EM routing

2 code implementations ICLR 2018 Geoffrey E. Hinton, Sara Sabour, Nicholas Frosst

A capsule in one layer votes for the pose matrix of many different capsules in the layer above by multiplying its own pose matrix by trainable viewpoint-invariant transformation matrices that could learn to represent part-whole relationships.

Image Classification

Dynamic Routing Between Capsules

78 code implementations NeurIPS 2017 Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton

We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters.

Image Classification

Who Said What: Modeling Individual Labelers Improves Classification

1 code implementation26 Mar 2017 Melody Y. Guan, Varun Gulshan, Andrew M. Dai, Geoffrey E. Hinton

We also show that our method performs better than competing algorithms by Welinder and Perona (2010), and by Mnih and Hinton (2012).

Classification General Classification

Layer Normalization

28 code implementations21 Jul 2016 Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

One way to reduce the training time is to normalize the activities of the neurons.

Modeling Documents with Deep Boltzmann Machines

no code implementations26 Sep 2013 Nitish Srivastava, Ruslan R. Salakhutdinov, Geoffrey E. Hinton

We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents.

Document Classification General Classification +1

ImageNet Classification with Deep Convolutional Neural Networks

19 code implementations NeurIPS 2012 Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

We trained a large, deep convolutional neural network to classify the 1. 3 million high-resolution images in the LSVRC-2010 ImageNet training set into the 1000 different classes.

General Classification Graph Classification +1

A Better Way to Pretrain Deep Boltzmann Machines

no code implementations NeurIPS 2012 Geoffrey E. Hinton, Ruslan R. Salakhutdinov

We describe how the pre-training algorithm for Deep Boltzmann Machines (DBMs) is related to the pre-training algorithm for Deep Belief Networks and we show that under certain conditions, the pre-training procedure improves the variational lower bound of a two-hidden-layer DBM.

Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine

no code implementations NeurIPS 2010 George Dahl, Marc'Aurelio Ranzato, Abdel-rahman Mohamed, Geoffrey E. Hinton

Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independent TIMIT phone recognition task.

Generating more realistic images using gated MRF's

no code implementations NeurIPS 2010 Marc'Aurelio Ranzato, Volodymyr Mnih, Geoffrey E. Hinton

Probabilistic models of natural images are usually evaluated by measuring performance on rather indirect tasks, such as denoising and inpainting.

Denoising

Gated Softmax Classification

no code implementations NeurIPS 2010 Roland Memisevic, Christopher Zach, Marc Pollefeys, Geoffrey E. Hinton

We describe a log-bilinear" model that computes class probabilities by combining an input vector multiplicatively with a vector of binary latent variables.

Classification General Classification

Learning to combine foveal glimpses with a third-order Boltzmann machine

no code implementations NeurIPS 2010 Hugo Larochelle, Geoffrey E. Hinton

We describe a model based on a Boltzmann machine with third-order connections that can learn how to accumulate information about a shape over several fixations.

General Classification Image Classification

Zero-shot Learning with Semantic Output Codes

no code implementations NeurIPS 2009 Mark Palatucci, Dean Pomerleau, Geoffrey E. Hinton, Tom M. Mitchell

To achieve this, we define the notion of a semantic output code classifier (SOC) which utilizes a knowledge base of semantic properties of $Y$ to extrapolate to novel classes.

Zero-Shot Learning

3D Object Recognition with Deep Belief Nets

no code implementations NeurIPS 2009 Vinod Nair, Geoffrey E. Hinton

Our model achieves 6. 5% error on the test set, which is close to the best published result for NORB (5. 9%) using a convolutional neural net that has built-in knowledge of translation invariance.

3D Object Recognition Object +1

Replicated Softmax: an Undirected Topic Model

no code implementations NeurIPS 2009 Geoffrey E. Hinton, Ruslan R. Salakhutdinov

Each member of the family models the probability distribution of documents of a specific length as a product of topic-specific distributions rather than as a mixture and this gives much better generalization than Latent Dirichlet Allocation for modeling the log probabilities of held-out documents.

The Recurrent Temporal Restricted Boltzmann Machine

no code implementations NeurIPS 2008 Ilya Sutskever, Geoffrey E. Hinton, Graham W. Taylor

The Temporal Restricted Boltzmann Machine (TRBM) is a probabilistic model for sequences that is able to successfully model (i. e., generate nice-looking samples of) several very high dimensional sequences, such as motion capture data and the pixels of low resolution videos of balls bouncing in a box.

Implicit Mixtures of Restricted Boltzmann Machines

no code implementations NeurIPS 2008 Vinod Nair, Geoffrey E. Hinton

We present a mixture model whose components are Restricted Boltzmann Machines (RBMs).

Using matrices to model symbolic relationship

no code implementations NeurIPS 2008 Ilya Sutskever, Geoffrey E. Hinton

We describe a way of learning matrix representations of objects and relationships.

A Scalable Hierarchical Distributed Language Model

no code implementations NeurIPS 2008 Andriy Mnih, Geoffrey E. Hinton

Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models.

Language Modelling

Modeling image patches with a directed hierarchy of Markov random fields

no code implementations NeurIPS 2007 Simon Osindero, Geoffrey E. Hinton

We describe an efficient learning procedure for multilayer generative models that combine the best aspects of Markov random fields and deep, directed belief nets.

Reducing the Dimensionality of Data with Neural Networks

1 code implementation Science 2006 Geoffrey E. Hinton, R. R. Salakhutdinov

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors.

A fast learning algorithm for deep belief nets

1 code implementation Neural Computation 2006 Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh

Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.

Stochastic Neighbor Embedding

1 code implementation NeurIPS 2002 Geoffrey E. Hinton, Sam Roweis

We describe a probabilistic approach to the task of placing objects, de- scribed by high-dimensional vectors or by pairwise dissimilarities, in a low-dimensional space in a way that preserves neighbor identities.

Dimensionality Reduction Object

Learning Internal Representations by Error Propagation

1 code implementation20 Feb 1986 David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams

The rule, called the generalized delta rule, is a simple scheme for implementing a gradient descent method for finding weights that minimize the sum squared error of the system's performance.

Cannot find the paper you are looking for? You can Submit a new open access paper.