Search Results for author: Qianli Liao

Found 20 papers, 4 papers with code

Explicit regularization and implicit bias in deep network classifiers trained with the square loss

no code implementations31 Dec 2020 Tomaso Poggio, Qianli Liao

Deep ReLU networks trained with the square loss have been observed to perform well in classification tasks.

Hierarchically Compositional Tasks and Deep Convolutional Networks

no code implementations24 Jun 2020 Arturo Deza, Qianli Liao, Andrzej Banburski, Tomaso Poggio

For object recognition we find, as expected, that scrambling does not affect the performance of shallow or deep fully connected networks contrary to the out-performance of convolutional networks.

Object Recognition

Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization

no code implementations25 Aug 2019 Tomaso Poggio, Andrzej Banburski, Qianli Liao

In approximation theory both shallow and deep networks have been shown to approximate any continuous functions on a bounded domain at the expense of an exponential number of parameters (exponential in the dimensionality of the function).

Theory III: Dynamics and Generalization in Deep Networks

no code implementations12 Mar 2019 Andrzej Banburski, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Fernanda De La Torre, Jack Hidary, Tomaso Poggio

In particular, gradient descent induces a dynamics of the normalized weights which converge for $t \to \infty$ to an equilibrium which corresponds to a minimum norm (or maximum margin) solution.

Biologically-plausible learning algorithms can scale to large datasets

2 code implementations ICLR 2019 Will Xiao, Honglin Chen, Qianli Liao, Tomaso Poggio

These results complement the study by Bartunov et al. (2018), and establish a new benchmark for future biologically plausible learning algorithms on more difficult datasets and more complex architectures.

Biologically-plausible Training

A Surprising Linear Relationship Predicts Test Performance in Deep Networks

3 code implementations25 Jul 2018 Qianli Liao, Brando Miranda, Andrzej Banburski, Jack Hidary, Tomaso Poggio

Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors?

General Classification Generalization Bounds

Theory IIIb: Generalization in Deep Networks

no code implementations29 Jun 2018 Tomaso Poggio, Qianli Liao, Brando Miranda, Andrzej Banburski, Xavier Boix, Jack Hidary

Here we prove a similar result for nonlinear multilayer DNNs near zero minima of the empirical loss.

Binary Classification

Theory of Deep Learning IIb: Optimization Properties of SGD

no code implementations7 Jan 2018 Chiyuan Zhang, Qianli Liao, Alexander Rakhlin, Brando Miranda, Noah Golowich, Tomaso Poggio

In Theory IIb we characterize with a mix of theory and experiments the optimization of deep convolutional networks by Stochastic Gradient Descent.

Theory of Deep Learning III: explaining the non-overfitting puzzle

no code implementations30 Dec 2017 Tomaso Poggio, Kenji Kawaguchi, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Xavier Boix, Jack Hidary, Hrushikesh Mhaskar

In this note, we show that the dynamics associated to gradient descent minimization of nonlinear networks is topologically equivalent, near the asymptotically stable minima of the empirical error, to linear gradient system in a quadratic potential with a degenerate (for square loss) or almost degenerate (for logistic or crossentropy loss) Hessian.

General Classification

Theory II: Landscape of the Empirical Risk in Deep Learning

no code implementations28 Mar 2017 Qianli Liao, Tomaso Poggio

Previous theoretical work on deep learning and neural network optimization tend to focus on avoiding saddle points and local minima.

Compression of Deep Neural Networks for Image Instance Retrieval

no code implementations18 Jan 2017 Vijay Chandrasekhar, Jie Lin, Qianli Liao, Olivier Morère, Antoine Veillard, Ling-Yu Duan, Tomaso Poggio

One major drawback of CNN-based {\it global descriptors} is that uncompressed deep neural network models require hundreds of megabytes of storage making them inconvenient to deploy in mobile applications or in custom hardware.

Image Instance Retrieval Model Compression +2

Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review

no code implementations2 Nov 2016 Tomaso Poggio, Hrushikesh Mhaskar, Lorenzo Rosasco, Brando Miranda, Qianli Liao

The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning.

Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning

no code implementations19 Oct 2016 Qianli Liao, Kenji Kawaguchi, Tomaso Poggio

We systematically explored a spectrum of normalization algorithms related to Batch Normalization (BN) and propose a generalized formulation that simultaneously solves two major limitations of BN: (1) online learning and (2) recurrent learning.

View-tolerant face recognition and Hebbian learning imply mirror-symmetric neural tuning to head orientation

no code implementations5 Jun 2016 Joel Z. Leibo, Qianli Liao, Winrich Freiwald, Fabio Anselmi, Tomaso Poggio

The primate brain contains a hierarchy of visual areas, dubbed the ventral stream, which rapidly computes object representations that are both specific for object identity and relatively robust against identity-preserving transformations like depth-rotations.

Face Recognition Object +1

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex

1 code implementation13 Apr 2016 Qianli Liao, Tomaso Poggio

We discuss relations between Residual Networks (ResNet), Recurrent Neural Networks (RNNs) and the primate visual cortex.

Learning Functions: When Is Deep Better Than Shallow

no code implementations3 Mar 2016 Hrushikesh Mhaskar, Qianli Liao, Tomaso Poggio

While the universal approximation property holds both for hierarchical and shallow networks, we prove that deep (hierarchical) networks can approximate the class of compositional functions with the same accuracy as shallow networks but with exponentially lower number of training parameters as well as VC-dimension.

How Important is Weight Symmetry in Backpropagation?

2 code implementations17 Oct 2015 Qianli Liao, Joel Z. Leibo, Tomaso Poggio

Gradient backpropagation (BP) requires symmetric feedforward and feedback connections -- the same weights must be used for forward and backward passes.

 Ranked #1 on Handwritten Digit Recognition on MNIST (PERCENTAGE ERROR metric)

Handwritten Digit Recognition Image Classification

Unsupervised learning of clutter-resistant visual representations from natural videos

no code implementations12 Sep 2014 Qianli Liao, Joel Z. Leibo, Tomaso Poggio

Populations of neurons in inferotemporal cortex (IT) maintain an explicit code for object identity that also tolerates transformations of object appearance e. g., position, scale, viewing angle [1, 2, 3].

Face Recognition

Learning invariant representations and applications to face verification

no code implementations NeurIPS 2013 Qianli Liao, Joel Z. Leibo, Tomaso Poggio

Next, we apply the model to non-affine transformations: as expected, it performs well on face verification tasks requiring invariance to the relatively smooth transformations of 3D rotation-in-depth and changes in illumination direction.

Face Verification Object Recognition

Can a biologically-plausible hierarchy effectively replace face detection, alignment, and recognition pipelines?

no code implementations16 Nov 2013 Qianli Liao, Joel Z. Leibo, Youssef Mroueh, Tomaso Poggio

The standard approach to unconstrained face recognition in natural photographs is via a detection, alignment, recognition pipeline.

Face Detection Face Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.