An Optimal Time Variable Learning Framework for Deep Neural Networks
Feature propagation in Deep Neural Networks (DNNs) can be associated to nonlinear discrete dynamical systems. The novelty, in this paper, lies in letting the discretization parameter (time step-size) vary from layer to layer, which needs to be learned, in an optimization framework. The proposed framework can be applied to any of the existing networks such as ResNet, DenseNet or Fractional-DNN. This framework is shown to help overcome the vanishing and exploding gradient issues. Stability of some of the existing continuous DNNs such as Fractional-DNN is also studied. The proposed approach is applied to an ill-posed 3D-Maxwell's equation.
PDF Abstract
Tasks
Datasets
Add Datasets
introduced or used in this paper
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
1x1 Convolution •
Average Pooling •
Batch Normalization •
Bottleneck Residual Block •
Concatenated Skip Connection •
Convolution •
Dense Block •
Dense Connections •
DenseNet •
Dropout •
Global Average Pooling •
Kaiming Initialization •
Max Pooling •
ReLU •
Residual Block •
Residual Connection •
ResNet •
Softmax
