Search Results for author:
Found 75 papers, 17 papers with code
Accelerating the diffusion-based ensemble sampling by non-reversible dynamics
Compared with the naive parallel-chain SGLD that updates multiple particles independently, ensemble methods update particles with their interactions.
End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training
End-to-end (E2E) training, optimizing the entire model through error backpropagation, fundamentally supports the advancements of deep learning.
Understanding Parameter Saliency via Extreme Value Theory
Furthermore, we show that the existing parameter saliency method exhibits a bias against the depth of layers in deep neural networks.
Initialization Bias of Fourier Neural Operator: Revisiting the Edge of Chaos
Building upon this observation, we also propose an edge of chaos initialization scheme for FNO to mitigate the negative initialization bias leading to training instability.
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?
Existing analyses of the expressive capacity of Transformer models have required excessively deep layers for data memorization, leading to a discrepancy with the Transformers actually used in practice.
Exploring Weight Balancing on Long-Tailed Recognition Problem
Recognition problems in long-tailed data, in which the sample size per class is heavily skewed, have gained importance because the distribution of the sample size per class in a dataset is generally exponential unless the sample size is intentionally adjusted.
Excess risk analysis for epistemic uncertainty with application to variational inference
Bayesian deep learning plays an important role especially for its ability evaluating epistemic uncertainty (EU).
Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective
The lottery ticket hypothesis (LTH) has attracted attention because it can explain why over-parameterized models often show high generalization ability.
Goldilocks-curriculum Domain Randomization and Fractal Perlin Noise with Application to Sim2Real Pneumonia Lesion Detection
An obstacle in developing a CAD system for a disease is that the number of medical images is typically too small to improve the performance of the machine learning model.
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey
Representation learning enables us to automatically extract generic feature representations from a dataset to solve another machine learning task.
Neural Lagrangian Schrödinger Bridge: Diffusion Modeling for Population Dynamics
Population dynamics is the study of temporal and spatial variation in the size of populations of organisms and is a major part of population ecology.
A Closer Look at Prototype Classifier for Few-shot Image Classification
The prototypical network is a prototype classifier based on meta-learning and is widely used for few-shot learning because it classifies unseen examples by constructing class-specific prototypes without adjusting hyper-parameters during meta-testing.
Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum
Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and flat minima selection.
Disentanglement Analysis with Partial Information Decomposition
We propose a framework to analyze how multivariate representations disentangle ground-truth generative factors.
Loss function based second-order Jensen inequality and its application to particle variational inference
First, we provide a new second-order Jensen inequality, which has the repulsion term based on the loss function.
Toward Neural-Network-Guided Program Synthesis and Verification
Another application is to a new program development framework called oracle-based programming, which is a neural-network-guided variation of Solar-Lezama's program synthesis by sketching.
Abelian Neural Networks
We study the problem of modeling a binary operation that satisfies some algebraic requirements.
Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning
Instance discriminative self-supervised representation learning has been attracted attention thanks to its unsupervised nature and informative feature representation for downstream tasks.
Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification
SSC can be solved by a standard (multi-class) classification method, and we use the SSC solution to obtain the final binary classifier through a certain linear-fractional transformation.
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective
Weight decay is a simple yet powerful regularization technique that has been very widely used in training of deep neural networks (DNNs).
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
Thus it motivates us to design a similar mechanism named {\it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks.
Stable Weight Decay Regularization
\citet{loshchilov2018decoupled} demonstrated that $L_{2}$ regularization is not identical to weight decay for adaptive gradient methods, such as Adaptive Momentum Estimation (Adam), and proposed Adam with Decoupled Weight Decay (AdamW).
Active Classification with Uncertainty Comparison Queries
We then propose an efficient adaptive labeling algorithm using the proposed oracle and the positivity comparison oracle.
Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain
We propose an evaluation framework for class probability estimates (CPEs) in the presence of label uncertainty, which is commonly observed as diagnosis disagreement between experts in the medical domain.
Adai: Separating the Effects of Adaptive Learning Rate and Momentum Inertia
Specifically, we disentangle the effects of Adaptive Learning Rate and Momentum of the Adam dynamics on saddle-point escaping and minima selection.
LFD-ProtoNet: Prototypical Network Based on Local Fisher Discriminant Analysis for Few-shot Learning
The prototypical network (ProtoNet) is a few-shot learning framework that performs metric learning and classification using the distance to prototype representations of each class.
$γ$-ABC: Outlier-Robust Approximate Bayesian Computation Based on a Robust Divergence Estimator
Approximate Bayesian computation (ABC) is a likelihood-free inference method that has been employed in various applications.
Pairwise Supervision Can Provably Elicit a Decision Boundary
A classifier built upon the representations is expected to perform well in downstream classification; however, little theory has been given in literature so far and thereby the relationship between similarity and classification has remained elusive.
Sequential Gallery for Interactive Visual Design Optimization
To help users respond to plane-search queries, we also propose using a gallery-based interface that provides options in the two-dimensional subspace arranged in an adaptive grid view.
Time-varying Gaussian Process Bandit Optimization with Non-constant Evaluation Time
If the black-box function varies with time, then time-varying Bayesian optimization is a promising framework.
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima
Stochastic Gradient Descent (SGD) and its variants are mainstream methods for training deep networks in practice.
Few-shot Domain Adaptation by Causal Mechanism Transfer
We take the structural equations in causal modeling as an example and propose a novel DA method, which is shown to be useful both theoretically and experimentally.
Bayesian interpretation of SGD as Ito process
The current interpretation of stochastic gradient descent (SGD) as a stochastic process lacks generality in that its numerical scheme restricts continuous-time dynamics as well as the loss function and the distribution of gradient noise.
Classification from Triplet Comparison Data
Although learning from triplet comparison data has been considered in many applications, an important fundamental question of whether we can learn a classifier only from triplet comparison data has remained unanswered.
Interactive Optimization of Generative Image Modeling using Sequential Subspace Search and Content-based Guidance
We present a human-in-the-optimization method that allows users to directly explore and search the latent vector space of generative image modeling.
Solving NP-Hard Problems on Graphs with Extended AlphaGo Zero
There have been increasing challenges to solve combinatorial optimization problems by machine learning.
Directing DNNs Attention for Facial Attribution Classification using Gradient-weighted Class Activation Mapping
Deep neural networks (DNNs) have a high accuracy on image classification tasks.
Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization
In this paper, we derive an unbiased risk estimator which can handle all of similarities/dissimilarities and unlabeled data.
Use of Ghost Cytometry to Differentiate Cells with Similar Gross Morphologic Characteristics
Imaging flow cytometry shows significant potential for increasing our understanding of heterogeneous and complex life systems and is useful for biomedical applications.
On Learning from Ghost Imaging without Imaging
Computational ghost imaging is an imaging technique in which an object is imaged from light collected using a single-pixel detector with no spatial resolution.
On Transformations in Stochastic Gradient MCMC
Although SGLD is designed for unbounded random variables, many practical models incorporate variables with boundaries such as non-negative ones or those in a finite interval.
PAC-Bayes Analysis of Sentence Representation
Learning sentence vectors from an unlabeled corpus has attracted attention because such vectors can represent sentences in a lower dimensional and continuous space.
Online Multiclass Classification Based on Prediction Margin for Partial Feedback
We consider the problem of online multiclass classification with partial feedback, where an algorithm predicts a class for a new instance in each round and only receives its correctness.
Multilevel Monte Carlo Variational Inference
We theoretically show that, with our method, the variance of the gradient estimator decreases as optimization proceeds and that a learning rate scheduler function helps improve the convergence.
Semi-Supervised Ordinal Regression Based on Empirical Risk Minimization
We further provide an estimation error bound to show that our risk estimator is consistent.
Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysis
However, existing definitions of the flatness are known to be sensitive to the rescaling of parameters.
Clipped Matrix Completion: A Remedy for Ceiling Effects
On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-rank completion.
On the Structural Sensitivity of Deep Convolutional Networks to the Directions of Fourier Basis Functions
Data-agnostic quasi-imperceptible perturbations on inputs are known to degrade recognition accuracy of deep convolutional networks severely.
Unsupervised Domain Adaptation Based on Source-guided Discrepancy
A previously proposed discrepancy that does not use the source domain labels requires high computational cost to estimate and may lead to a loose generalization error bound in the target domain.
Bayesian posterior approximation via greedy particle optimization
Another example is the Stein points (SP) method, which minimizes kernelized Stein discrepancy directly.
Variational Inference for Gaussian Process with Panel Count Data
We present the first framework for Gaussian-process-modulated Poisson processes when the temporal data appear in the form of panel counts.
Analysis of Minimax Error Rate for Crowdsourcing and Its Application to Worker Clustering Model
In this paper, we derive a minimax error rate under more practical setting for a broader class of crowdsourcing models including the DS model as a special case.
Gaussian Process Classification with Privileged Information by Soft-to-Hard Labeling Transfer
A state-of-the-art method of Gaussian process classification (GPC) with privileged information is GPC+, which incorporates privileged information into a noise term of the likelihood.
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
High sensitivity of neural networks against malicious perturbations on inputs causes security concerns.
A Quantum-Inspired Ensemble Method and Quantum-Inspired Forest Regressors
The contribution of this work is two-fold, a novel ensemble regression algorithm inspired by quantum mechanics and the theoretical connection between quantum interpretations and machine learning algorithms.
Variational Inference based on Robust Divergences
In this paper, based on Zellner's optimization and variational formulation of Bayesian inference, we propose an outlier-robust pseudo-Bayesian variational method by replacing the Kullback-Leibler divergence used for data fitting to a robust divergence such as the beta- and gamma-divergences.
On the Model Shrinkage Effect of Gamma Process Edge Partition Models
In order to ensure that the model shrinkage effect of the EPM works in an appropriate manner, we proposed two novel generative constructions of the EPM: CEPM incorporating constrained gamma priors, and DEPM incorporating Dirichlet priors instead of the gamma priors.
Evaluating the Variance of Likelihood-Ratio Gradient Estimators
The framework gives a natural derivation of the optimal estimator that can be interpreted as a special case of the likelihood-ratio method so that we can evaluate the optimal degree of practical techniques with it.
Expectation Propagation for t-Exponential Family Using Q-Algebra
Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters.
Bayesian Nonparametric Poisson-Process Allocation for Time-Sequence Modeling
We model the intensity of each sequence as an infinite mixture of latent functions, each of which is obtained using a function drawn from a Gaussian process.
Stochastic Divergence Minimization for Biterm Topic Model
As the emergence and the thriving development of social networks, a huge number of short texts are accumulated and need to be processed.
Convex Formulation of Multiple Instance Learning from Positive and Unlabeled Bags
Multiple instance learning (MIL) is a variation of traditional supervised learning problems where data (referred to as bags) are composed of sub-elements (referred to as instances) and only bag labels are available.
Differential Privacy without Sensitivity
The exponential mechanism is a general method to construct a randomized estimator that satisfies $(\varepsilon, 0)$-differential privacy.
Does Distributionally Robust Supervised Learning Give Robust Classifiers?
Since the DRSL is explicitly formulated for a distribution shift scenario, we naturally expect it to give a robust classifier that can aggressively handle shifted distributions.
Reparameterization trick for discrete variables
Low-variance gradient estimation is crucial for learning directed graphical models parameterized by neural networks, where the reparameterization trick is widely used for those with continuous variables.
Generative Adversarial Nets from a Density Ratio Estimation Perspective
Generative adversarial networks (GANs) are successful deep generative models.
Analysis of Variational Bayesian Latent Dirichlet Allocation: Weaker Sparsity Than MAP
Latent Dirichlet allocation (LDA) is a popular generative model of various objects such as texts and images, where an object is expressed as a mixture of latent topics.
Collapsed Variational Bayes Inference of Infinite Relational Model
The Infinite Relational Model (IRM) is a probabilistic model for relational data clustering that partitions objects into clusters based on observed relationships.
Quantum Annealing for Variational Bayes Inference
This paper presents studies on a deterministic annealing algorithm based on quantum annealing for variational Bayes (QAVB) inference, which can be seen as an extension of the simulated annealing for variational Bayes (SAVB) inference.
Quantum Annealing for Dirichlet Process Mixture Models with Applications to Network Clustering
We developed a new quantum annealing (QA) algorithm for Dirichlet process mixture (DPM) models based on the Chinese restaurant process (CRP).
Deterministic Single-Pass Algorithm for LDA
We develop a deterministic single-pass algorithm for latent Dirichlet allocation (LDA) in order to process received documents one at a time and then discard them in an excess text stream.
