Search Results for author:
Found 32 papers, 6 papers with code
Fermi-Bose Machine
Distinct from human cognitive processing, deep neural networks trained by backpropagation can be easily fooled by adversarial examples.
Enhanced Short Text Modeling: Leveraging Large Language Models for Topic Refinement
Crafting effective topic models for brief texts, like tweets and news headlines, is essential for capturing the swift shifts in social dynamics.
An optimization-based equilibrium measure describes non-equilibrium steady state dynamics: application to edge of chaos
Here, we treat searching for the steady state as an optimization problem, and construct an approximate potential closely related to the speed of the dynamics, and find that searching for the ground state of this potential is equivalent to running a stochastic gradient dynamics.
Spiking mode-based neural networks
Our work thus derives a mode-based learning rule for spiking neural networks.
Meta predictive learning model of languages in neural circuits
Therefore, our model provides a starting point to investigate the connection among brain computation, next-token prediction and general intelligence.
What are Public Concerns about ChatGPT? A Novel Self-Supervised Neural Topic Model Tells You
The recently released artificial intelligence conversational agent, ChatGPT, has gained significant attention in academia and real life.
Eight challenges in developing theory of intelligence
A good theory of mathematical beauty is more practical than any current observation, as new predictions of physical reality can be verified self-consistently.
Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings
Dynamical mean-field theory is a powerful physics tool used to analyze the typical behavior of neural networks, where neurons can be recurrently connected, or multiple layers of neurons can be stacked.
Statistical mechanics of continual learning: variational principle and mean-field potential
A variational Bayesian learning setting is thus proposed, where the neural networks are trained in a field-space, rather than gradient-ill-defined discrete-weight space, and furthermore, weight uncertainty is naturally incorporated, and modulates synaptic resources among tasks.
Spectrum of non-Hermitian deep-Hebbian neural networks
Neural networks with recurrent asymmetric couplings are important to understand how episodic memories are encoded in the brain.
Emergence of hierarchical modes from deep learning
Large-scale deep neural networks consume expensive training costs, but the training results in less-interpretable weight matrices constructing the networks.
Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systems
Binary perceptron is a fundamental model of supervised learning for the non-convex optimization, which is a root of the popular deep learning.
Variational Gaussian Topic Model with Invertible Neural Projections
Based on the variational auto-encoder, the proposed VaGTM models each topic with a multivariate Gaussian in decoder to incorporate word relatedness.
Ensemble perspective for understanding temporal credit assignment
Recurrent neural networks are widely used for modeling spatio-temporal sequences in both nature language processing and neural population dynamics.
Data-driven effective model shows a liquid-like deep learning
Here, we propose a statistical mechanics framework by directly building a least structured model of the high-dimensional weight space, considering realistic structured data, stochastic gradient descent training, and the computational depth of neural networks.
Relationship between manifold smoothness and adversarial vulnerability in deep learning with local errors
Here, we establish a fundamental relationship between geometry of hidden representations (manifold perspective) and the generalization capability of the deep networks.
Weakly-correlated synapses promote dimension reduction in deep neural networks
Here we propose a simplified model of dimension reduction, taking into account pairwise correlations among synapses, to reveal the mechanism underlying how the synaptic correlations affect dimension reduction.
Classification and Recognition of Encrypted EEG Data Neural Network
With the rapid development of Machine Learning technology applied in electroencephalography (EEG) signals, Brain-Computer Interface (BCI) has emerged as a novel and convenient human-computer interaction for smart home, intelligent medical and other Internet of Things (IoT) scenarios.
Privacy-Preserving Approach PBCN in Social Network With Differential Privacy
This proposal is composed of five algorithms including random disturbance based on clustering, graph reconstruction after disturbing degree sequence and noise nodes generation, etc.
Learning credit assignment
Therefore, our model learns the credit assignment leading to the decision, and predicts an ensemble of sub-networks that can accomplish the same task, thereby providing insights toward understanding the macroscopic behavior of deep learning through the lens of distinct roles of synaptic weights.
Variational mean-field theory for training restricted Boltzmann machines with binary synapses
Here, we propose a variational mean-field theory in which the distribution of synaptic weights is considered.
Statistical physics of unsupervised learning with prior knowledge in neural networks
Here, we propose a statistical physics model of unsupervised learning with prior knowledge, revealing that the sensory inputs drive a series of continuous phase transitions related to spontaneous intrinsic-symmetry breaking.
Minimal model of permutation symmetry in unsupervised learning
Remarkably, we find that the embedded correlation between two receptive fields of hidden units reduces the critical data size.
Mechanisms of dimensionality reduction and decorrelation in deep neural networks
Deep neural networks are widely used in various domains.
Random active path model of deep neural networks with diluted binary synapses
Deep learning has become a powerful and popular tool for a variety of machine learning tasks.
Role of zero synapses in unsupervised feature learning
Synapses in real neural circuits can take discrete values, including zero (silent or potential) synapses.
Reinforced stochastic gradient descent for deep neural network learning
Therefore, it is highly desirable to design an efficient algorithm to escape from these saddle points and reach a parameter region of better generalization capabilities.
Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
Our analysis confirms an entropy crisis preceding the non-convergence of the message passing equation, suggesting a discontinuous phase transition as a key characteristic of the restricted Boltzmann machine.
Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
This study deepens our understanding of unsupervised learning from a finite number of data, and may provide insights into its role in training deep networks.
Advanced Mean Field Theory of Restricted Boltzmann Machine
Learning in restricted Boltzmann machine is typically hard due to the computation of gradients of log-likelihood function.
Origin of the computational hardness for learning with binary synapses
Supervised learning in a binary perceptron is able to classify an extensive number of random patterns by a proper assignment of binary synaptic weights.
Entropy landscape of solutions in the binary perceptron problem
The geometrical organization is elucidated by the entropy landscape from a reference configuration and of solution-pairs separated by a given Hamming distance in the solution space.
