Search Results for author:
Found 17 papers, 7 papers with code
Robustifying and Boosting Training-Free Neural Architecture Search
Nevertheless, the estimation ability of these metrics typically varies across different tasks, making it challenging to achieve robust and consistently good search performance on diverse tasks with only a single training-free metric.
Localized Zeroth-Order Prompt Optimization
Existing methodologies usually prioritize a global optimization for finding the global optimum, which however will perform poorly in certain tasks.
OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations
First-order optimization (FOO) algorithms are pivotal in numerous computational domains such as machine learning and signal denoising.
Use Your INSTINCT: INSTruction optimization usIng Neural bandits Coupled with Transformers
We perform instruction optimization for ChatGPT and use extensive experiments to show that our INSTINCT consistently outperforms the existing methods in different tasks, such as in various instruction induction tasks and the task of improving the zero-shot chain-of-thought instruction.
Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradients
To this end, we (a) introduce trajectory-informed gradient surrogates which is able to use the history of function queries during optimization for accurate and query-efficient gradient estimation, and (b) develop the technique of adaptive gradient correction using these gradient surrogates to mitigate the aforementioned disparity.
Sample-Then-Optimize Batch Neural Thompson Sampling
linear model), which is equivalently sampled from the GP posterior with the NTK as the kernel function.
Federated Neural Bandits
To better exploit the federated setting, FN-UCB adopts a weighted combination of two UCBs: $\text{UCB}^{a}$ allows every agent to additionally use the observations from the other agents to accelerate exploration (without sharing raw observations), while $\text{UCB}^{b}$ uses an NN with aggregated parameters for reward prediction in a similar way to federated averaging for supervised learning.
Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search
As a consequence, (a) the relationships among these metrics are unclear, (b) there is no theoretical interpretation for their empirical performances, and (c) there may exist untapped potential in existing training-free NAS, which probably can be unveiled through a unified theoretical understanding.
Neural Ensemble Search via Bayesian Sampling
Unfortunately, these NAS algorithms aim to select only one single well-performing architecture from their search spaces and thus have overlooked the capability of neural network ensemble (i. e., an ensemble of neural networks with diverse architectures) in achieving improved performance over a single final selected architecture.
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization
Recent years have witnessed a surging interest in Neural Architecture Search (NAS).
Tight Lower Complexity Bounds for Strongly Convex Finite-Sum Optimization
Finite-sum optimization plays an important role in the area of machine learning, and hence has triggered a surge of interest in recent years.
ISBNet: Instance-aware Selective Branching Networks
Recent years have witnessed growing interests in designing efficient neural networks and neural architecture search (NAS).
Understanding Architectures Learnt by Cell-based Neural Architecture Search
Neural architecture search (NAS) searches architectures automatically for given tasks, e. g., image classification and language modeling.
Dynamic Routing Networks
The deployment of deep neural networks in real-world applications is mostly restricted by their high inference costs.
Effective and Efficient Dropout for Deep Convolutional Neural Networks
However, many recent works show that the standard dropout is ineffective or even detrimental to the training of CNNs.
Efficient Memory Management for GPU-based Deep Learning Systems
They are achieved by exploiting the iterative nature of the training algorithm of deep learning to derive the lifetime and read/write order of all variables.
Randomness in Deconvolutional Networks for Visual Representation
Toward a deeper understanding on the inner work of deep neural networks, we investigate CNN (convolutional neural network) using DCN (deconvolutional network) and randomization technique, and gain new insights for the intrinsic property of this network architecture.
