Search Results for author:
Found 21 papers, 8 papers with code
Communication Efficient Distributed Training with Distributed Lion
The Lion optimizer has been a promising competitor with the AdamW for training large AI models, with advantages on memory, computation, and sample efficiency.
Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows
Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e. g., ~4 AP on COCO/LVIS) over other fast SAM models.
Ranked #3 on
Zero-Shot Instance Segmentation
on LVIS v1.0 val
AutoML-GPT: Automatic Machine Learning with GPT
Ultimately, with this prompt paragraph, AutoML-GPT will automatically conduct the experiments from data processing to model architecture, hyperparameter tuning, and predicted training log.
FlowGrad: Controlling the Output of Generative ODEs With Gradients
To further accelerate the computation of the back-propagation, we propose to use a non-uniform discretization to approximate the ODE trajectory, where we measure how straight the trajectory is and gather the straight parts into one discretization step.
PathFusion: Path-consistent Lidar-Camera Deep Feature Fusion
Fusing 3D LiDAR features with 2D camera features is a promising technique for enhancing the accuracy of 3D detection, thanks to their complementary physical properties.
Fast Point Cloud Generation with Straight Flows
We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods.
Neural Volumetric Mesh Generator
We first propose a diffusion-based generative model to tackle this problem by generating voxelized shapes with close-to-reality outlines and structures.
First Hitting Diffusion Models for Generating Manifold, Graph and Categorical Data
We propose a family of First Hitting Diffusion Models (FHDM), deep generative models that generate data with a diffusion process that terminates at a random first hitting time.
Diffusion-based Molecule Generation with Informative Prior Bridges
AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development.
Let us Build Bridges: Understanding and Extending Diffusion Generative Models
Diffusion-based generative models have achieved promising results recently, but raise an array of open questions in terms of conceptual understanding, theoretical analysis, algorithm improvement and extensions to discrete, structured, non-Euclidean domains.
Residual Mixture of Experts
In this paper, we propose Residual Mixture of Experts (RMoE), an efficient training pipeline for MoE vision transformers on downstream tasks, such as segmentation and detection.
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima.
FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization
We approach text-to-image generation by combining the power of the retrained CLIP representation with an off-the-shelf image generator (GANs), optimizing in the latent space of GAN to find images that achieve maximum CLIP score with the given input text.
Ranked #48 on
Text-to-Image Generation
on MS COCO
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures.
Centroid Transformers: Learning to Abstract with Attention
Self-attention, as the key block of transformers, is a powerful mechanism for extracting features from the inputs.
Ranked #618 on
Image Classification
on ImageNet
Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough
Despite the great success of deep learning, recent works show that large deep neural networks are often highly redundant and can be significantly reduced in size.
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
Recently, Liu et al.[19] proposed a splitting steepest descent (S2D) method that jointly optimizes the neural parameters and architectures based on progressively growing network structures by splitting neurons into multiple copies in a steepest descent fashion.
Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.
Splitting Steepest Descent for Growing Neural Architectures
We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs.
Path-Invariant Map Networks
Optimizing a network of maps among a collection of objects/domains (or map synchronization) is a central problem across computer vision and many other relevant fields.
