Search Results for author:
Found 67 papers, 42 papers with code
Variable-Bitrate Neural Compression via Bayesian Arithmetic Coding
Deep Bayesian latent variable models have enabled new approaches to both model and data compression.
GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning
To tackle these challenges, we present GenView, a controllable framework that augments the diversity of positive views leveraging the power of pretrained generative models while preserving semantics.
RAP-SAM: Towards Real-Time All-Purpose Segment Anything
Segment Anything Model (SAM) is one remarkable model that can achieve generalized segmentation.
Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges
While there have been extensive studies on optimization characteristics showing the global optimality of neural collapse, little research has been done on the generalization behaviors during the occurrence of NC.
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model
Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions.
ShadowNet for Data-Centric Quantum System Learning
Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality.
Deformable Mixer Transformer with Gating for Multi-Task Learning of Dense Prediction
In this work, we present a novel MTL model by combining both merits of deformable CNN and query-based Transformer with shared gating for multi-task learning of dense prediction.
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.
Towards Open Vocabulary Learning: A Survey
To our knowledge, this is the first comprehensive literature review of open vocabulary learning.
Computationally-Efficient Neural Image Compression with Shallow Decoders
We theoretically formalize the intuition behind, and our experimental results establish a new frontier in the trade-off between rate-distortion and decoding complexity for neural image compression.
Neural Collapse Inspired Federated Learning with Non-iid Data
One of the challenges in federated learning is the non-independent and identically distributed (non-iid) characteristics between heterogeneous devices, which cause significant differences in local updates and affect the performance of the central server.
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System
Automated machine learning (AutoML) seeks to build ML models with minimal human effort.
General Rotation Invariance Learning for Point Clouds via Weight-Feature Alignment
Compared to 2D images, 3D point clouds are much more sensitive to rotations.
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Ranked #3 on
Few-Shot Class-Incremental Learning
on CUB-200-2011
DeMT: Deformable Mixer Transformer for Multi-Task Learning of Dense Prediction
In this work, we present a novel MTL model by combining both merits of deformable CNN and query-based Transformer for multi-task learning of dense prediction.
Understanding Imbalanced Semantic Segmentation Through Neural Collapse
Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
Multi-Task Learning with Knowledge Distillation for Dense Prediction
With the less sensitive divergence, our knowledge distillation with an alternative match is applied for capturing inter-task and intra-task information between the teacher model and the student model of each task, thereby learning more "dark knowledge" for effective distillation.
Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification
Using these findings, we propose a method that uses loss dynamics to probe whether a QC may be more effective than a classical classifier on a particular learning task.
Towards Theoretically Inspired Neural Initialization Optimization
With NIO, we improve the classification performance of a variety of neural architectures on CIFAR-10, CIFAR-100, and ImageNet.
Eliminating Gradient Conflict in Reference-based Line-Art Colorization
To understand the instability in training, we detect the gradient flow of attention and observe gradient conflict among attention branches.
SFNet: Faster, Accurate, and Domain Agnostic Semantic Segmentation via Semantic Flow
In this paper, we focus on exploring effective methods for faster, accurate, and domain agnostic semantic segmentation.
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm
Motivated by biological evolution, this paper explains the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derives that both have consistent mathematical formulation.
Multi-Task Learning with Multi-Query Transformer for Dense Prediction
We first model each task with a task-relevant query.
Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning
Class imbalance distribution widely exists in real-world engineering.
Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation
To the best of our knowledge, we are the first to solve the PPS problem via \textit{a unified and end-to-end transformer model.
Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?
Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class.
Ranked #26 on
Long-tail Learning
on CIFAR-10-LT (ρ=100)
ART-Point: Improving Rotation Robustness of Point Cloud Classifiers via Adversarial Rotation
Specifically, our proposed framework named ART-Point regards the rotation of the point cloud as an attack and improves rotation robustness by training the classifier on inputs with Adversarial RoTations.
Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors
Finally, we provide an optimized JAX library called {\em UQDeepONet} that can accommodate large model architectures, large ensemble sizes, as well as large data-sets with excellent parallel performance on accelerated hardware, thereby enabling uncertainty quantification for DeepONets in realistic large-scale applications.
An Introduction to Neural Data Compression
Neural compression is the application of neural networks and other machine learning methods to data compression.
TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers
Detection Transformer (DETR) and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors.
Ranked #4 on
Video Object Detection
on ImageNet VID
(using extra training data)
PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation
The Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously.
Dual-Flow Transformation Network for Deformable Image Registration with Region Consistency Constraint
In this paper, we present a novel dual-flow transformation network with region consistency constraint which maximizes the similarity of ROIs within a pair of images and estimates both global and region spatial transformations simultaneously.
Towards Empirical Sandwich Bounds on the Rate-Distortion Function
By contrast, this paper makes the first attempt at an algorithm for sandwiching the R-D function of a general (not necessarily discrete) source requiring only i. i. d.
View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums
Different frequency bands in the trajectory spectrums could hierarchically reflect agents' motion preferences at different scales.
Ranked #3 on
Trajectory Prediction
on ETH/UCY
Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networks
We develop biologically plausible training mechanisms for self-supervised learning (SSL) in deep networks.
Insights from Generative Modeling for Neural Video Compression
While recent machine learning research has revealed connections between deep generative models such as VAEs and rate-distortion losses used in learned compression, most of this work has focused on images.
Improving Video Instance Segmentation via Temporal Pyramid Routing
To incorporate both temporal and scale information, we propose a Temporal Pyramid Routing (TPR) strategy to conditionally align and conduct pixel-level aggregation from a feature pyramid pair of two adjacent frames.
BoundarySqueeze: Image Segmentation as Boundary Squeezing
This module is used to squeeze the object boundary from both inner and outer directions, which contributes to precise mask representation.
Lower Bounding Rate-Distortion From Samples
The rate-distortion function tells us the minimal number of bits on average to compress a random object within a given distortion tolerance.
SCALE SPACE FLOW WITH AUTOREGRESSIVE PRIORS
There has been a recent surge of interest in neural video compression models that combines data-driven dimensionality reduction with learned entropy coding.
Output-Weighted Sampling for Multi-Armed Bandits with Extreme Payoffs
We present a new type of acquisition functions for online decision making in multi-armed and contextual bandit problems with extreme payoffs.
Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search
Our method enables differentiable sparsification, and keeps the derived architecture equivalent to that of Engine-cell, which further improves the consistency between search and evaluation.
EnTranNAS: Towards Closing the Gap between the Architectures in Search and Evaluation
The Engine-cell is differentiable for architecture search, while the Transit-cell only transits the current sub-graph by architecture derivation.
Explicit Learning Topology for Differentiable Neural Architecture Search
Differentiable neural architecture search (NAS) has gained much success in discovering more flexible and diverse cell types.
Generative Video Compression as Hierarchical Variational Inference
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
Stretchable Cells Help DARTS Search Better
However, even for this consistent search, the searched cells often suffer from poor performance, especially for the supernet with fewer layers, as current DARTS methods are prone to wide and shallow cells, and this topology collapse induces sub-optimal searched cells.
Hierarchical Autoregressive Modeling for Neural Video Compression
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Coding
By doing so, our network for search at each update satisfies the sparsity constraint and is efficient to train.
Improving Inference for Neural Image Compression
We consider the problem of lossy image compression with deep latent variable models.
Bayesian differential programming for robust systems identification under uncertainty
This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems.
Spatial Pyramid Based Graph Reasoning for Semantic Segmentation
The convolution operation suffers from a limited receptive filed, while global modeling is fundamental to dense prediction tasks, such as semantic segmentation.
Variational Bayesian Quantization
Our experimental results demonstrate the importance of taking into account posterior uncertainties, and show that image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates using only a single standard VAE.
Lifted Hybrid Variational Inference
We demonstrate that the proposed variational methods are both scalable and can take advantage of approximate model symmetries, even in the presence of a large amount of continuous evidence.
Dynamical System Inspired Adaptive Time Stepping Controller for Residual Network Families
We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance.
SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances.
Ranked #8 on
Panoptic Segmentation
on Cityscapes test
Expectation-Maximization Attention Networks for Semantic Segmentation
It is designed to compute the representation of each position by a weighted sum of the features at all positions.
Ranked #11 on
Semantic Segmentation
on COCO-Stuff test
Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
Such models can be nowadays deployed on large patient-specific topologies of systemic arterial networks and return detailed predictions on flow patterns, wall shear stresses, and pulse wave propagation.
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems
We present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems.
Physics-informed deep generative models
We consider the application of deep generative models in propagating uncertainty through complex physical systems.
Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
We present a deep learning framework for quantifying and propagating uncertainty in systems governed by non-linear differential equations using physics-informed neural networks.
Optimization Algorithm Inspired Deep Neural Network Structure Design
In this paper, we propose the hypothesis that the neural network structure design can be inspired by optimization algorithms and a faster optimization algorithm may lead to a better neural network structure.
Joint Sub-bands Learning with Clique Structures for Wavelet Domain Super-Resolution
To solve these problems, we propose the Super-Resolution CliqueNet (SRCliqueNet) to reconstruct the high resolution (HR) image with better textural details in the wavelet domain.
Scalable Neural Network Compression and Pruning Using Hard Clustering and L1 Regularization
We propose a simple and easy to implement neural network compression algorithm that achieves results competitive with more complicated state-of-the-art methods.
Convolutional Neural Networks with Alternately Updated Clique
In contrast to prior networks, there are both forward and backward connections between any two layers in the same block.
Automatic Parameter Tying in Neural Networks
Recently, there has been growing interest in methods that perform neural network compression, namely techniques that attempt to substantially reduce the size of a neural network without significant reduction in performance.
