Search Results for author:
Found 41 papers, 23 papers with code
MoVA: Adapting Mixture of Vision Experts to Multimodal Context
Although some large-scale pretrained vision encoders such as vision encoders in CLIP and DINOv2 have brought promising performance, we found that there is still no single vision encoder that can dominate various image content understanding, e. g., the CLIP vision encoder leads to outstanding results on general image understanding but poor performance on document or chart content.
Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance
Classifier-Free Guidance (CFG) has been widely used in text-to-image diffusion models, where the CFG scale is introduced to control the strength of text guidance on the whole image space.
CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching
We further attribute this phenomenon to the diffusion model's insufficient condition utilization, which is caused by its training paradigm.
Visual CoT: Unleashing Chain-of-Thought Reasoning in Multi-Modal Language Models
This paper presents Visual CoT, a novel pipeline that leverages the reasoning capabilities of multi-modal large language models (MLLMs) by incorporating visual Chain-of-Thought (CoT) reasoning.
Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptation
We introduce MOTIA Mastering Video Outpainting Through Input-Specific Adaptation, a diffusion-based pipeline that leverages both the intrinsic data-specific patterns of the source video and the image/video generative prior for effective outpainting.
FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis
In this study, we delve into the generation of high-resolution images from pre-trained diffusion models, addressing persistent challenges, such as repetitive patterns and structural distortions, that emerge when models are applied beyond their trained resolutions.
AnimateLCM: Accelerating the Animation of Personalized Diffusion Models and Adapters with Decoupled Consistency Learning
We validate the proposed strategy in image-conditioned video generation and layout-conditioned video generation, all achieving top-performing results.
Towards Large-scale Masked Face Recognition
During the COVID-19 coronavirus epidemic, almost everyone is wearing masks, which poses a huge challenge for deep learning-based face recognition algorithms.
Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection
We observe that different regions of interest in the visual feature map are suitable for performing query classification and box localization tasks, even for the same object.
RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths
Text-to-image generation has recently witnessed remarkable achievements.
Ranked #11 on
Text-to-Image Generation
on MS COCO
Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising
To address this challenge, we introduce a novel paradigm dubbed as Gen-L-Video, capable of extending off-the-shelf short video diffusion models for generating and editing videos comprising hundreds of frames with diverse semantic segments without introducing additional training, all while preserving content consistency.
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
The HoP approach is straightforward: given the current timestamp t, we generate a pseudo Bird's-Eye View (BEV) feature of timestamp t-k from its adjacent frames and utilize this feature to predict the object set at timestamp t-k. Our approach is motivated by the observation that enforcing the detector to capture both the spatial location and temporal motion of objects occurring at historical timestamps can lead to more accurate BEV feature learning.
Ranked #3 on
3D Object Detection
on nuScenes Camera Only
UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors
Bridging this semantic gap now requires case-by-case algorithm design which is time-consuming and heavily relies on experienced adjustment.
Masked Autoencoders Are Stronger Knowledge Distillers
In MKD, random patches of the input image are masked, and the corresponding missing feature is recovered by forcing it to imitate the output of the teacher.
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
DETRs with Collaborative Hybrid Assignments Training
This new training scheme can easily enhance the encoder's learning ability in end-to-end detectors by training the multiple parallel auxiliary heads supervised by one-to-many label assignments such as ATSS and Faster RCNN.
Ranked #1 on
Object Detection
on LVIS v1.0 val
(using extra training data)
Large-batch Optimization for Dense Visual Predictions
To address this challenge, we propose a simple yet effective algorithm, named Adaptive Gradient Variance Modulator (AGVM), which can train dense visual predictors with very large batch size, enabling several benefits more appealing than prior arts.
Towards Robust Face Recognition with Comprehensive Search
To eliminate the bias of single-aspect research and provide an overall understanding of the face recognition model design, we first carefully design the search space for each aspect, then a comprehensive search method is introduced to jointly search optimal data cleaning, architecture, and loss function design.
Unifying Visual Perception by Dispersible Points Learning
The method, called UniHead, views different visual perception tasks as the dispersible points learning via the transformer encoder architecture.
Rethinking Robust Representation Learning Under Fine-grained Noisy Faces
To better understand this, we reformulate the noise type of each class in a more fine-grained manner as N-identities|K^C-clusters.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Finally, we integrate configurable operators and DSMs into a unified search space and search with a Reinforcement Learning-based search algorithm to fully explore the optimal combination of the operators.
Ranked #12 on
Neural Architecture Search
on ImageNet
UniFormer: Unifying Convolution and Self-attention for Visual Recognition
Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing to tackle both redundancy and dependency for efficient and effective representation learning.
Ranked #153 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 9% and 71. 2% top-1 accuracy respectively.
Self-slimmed Vision Transformer
Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs by dynamic token aggregation.
INTERN: A New Learning Paradigm Towards General Vision
Enormous waves of technological innovations over the past several years, marked by the advances in AI technologies, are profoundly reshaping the industry and the society.
Rectifying the Data Bias in Knowledge Distillation
In this paper, we first quantitatively define the uniformity of the sampled data for training, providing a unified view for methods that learn from biased data.
Ranked #1 on
Face Verification
on IJB-C
(training dataset metric)
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Recently, transformer and multi-layer perceptron (MLP) architectures have achieved impressive results on various vision tasks.
Ranked #239 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatial-Temporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 8% and 71. 4% top-1 accuracy respectively.
Ranked #8 on
Action Recognition
on Something-Something V1
Self-Slimming Vision Transformer
Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs by dynamic token aggregation.
FNAS: Uncertainty-Aware Fast Neural Architecture Search
Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times.
Switchable K-Class Hyperplanes for Noise-Robust Representation Learning
When collaborated with the popular ArcFace on million-level data representation learning, we found that the switchable manner in SKH can effectively eliminate the gradient conflict generated by real-world label noise on a single K-class hyperplane.
Discriminability Distillation in Group Representation Learning
We show the discrimiability knowledge has good properties that can be distilled by a light-weight distillation network and can be generalized on the unseen target set.
1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
This technical report introduces our winning solution to the spatio-temporal action localization track, AVA-Kinetics Crossover, in ActivityNet Challenge 2020.
1st Place Solutions for OpenImage2019 -- Object Detection and Instance Segmentation
Given such good instance bounding box, we further design a simple instance-level semantic segmentation pipeline and achieve the 1st place on the segmentation challenge.
Revisiting the Sibling Head in Object Detector
The ``shared head for classification and localization'' (sibling head), firstly denominated in Fast RCNN~\cite{girshick2015fast}, has been leading the fashion of the object detection community in the past five years.
Ranked #67 on
Object Detection
on COCO test-dev
KPNet: Towards Minimal Face Detector
The small receptive field and capacity of minimal neural networks limit their performance when using them to be the backbone of detectors.
Top-1 Solution of Multi-Moments in Time Challenge 2019
In this technical report, we briefly introduce the solutions of our team 'Efficient' for the Multi-Moments in Time challenge in ICCV 2019.
Towards Flops-constrained Face Recognition
Large scale face recognition is challenging especially when the computational budget is limited.
Transductive Centroid Projection for Semi-supervised Large-scale Recognition
It is inspired by the observation of the weights in classification layer (called extit{anchors}) converge to the central direction of each class in hyperspace.
Beyond Trade-off: Accelerate FCN-based Face Detector with Higher Accuracy
Fully convolutional neural network (FCN) has been dominating the game of face detection task for a few years with its congenital capability of sliding-window-searching with shared kernels, which boiled down all the redundant calculation, and most recent state-of-the-art methods such as Faster-RCNN, SSD, YOLO and FPN use FCN as their backbone.
Region-based Quality Estimation Network for Large-scale Person Re-identification
One of the major restrictions on the performance of video-based person re-id is partial noise caused by occlusion, blur and illumination.
