Search Results for author:
Found 256 papers, 87 papers with code
DGMamba: Domain Generalization via Generalized State Space Model
SPR strives to encourage the model to concentrate more on objects rather than context, consisting of two designs: Prior-Free Scanning~(PFS), and Domain Context Interchange~(DCI).
DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queries
Modern video segmentation methods adopt object queries to perform inter-frame association and demonstrate satisfactory performance in tracking continuously appearing objects despite large-scale motion and transient occlusion.
Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction
We tackle the challenge of efficiently reconstructing a 3D asset from a single image with growing demands for automated 3D content creation pipelines.
AgentStudio: A Toolkit for Building General Virtual Agents
We have open-sourced the environments, datasets, benchmarks, and interfaces to promote research towards developing general virtual agents for the future.
Explore In-Context Segmentation via Latent Diffusion Models
In-context segmentation has drawn more attention with the introduction of vision foundation models.
Data Augmentation in Human-Centric Vision
This survey presents a comprehensive analysis of data augmentation techniques in human-centric vision tasks, a first of its kind in the field.
Point Cloud Mamba: Point Cloud Learning via State Space Model
To enable more effective processing of 3-D point cloud data by Mamba, we propose a novel Consistent Traverse Serialization to convert point clouds into 1-D point sequences while ensuring that neighboring points in the sequence are also spatially adjacent.
Have Seen Me Before? Automating Dataset Updates Towards Reliable and Timely Evaluation
The basic idea is to generate unseen and high-quality testing samples based on existing ones to mitigate leakage issues.
See the Unseen: Better Context-Consistent Knowledge-Editing by Noises
In this paper, we empirically find that the effects of different contexts upon LLMs in recalling the same knowledge follow a Gaussian-like distribution.
Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket
To the best of our knowledge, this is the first time that the SNN achieves 80+% accuracy on ImageNet.
Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement
The generalization capability of existing image restoration and enhancement (IRE) methods is constrained by the limited pre-trained datasets, making it difficult to handle agnostic inputs such as different degradation levels and scenarios beyond their design scopes.
Instant3D: Instant Text-to-3D Generation
Once trained, Instant3D is able to create a 3D object for an unseen text prompt in less than one second with a single run of a feedforward network.
Towards Garment Sewing Pattern Reconstruction from a Single Image
In this work, we explore the challenging problem of recovering garment sewing patterns from daily photos for augmenting these applications.
Skywork: A More Open Bilingual Foundation Model
In this technical report, we present Skywork-13B, a family of large language models (LLMs) trained on a corpus of over 3. 2 trillion tokens drawn from both English and Chinese texts.
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection
Besides, we also observe the theoretical benefits of the LSC coefficient scaling of UNet in the stableness of hidden features and gradient and also robustness.
Heterogenous Memory Augmented Neural Networks
It has been shown that semi-parametric methods, which combine standard neural networks with non-parametric components such as external memory modules and data retrieval, are particularly helpful in data scarcity and out-of-distribution (OOD) scenarios.
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
To address these limitations, we present a novel cascaded motion diffusion model, DiffDance, designed for high-resolution, long-form dance generation.
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning
More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage).
Ranked #1 on
Node Property Prediction
on ogbn-arxiv
Decoupled Prioritized Resampling for Offline RL
OPER is a plug-and-play component for offline RL algorithms.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
Evolving Knowledge Mining for Class Incremental Segmentation
In this paper, we for the first time investigate the efficient multi-grained knowledge reuse for CISS, and propose a novel method, Evolving kNowleDge minING (ENDING), employing a frozen backbone.
Class-Incremental Semantic Segmentation
Knowledge Distillation
Improving and Benchmarking Offline Reinforcement Learning Algorithms
Recently, Offline Reinforcement Learning (RL) has achieved remarkable progress with the emergence of various algorithms and datasets.
Efficient Diffusion Policies for Offline Reinforcement Learning
2) It is incompatible with maximum likelihood-based RL algorithms (e. g., policy gradient methods) as the likelihood of diffusion models is intractable.
Generative Table Pre-training Empowers Models for Tabular Prediction
Recently, the topic of table pre-training has attracted considerable research interest.
Nonparametric Generative Modeling with Conditional Sliced-Wasserstein Flows
Sliced-Wasserstein Flow (SWF) is a promising approach to nonparametric generative modeling but has not been widely adopted due to its suboptimal generative quality and lack of conditional modeling capabilities.
A Review of Deep Learning for Video Captioning
Video captioning (VC) is a fast-moving, cross-disciplinary area of research that bridges work in the fields of computer vision, natural language processing (NLP), linguistics, and human-computer interaction.
Exploring Incompatible Knowledge Transfer in Few-shot Image Generation
To this end, we propose knowledge truncation to mitigate this issue in FSIG, which is a complementary operation to knowledge preservation and is implemented by a lightweight pruning-based method.
CoSDA: Continual Source-Free Domain Adaptation
Recently, SFDA has gained popularity due to the need to protect the data privacy of the source domain, but it suffers from catastrophic forgetting on the source domain due to the lack of data.
InceptionNeXt: When Inception Meets ConvNeXt
Inspired by the long-range modeling ability of ViTs, large-kernel convolutions are widely studied and adopted recently to enlarge the receptive field and improve model performance, like the remarkable work ConvNeXt which employs 7x7 depthwise convolution.
MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer
To solve this issue, we propose a Masked Diffusion Transformer (MDT) that introduces a mask latent modeling scheme to explicitly enhance the DPMs' ability to contextual relation learning among object semantic parts in an image.
Ranked #2 on
Image Generation
on ImageNet 256x256
D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory
Kohn-Sham Density Functional Theory (KS-DFT) has been traditionally solved by the Self-Consistent Field (SCF) method.
Contrastive Video Question Answering via Video Graph Transformer
CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning.
Ranked #11 on
Video Question Answering
on NExT-QA
(using extra training data)
Bag of Tricks for Training Data Extraction from Language Models
With the advance of language models, privacy protection is receiving more attention.
Better Diffusion Models Further Improve Adversarial Training
Under the $\ell_\infty$-norm threat model with $\epsilon=8/255$, our models achieve $70. 69\%$ and $42. 67\%$ robust accuracy on CIFAR-10 and CIFAR-100, respectively, i. e. improving upon previous state-of-the-art models by $+4. 58\%$ and $+8. 03\%$.
Learning to Optimize for Reinforcement Learning
Reinforcement learning (RL) is essentially different from supervised learning and in practice these learned optimizers do not work well even in simple RL tasks.
Visual Imitation Learning with Patch Rewards
Specifically, we present Adversarial Imitation Learning with Patch Rewards (PatchAIL), which employs a patch-based discriminator to measure the expertise of different local parts from given images and provide patch rewards.
Does Federated Learning Really Need Backpropagation?
BAFFLE is 1) memory-efficient and easily fits uploading bandwidth; 2) compatible with inference-only hardware optimization and model quantization or pruning; and 3) well-suited to trusted execution environments, because the clients in BAFFLE only execute forward propagation and return a set of scalars to the server.
Reinforcement Learning from Diverse Human Preferences
The complexity of designing reward functions has been a major obstacle to the wide application of deep reinforcement learning (RL) techniques.
MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation
Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data.
STPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i. e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective.
Position-guided Text Prompt for Vision-Language Pre-training
In this work, we propose a novel Position-guided Text Prompt (PTP) paradigm to enhance the visual grounding ability of cross-modal models trained with VLP.
Ranked #5 on
Zero-Shot Cross-Modal Retrieval
on COCO 2014
NoiSER: Noise is All You Need for Low-Light Image Enhancement
In this paper, we present an embarrassingly simple yet effective solution to a seemingly impossible mission, low-light image enhancement (LLIE) without access to any task-related data.
Decoupled Cross-Scale Cross-View Interaction for Stereo Image Enhancement in The Dark
Specifically, we present a decoupled interaction module (DIM) that aims for sufficient dual-view information interaction.
MetaFormer Baselines for Vision
By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85. 5% at 224x224 resolution, under normal supervised training without external data or distillation.
Ranked #2 on
Domain Generalization
on ImageNet-C
(using extra training data)
Towards Sustainable Self-supervised Learning
In this work, we explore a sustainable SSL framework with two major challenges: i) learning a stronger new SSL model based on the existing pretrained SSL model, also called as "base" model, in a cost-friendly manner, ii) allowing the training of the new model to be compatible with various base models.
Ranked #1 on
Semantic Segmentation
on ImageNet-S
RPM: Generalizable Behaviors for Multi-Agent Reinforcement Learning
Despite the recent advancement in multi-agent reinforcement learning (MARL), the MARL agents easily overfit the training environment and perform poorly in the evaluation scenarios where other agents behave differently.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Boosting Offline Reinforcement Learning via Data Rebalancing
Therefore, we propose a simple yet effective method to boost offline RL algorithms based on the observation that resampling a dataset keeps the distribution support unchanged.
Mutual Information Regularized Offline Reinforcement Learning
In this work, we propose a novel MISA framework to approach offline RL from the perspective of Mutual Information between States and Actions in the dataset by directly constraining the policy improvement direction.
Efficient Offline Policy Optimization with a Learned Model
Instead of planning with the expensive MCTS, we use the learned model to construct an advantage estimation based on a one-step rollout.
AdaptivePose++: A Powerful Single-Stage Network for Multi-Person Pose Regression
With the proposed body representation, we further deliver a compact single-stage multi-person pose regression network, termed as AdaptivePose.
LPT: Long-tailed Prompt Tuning for Image Classification
For better effectiveness, we divide prompts into two groups: 1) a shared prompt for the whole long-tailed dataset to learn general features and to adapt a pretrained model into target domain; and 2) group-specific prompts to gather group-specific features for the samples which have similar features and also to empower the pretrained model with discrimination ability.
Ranked #1 on
Long-tail Learning
on CIFAR-100-LT (ρ=100)
(using extra training data)
Spikformer: When Spiking Neural Network Meets Transformer
Spikformer (66. 3M parameters) with comparable size to SEW-ResNet-152 (60. 2M, 69. 26%) can achieve 74. 81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models.
Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models
Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point.
Video Graph Transformer for Video Question Answering
VGT's uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled cross-modal Transformer for answer classification.
Ranked #18 on
Video Question Answering
on NExT-QA
(using extra training data)
Value-Consistent Representation Learning for Data-Efficient Reinforcement Learning
Recently, visual representation learning has been shown to be effective and promising for boosting sample efficiency in RL.
EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Engine
EnvPool is open-sourced at https://github. com/sail-sg/envpool.
Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Tasks
To solve these problems, we first theoretically show that on an auto-encoder of a two/one-layered convolution encoder/decoder, MRP can capture all discriminative features of each potential semantic class in the pretraining dataset.
$O(N^2)$ Universal Antisymmetry in Fermionic Neural Networks
In this paper, we substitute the Slater determinant with a pairwise antisymmetry construction, which is easy to implement and can reduce the computational cost to $O(N^2)$.
Inception Transformer
Recent studies show that Transformer has strong capability of building long-range dependencies, yet is incompetent in capturing high frequencies that predominantly convey local information.
Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization
To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics to imitate the distribution of real data, so that the performance degradation is alleviated.
Improving Vision Transformers by Revisiting High-frequency Components
Inspired by this finding, we first investigate the effects of existing techniques for improving ViT models from a new frequency perspective, and find that the success of some techniques (e. g., RandAugment) can be attributed to the better usage of the high-frequency components.
Ranked #2 on
Domain Generalization
on Stylized-ImageNet
Mugs: A Multi-Granular Self-Supervised Learning Framework
It provides complementary instance supervision to IDS via an extra alignment on local neighbors, and scatters different local-groups separately to increase discriminability.
Ranked #13 on
Self-Supervised Image Classification
on ImageNet
Contrastive Learning
Self-Supervised Image Classification
+3
Self-Promoted Supervision for Few-Shot Transformer
The few-shot learning ability of vision transformers (ViTs) is rarely investigated though heavily desired.
Robustness and Accuracy Could Be Reconcilable by (Proper) Definition
The trade-off between robustness and accuracy has been widely studied in the adversarial literature.
Modern Augmented Reality: Applications, Trends, and Future Directions
Augmented reality (AR) is one of the relatively old, yet trending areas in the intersection of computer vision and computer graphics with numerous applications in several areas, from gaming and entertainment, to education and healthcare.
Deep Color Consistent Network for Low-Light Image Enhancement
Current low-light image enhancement methods can well improve the illumination.
How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?
The fine-tuning of pre-trained language models has a great success in many NLP fields.
TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning
Analogously, VAT uses the similar feature augmentation encoder to refine the visual features, which are further applied in visual$\rightarrow$attribute decoder to learn visual-based attribute features.
DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition
While transformers have shown great potential on video recognition with their strong capability of capturing long-range dependencies, they often suffer high computational costs induced by the self-attention to the huge number of 3D tokens.
Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering
Meanwhile, for achieving higher rendering efficiency, we introduce a progressive rendering pipeline through geometry guidance, which leverages the geometric feature volume and the predicted density values to progressively reduce the number of sampling points and speed up the rendering process.
Towards Understanding Why Lookahead Generalizes Better Than SGD and Beyond
Specifically, we prove that lookahead using SGD as its inner-loop optimizer can better balance the optimization error and generalization error to achieve smaller excess risk error than vanilla SGD on (strongly) convex problems and nonconvex problems with Polyak-{\L}ojasiewicz condition which has been observed/proved in neural networks.
Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing
Based on this dataset, we then propose the Arbitrary Virtual Try-On Network (AVTON) that is utilized for all-type clothes, which can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person.
MetaFormer Is Actually What You Need for Vision
Based on this observation, we hypothesize that the general architecture of the Transformers, instead of the specific token mixer module, is more essential to the model's performance.
Ranked #9 on
Semantic Segmentation
on DensePASS
Direct Multi-view Multi-person 3D Pose Estimation
Instead of estimating 3D joint locations from costly volumetric representation or reconstructing the per-person 3D pose from multiple detected 2D poses as in previous methods, MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks.
Ranked #3 on
3D Multi-Person Pose Estimation
on Panoptic
(using extra training data)
Deep Long-Tailed Learning: A Survey
Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution.
PnP-DETR: Towards Efficient Visual Analysis with Transformers
Recently, DETR pioneered the solution of vision tasks with transformers, it directly translates the image feature map into the object detection result.
VOLO: Vision Outlooker for Visual Recognition
Though recently the prevailing vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification, their performance is still inferior to that of the latest SOTA CNNs if no extra data are provided.
Ranked #1 on
Image Classification
on VizWiz-Classification
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition
By realizing the importance of the positional information carried by 2D feature representations, unlike recent MLP-like models that encode the spatial information along the flattened spatial dimensions, Vision Permutator separately encodes the feature representations along the height and width dimensions with linear projections.
PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.
Human-centric Relation Segmentation: Dataset and Solution
For the above exemplar case, our HRS task produces results in the form of relation triplets <girl [left hand], hold, book> and exacts segmentation masks of the book, with which the robot can easily accomplish the grabbing task.
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
To overcome such limitations, we propose a new Tokens-To-Token Vision Transformer (T2T-ViT), which incorporates 1) a layer-wise Tokens-to-Token (T2T) transformation to progressively structurize the image to tokens by recursively aggregating neighboring Tokens into one Token (Tokens-to-Token), such that local structure represented by surrounding tokens can be modeled and tokens length can be reduced; 2) an efficient backbone with a deep-narrow structure for vision transformer motivated by CNN architecture design after empirical study.
Ranked #400 on
Image Classification
on ImageNet
ORDNet: Capturing Omni-Range Dependencies for Scene Parsing
Our ORDNet is able to extract more comprehensive context information and well adapt to complex spatial variance in scene images.
ProxylessKD: Direct Knowledge Distillation with Inherited Classifier for Face Recognition
We observe that existing knowledge distillation models optimize the proxy tasks that force the student to mimic the teacher's behavior, instead of directly optimizing the face recognition accuracy.
A Simple Baseline for Pose Tracking in Videos of Crowded Scenes
This paper presents our solution to ACM MM challenge: Large-scale Human-centric Video Analysis in Complex Events\cite{lin2020human}; specifically, here we focus on Track3: Crowd Pose Tracking in Complex Events.
Towards Accurate Human Pose Estimation in Videos of Crowded Scenes
In this paper, we focus on improving human pose estimation in videos of crowded scenes from the perspectives of exploiting temporal context and collecting new data.
Toward Accurate Person-level Action Recognition in Videos of Crowded Scenes
Prior works always fail to deal with this problem in two aspects: (1) lacking utilizing information of the scenes; (2) lacking training data in the crowd and complex scenes.
Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior
Nonnegative matrix factorization is usually powerful for learning the "shallow" parts-based representation, but it clearly fails to discover deep hierarchical information within both the basis and representation spaces.
Learning Target Domain Specific Classifier for Partial Domain Adaptation
Previous UDA methods assume that the source and target domains share an identical label space, which is unrealistic in practice since the label information of the target domain is agnostic.
Dual Adversarial Auto-Encoders for Clustering
By performing variational inference on the objective function of Dual-AAE, we derive a new reconstruction loss which can be optimized by training a pair of Auto-encoders.
ConvBERT: Improving BERT with Span-based Dynamic Convolution
The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning.
A Survey on Concept Factorization: From Shallow to Deep Representation Learning
In this paper, we therefore survey the recent advances on CF methodologies and the potential benchmarks by categorizing and summarizing the current methods.
Rethinking Bottleneck Structure for Efficient Mobile Network Design
In this paper, we rethink the necessity of such design changes and find it may bring risks of information loss and gradient confusion.
Recapture as You Want
LGR module utilizes body skeleton knowledge to construct a layout graph that connects all relevant part features, where graph reasoning mechanism is used to propagate information among part nodes to mine their relations.
Semantic Domain Adversarial Networks for Unsupervised Domain Adaptation
To successfully align the multi-modal data structures across domains, the following works exploit discriminative information in the adversarial training process, e. g., using multiple class-wise discriminators and introducing conditional information in input or output of the domain discriminator.
Highly Efficient Salient Object Detection with 100K Parameters
Salient object detection models often demand a considerable amount of computation cost to make precise prediction for each pixel, making them hardly applicable on low-power devices.
RC-DARTS: Resource Constrained Differentiable Architecture Search
In this paper, we propose the resource constrained differentiable architecture search (RC-DARTS) method to learn architectures that are significantly smaller and faster while achieving comparable accuracy.
Very Long Natural Scenery Image Prediction by Outpainting
The second challenge is how to maintain high quality in generated results, especially for multi-step generations in which generated regions are spatially far away from the initial input.
Learning Hybrid Representation by Robust Dictionary Learning in Factorized Compressed Space
Specifically, J-RFDL performs the robust representation by DL in a factorized compressed space to eliminate the negative effects of noise and outliers on the results, which can also make the DL process efficient.
Asymmetric GAN for Unpaired Image-to-image Translation
While in situations where two domains are asymmetric in complexity, i. e., the amount of information between two domains is different, these approaches pose problems of poor generation quality, mapping ambiguity, and model sensitivity.
Convolutional Dictionary Pair Learning Network for Image Representation Learning
To address this issue, we propose a novel generalized end-to-end representation learning architecture, dubbed Convolutional Dictionary Pair Learning Network (CDPL-Net) in this paper, which integrates the learning schemes of the CNN and dictionary pair learning into a unified framework.
DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking
However, in practice it is rather common to have no un-paired images in real deraining task, in such cases how to remove the rain streaks in an unsupervised way will be a very challenging task due to lack of constraints between images and hence suffering from low-quality recovery results.
Compressed DenseNet for Lightweight Character Recognition
LDB is a convolutional block similarly as dense block, but it can reduce the computation cost and weight size to (1/L, 2/L), compared with original ones, where L is the number of layers in blocks.
Multilayer Collaborative Low-Rank Coding Network for Robust Deep Subspace Discovery
In this paper, we explore the deep multi-subspace recovery problem by designing a multilayer architecture for latent LRR.
Efficient Differentiable Neural Architecture Search with Meta Kernels
We demonstrate the effectiveness of our search strategy by conducting extensive experiments.
AdversarialNAS: Adversarial Neural Architecture Search for GANs
In this paper, we propose an AdversarialNAS method specially tailored for Generative Adversarial Networks (GANs) to search for a superior generative model on the task of unconditional image generation.
Efficient Meta Learning via Minibatch Proximal Update
We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks.
Discriminative Local Sparse Representation by Robust Adaptive Dictionary Pair Learning
In this paper, we propose a structured Robust Adaptive Dic-tionary Pair Learning (RA-DPL) framework for the discrim-inative sparse representation learning.
PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image.
Flexible Auto-weighted Local-coordinate Concept Factorization: A Robust Framework for Unsupervised Clustering
For auto-weighting, RFA-LCF jointly preserves the manifold structures in the basis concept space and new coordinate space in an adaptive manner by minimizing the reconstruction errors on clean data, anchor points and coordinates.
Single-Stage Multi-Person Pose Machines
Based on SPR, we develop the SPM model that can directly predict structured poses for multiple persons in a single stage, and thus offer a more compact pipeline and attractive efficiency advantage over two-stage methods.
Ranked #3 on
Keypoint Detection
on MPII Multi-Person
Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning
We propose a joint subspace recovery and enhanced locality based robust flexible label consistent dictionary learning method called Robust Flexible Discriminative Dictionary Learning (RFDDL).
Kernel-Induced Label Propagation by Mapping for Semi-Supervised Classification
A Kernel-Induced Label Propagation (Kernel-LP) framework by mapping is proposed for high-dimensional data classification using the most informative patterns of data in kernel space.
Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier
Then we compute a linear classifier based on the approximated sparse codes by an analysis mechanism to simultaneously consider the classification and representation powers.
Joint Label Prediction based Semi-Supervised Adaptive Concept Factorization for Robust Data Representation
To enrich prior knowledge to enhance the discrimination, RS2ACF clearly uses class information of labeled data and more importantly propagates it to unlabeled data by jointly learning an explicit label indicator for unlabeled data.
Robust Unsupervised Flexible Auto-weighted Local-Coordinate Concept Factorization for Image Clustering
RFA-LCF integrates the robust flexible CF, robust sparse local-coordinate coding and the adaptive reconstruction weighting learning into a unified model.
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies.
Ranked #147 on
Action Classification
on Kinetics-400
Multi-Prototype Networks for Unconstrained Set-based Face Recognition
In this paper, we study the challenging unconstrained set-based face recognition problem where each subject face is instantiated by a set of media (images and videos) instead of a single image.
A^2-Nets: Double Attention Networks
Learning to capture long-range relations is fundamental to image/video recognition.
Graph-Based Global Reasoning Networks
In this work, we propose a new approach for reasoning globally in which a set of features are globally aggregated over the coordinate space and then projected to an interaction space where relational reasoning can be efficiently computed.
Style Separation and Synthesis via Generative Adversarial Networks
The S3-GAN consists of an encoder network, a generator network, and an adversarial network.
$A^2$-Nets: Double Attention Networks
Learning to capture long-range relations is fundamental to image/video recognition.
Ranked #35 on
Action Recognition
on UCF101
Look Across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition
Benchmarking our model on one of the most popular unconstrained face recognition datasets IJB-C additionally verifies the promising generalizability of AIM in recognizing faces in the wild.
Ranked #1 on
Age-Invariant Face Recognition
on MORPH Album2
Pose Partition Networks for Multi-Person Pose Estimation
This paper proposes a novel Pose Partition Network (PPN) to address the challenging multi-person pose estimation problem.
Mutual Learning to Adapt for Joint Human Parsing and Pose Estimation
This paper presents a novel Mutual Learning to Adapt model (MuLA) for joint human parsing and pose estimation.
Ranked #11 on
Semantic Segmentation
on LIP val
Dynamic Conditional Networks for Few-Shot Learning
This paper proposes a novel Dynamic Conditional Convolutional Network (DCCN) to handle conditional few-shot learning, i. e, only a few training samples are available for each condition.
Multi-Fiber Networks for Video Recognition
In this paper, we aim to reduce the computational cost of spatio-temporal deep neural networks, making them run as fast as their 2D counterparts while preserving state-of-the-art accuracy on video recognition benchmarks.
Ranked #36 on
Action Recognition
on UCF101
(using extra training data)
Exact Low Tubal Rank Tensor Recovery from Gaussian Measurements
Specifically, we show that by solving a TNN minimization problem, the underlying tensor of size $n_1\times n_2\times n_3$ with tubal rank $r$ can be exactly recovered when the given number of Gaussian measurements is $O(r(n_1+n_2-r)n_3)$.
Towards Pose Invariant Face Recognition in the Wild
To this end, we propose a Pose Invariant Model (PIM) for face recognition in the wild, with three distinct novelties.
Neural Style Transfer via Meta Networks
Recent works on style transfer typically need to train image transformation networks for every new style, and the style is encoded in the network parameters by enormous iterations of stochastic gradient descent, which lacks the generalization ability to new style in the inference stage.
Human Pose Estimation With Parsing Induced Learner
Comprehensive experiments on benchmarks LIP and extended PASCAL-Person-Part show that the proposed Parsing Induced Learner can improve performance of both single- and multi-person pose estimation to new state-of-the-art.
Subspace Clustering by Block Diagonal Representation
Second, we observe that many existing methods approximate the block diagonal representation matrix by using different structure priors, e. g., sparsity and low-rankness, which are indirect.
Tensor Robust Principal Component Analysis with A New Tensor Nuclear Norm
Equipped with the new tensor nuclear norm, we then solve the TRPCA problem by solving a convex program and provide the theoretical guarantee for the exact recovery.
Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing
Despite the noticeable progress in perceptual tasks like detection, instance segmentation and human parsing, computers still perform unsatisfactorily on visually understanding humans in crowded scenes, such as group behavior analysis, person re-identification and autonomous driving, etc.
Ranked #1 on
Multi-Human Parsing
on PASCAL-Part
Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation
We propose to detect scene text by localizing corner points of text bounding boxes and segmenting text regions in relative positions.
Ranked #2 on
Scene Text Detection
on ICDAR 2017 MLT
Face Aging with Contextual Generative Adversarial Nets
The age discriminative network guides the synthesized face to fit the real conditional distribution.
WSNet: Learning Compact and Efficient Networks with Weight Sampling
We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.
BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
Recently, deep neural networks (DNNs) have been regarded as the state-of-the-art classification methods in a wide range of applications, especially in image classification.
Weaving Multi-scale Context for Single Shot Detector
For 320x320 input of batch size = 8, WeaveNet reaches 79. 5% mAP on PASCAL VOC 2007 test in 101 fps with only 4 fps extra cost, and further improves to 79. 7% mAP with more iterations.
Nonconvex Sparse Spectral Clustering by Alternating Direction Method of Multipliers and Its Convergence Analysis
Experimental analysis on several real data sets verifies the effectiveness of our method.
Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis
In particular, we employ an off-the-shelf 3D face model as a simulator to generate profile face images with varying poses.
Ranked #1 on
Face Verification
on IJB-A
WSNet: Compact and Efficient Networks Through Weight Sampling
We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.
Personalized and Occupational-aware Age Progression by Generative Adversarial Networks
Second, we propose a new occupational-aware adversarial face aging network, which learns human aging process under different occupations.
HashGAN:Attention-aware Deep Adversarial Hashing for Cross Modal Retrieval
The proposed new adversarial network, HashGAN, consists of three building blocks: 1) the feature learning module to obtain feature representations, 2) the generative attention module to generate an attention mask, which is used to obtain the attended (foreground) and the unattended (background) feature representations, 3) the discriminative hash coding module to learn hash functions that preserve the similarities between different modalities.
Integrated Face Analytics Networks through Cross-Dataset Hybrid Training
Specifically, iFAN achieves an overall F-score of 91. 15% on the Helen dataset for face parsing, a normalized mean error of 5. 81% on the MTFL dataset for facial landmark localization and an accuracy of 45. 73% on the BNU dataset for emotion recognition with a single model.
Predicting Scene Parsing and Motion Dynamics in the Future
The ability of predicting the future is important for intelligent systems, e. g. autonomous vehicles and robots to plan early and make decisions accordingly.
Learning to Segment Human by Watching YouTube
An intuition on human segmentation is that when a human is moving in a video, the video-context (e. g., appearance and motion clues) may potentially infer reasonable mask information for the whole human body.
Scale-Adaptive Convolutions for Scene Parsing
Through adding a new scale regression layer, we can dynamically infer the position-adaptive scale coefficients which are adopted to resize the convolutional patches.
Recurrent 3D-2D Dual Learning for Large-Pose Facial Landmark Detection
To address these challenging issues, we introduce a novel recurrent 3D-2D dual learning model that alternatively performs 2D-based 3D face model refinement and 3D-to-2D projection based 2D landmark refinement to reliably reason about self-occluded landmarks, precisely capture the subtle landmark displacement and accurately detect landmarks even in presence of extremely large poses.
Deep Sparse Subspace Clustering
In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC).
Meta Networks for Neural Style Transfer
Recent works on style transfer typically need to train image transformation networks for every new style, and the style is encoded in the network parameters by enormous iterations of stochastic gradient descent.
Discriminative Similarity for Clustering and Semi-Supervised Learning
By generalization analysis via Rademacher complexity, the generalization error bound for the kernel classifier learned from hypothetical labeling is expressed as the sum of pairwise similarity between the data from different classes, parameterized by the weights of the kernel classifier.
Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback
However, most of the existing CNN models only learn features through a feedforward structure and no feedback information from top to bottom layers is exploited to enable the networks to refine themselves.
Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization
In this work, we prove that under certain suitable assumptions, we can recover both the low-rank and the sparse components exactly by simply solving a convex program whose objective is a weighted combination of the tensor nuclear norm and the $\ell_1$-norm, i. e., $\min_{{\mathcal{L}},\ {\mathcal{E}}} \ \|{{\mathcal{L}}}\|_*+\lambda\|{{\mathcal{E}}}\|_1, \ \text{s. t.}
FoveaNet: Perspective-aware Urban Scene Parsing
Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors.
Neural Person Search Machines
We investigate the problem of person search in the wild in this work.
Ranked #4 on
Person Re-Identification
on CUHK-SYSU
Dual Path Networks
In this work, we present a simple, highly efficient and modularized Dual Path Network (DPN) for image classification which presents a new topology of connection paths internally.
Semantic Segmentation via Structured Patch Prediction, Context CRF and Guidance CRF
The proposed joint model also employs a guidance CRF to further enhance the segmentation performance.
Memory-Augmented Attribute Manipulation Networks for Interactive Fashion Search
We introduce a new fashion search protocol where attribute manipulation is allowed within the interaction between users and search engines, e. g. manipulating the color attribute of the clothing from red to blue.
Perceptual Generative Adversarial Networks for Small Object Detection
In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to "super-resolved" ones, achieving similar characteristics as large objects and thus more discriminative for detection.
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
Personalized Age Progression with Bi-level Aging Dictionary Learning
Basically, for each age group, we learn an aging dictionary to reveal its aging characteristics (e. g., wrinkles), where the dictionary bases corresponding to the same index yet from two neighboring aging dictionaries form a particular aging pattern cross these two age groups, and a linear combination of all these patterns expresses a particular personalized aging process.
Generative Partition Networks for Multi-Person Pose Estimation
This paper proposes a new Generative Partition Network (GPN) to address the challenging multi-person pose estimation problem.
Ranked #1 on
Multi-Person Pose Estimation
on WAF
(AP metric)
Multiple-Human Parsing in the Wild
To address the multi-human parsing problem, we introduce a new multi-human parsing (MHP) dataset and a novel multi-human parsing model named MH-Parser.
Ranked #3 on
Multi-Human Parsing
on MHP v1.0
More is Less: A More Complicated Network with Less Inference Complexity
In this paper, we present a novel and general network structure towards accelerating the inference process of convolutional neural networks, which is more complicated in network structure yet with less inference complexity.
Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
We investigate a principle way to progressively mine discriminative object regions using classification networks to address the weakly-supervised semantic segmentation problems.
Interpretable Structure-Evolving LSTM
Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.
Tree-Structured Reinforcement Learning for Sequential Object Localization
Therefore, Tree-RL can better cover different objects with various scales which is quite appealing in the context of object proposal.
Training Group Orthogonal Neural Networks with Privileged Information
Learning rich and diverse representations is critical for the performance of deep convolutional neural networks (CNNs).
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
Robust LSTM-Autoencoders for Face De-Occlusion in the Wild
The first one, named multi-scale spatial LSTM encoder, reads facial patches of various scales sequentially to output a latent representation, and occlusion-robustness is achieved owing to the fact that the influence of occlusion is only upon some of the patches.
Video Scene Parsing with Predictive Feature Learning
In this way, the network can effectively learn to capture video dynamics and temporal context, which are critical clues for video scene parsing, without requiring extra manual annotations.
Deep Joint Rain Detection and Removal from a Single Image
Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output.
Multi-Path Feedback Recurrent Neural Network for Scene Parsing
In this paper, we consider the scene parsing problem and propose a novel Multi-Path Feedback recurrent neural network (MPF-RNN) for parsing scene images.
Visual Processing by a Unified Schatten-$p$ Norm and $\ell_q$ Norm Regularized Principal Component Pursuit
Typically, the specific structure is assumed to be low rank, which holds for a wide range of data, such as images and videos.
Multi-stage Object Detection with Group Recursive Learning
Most of existing detection pipelines treat object proposals independently and predict bounding box locations and classification scores over them separately.
Peak-Piloted Deep Network for Facial Expression Recognition
Objective functions for training of deep networks for face-related recognition tasks, such as facial expression recognition (FER), usually consider each sample independently.
Ranked #2 on
Facial Expression Recognition (FER)
on Oulu-CASIA
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
In this paper, we propose a layer-wise discriminative learning method to enhance the discriminative capability of a deep network by allowing its layers to work collaboratively for classification.
Training Skinny Deep Neural Networks with Iterative Hard Thresholding Methods
In this paper, we propose an iterative hard thresholding (IHT) approach to train Skinny Deep Neural Networks (SDNNs).
Diversified Visual Attention Networks for Fine-Grained Object Classification
Fine-grained object classification is a challenging task due to the subtle inter-class difference and large intra-class variation.
End-to-End Comparative Attention Networks for Person Re-identification
The CAN model is able to learn which parts of images are relevant for discerning persons and automatically integrates information from different parts to determine whether a pair of images belongs to the same person.
Recurrent Face Aging
Modeling the aging process of human face is important for cross-age face verification and recognition.
Recurrently Target-Attending Tracking
The produced confidence maps from the RNNs are employed to adaptively regularize the learning of discriminative correlation filters by suppressing clutter background noises while making full use of the information from reliable parts.
Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution
To address this essentially ill-posed problem, we introduce a Deep Edge Guided REcurrent rEsidual~(DEGREE) network to progressively recover the high-frequency details.
A Focused Dynamic Attention Model for Visual Question Answering
Solving VQA problems requires techniques from both computer vision for understanding the visual contents of a presented image or video, as well as the ones from natural language processing for understanding semantics of the question and generating the answers.
Attentive Contexts for Object Detection
We provide preliminary answers to these questions through developing a novel Attention to Context Convolution Neural Network (AC-CNN) based object detection model.
Semantic Object Parsing with Graph LSTM
By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
Instance-Aware Hashing for Multi-Label Image Retrieval
The instance-aware representations not only bring advantages to semantic hashing, but also can be used in category-aware hashing, in which an image is represented by multiple pieces of hash codes and each piece of code corresponds to a category.
Seq-NMS for Video Object Detection
Video object detection is challenging because objects that are easily detected in one frame may be difficult to detect in another frame within the same clip.
Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation
Learned from a large-scale training dataset, CNN features are much more discriminative and accurate than the hand-crafted features.
Scale-aware Pixel-wise Object Proposal Networks
In particular, in order to improve the localization accuracy, a fully convolutional network is employed which predicts locations of object proposals for each pixel.
Deep Learning with S-shaped Rectified Linear Activation Units
Rectified linear activation units are important components for state-of-the-art deep convolutional networks.
Online Crowdsourcing
With the success of modern internet based platform, such as Amazon Mechanical Turk, it is now normal to collect a large number of hand labeled samples from non-experts.
Conditional Convolutional Neural Network for Modality-Aware Face Recognition
Faces in the wild are usually captured with various poses, illuminations and occlusions, and thus inherently multimodally distributed in many tasks.
Task-Driven Feature Pooling for Image Classification
Feature pooling is an important strategy to achieve high performance in image classification.
Towards Computational Baby Learning: A Weakly-Supervised Approach for Object Detection
Then the concept detector can be fine-tuned based on these new instances.
Human Parsing With Contextualized Convolutional Neural Network
In this work, we address the human parsing task with a novel Contextualized Convolutional Neural Network (Co-CNN) architecture, which well integrates the cross-layer context, global image-level context, within-super-pixel context and cross-super-pixel neighborhood context into a unified network.
Additive Nearest Neighbor Feature Maps
In this paper, we present a concise framework to approximately construct feature maps for nonlinear additive kernels such as the Intersection, Hellinger's, and Chi^2 kernels.
Convex Sparse Spectral Clustering: Single-view to Multi-view
Spectral Clustering (SC) is one of the most widely used methods for data clustering.
Semantic Object Parsing with Local-Global Long Short-Term Memory
The long chains of sequential computation by stacked LG-LSTM layers also enable each pixel to sense a much larger region for inference benefiting from the memorization of previous dependencies in all positions along all dimensions.
