Search Results for author:
Found 229 papers, 110 papers with code
UVEB: A Large-scale Benchmark and Baseline Towards Real-World Underwater Video Enhancement
Learning-based underwater image enhancement (UIE) methods have made great progress.
Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
Large Vision-Language Models (LVLMs), due to the remarkable visual reasoning ability to understand images and videos, have received widespread attention in the autonomous driving domain, which significantly advances the development of interpretable end-to-end autonomous driving.
Adapting LLaMA Decoder to Vision Transformer
We first "LLaMAfy" a standard ViT step-by-step to align with LLaMA's architecture, and find that directly applying a casual mask to the self-attention brings an attention collapse issue, resulting in the failure to the network training.
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution.
Ranked #11 on
Visual Question Answering
on MM-Vet
Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks
Recently, the large language model (LLM) community has shown increasing interest in enhancing LLMs' capability to handle extremely long documents.
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
Scene graph generation (SGG) aims to parse a visual scene into an intermediate graph representation for downstream reasoning tasks.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text
Creating and animating 3D biped cartoon characters is crucial and valuable in various applications.
Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding
Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models.
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
Furthermore, image syntheses from DetDiffusion can effectively augment training data, significantly enhancing downstream detection performance.
Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models
Open-sourced Large Language Models (LLMs) have achieved great success in various NLP tasks, however, they are still far inferior to API-based models when acting as agents.
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
Multimodal large language models (MLLMs) have shown impressive reasoning abilities, which, however, are also more vulnerable to jailbreak attacks than their LLM predecessors.
DevBench: A Comprehensive Benchmark for Software Development
Recent advancements in large language models (LLMs) have significantly enhanced their coding capabilities.
Towards Fair and Efficient Learning-based Congestion Control
We present Astraea, a new learning-based congestion control that ensures fast convergence to fairness with stability.
Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction
We evaluate DRA across various open-source and close-source models, showcasing state-of-the-art jailbreak success rates and attack efficiency.
Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology
Our research provides a psychological explanation of the jailbreak prompts.
CriticBench: Evaluating Large Language Models as Critic
Critique ability are crucial in the scalable oversight and self-improvement of Large Language Models (LLMs).
How Susceptible are Large Language Models to Ideological Manipulation?
Large Language Models (LLMs) possess the potential to exert substantial influence on public perceptions and interactions with information.
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math.
Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts
It also obtains new state-of-the-art self-supervised learning results on detection and segmentation.
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model excelling in free-form text-image composition and comprehension.
Ranked #16 on
Visual Question Answering
on MM-Vet
MEA-Defender: A Robust Watermark against Model Extraction Attack
To protect the Intellectual Property (IP) of the original owners over such DNN models, backdoor-based watermarks have been extensively studied.
Can AI Assistants Know What They Don't Know?
To answer this question, we construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions, based on existing open-domain question answering datasets.
Guided Diffusion for Fast Inverse Design of Density-based Mechanical Metamaterials
Mechanical metamaterial is a synthetic material that can possess extraordinary physical characteristics, such as abnormal elasticity, stiffness, and stability, by carefully designing its internal structure.
Towards Language-Driven Video Inpainting via Multimodal Large Language Models
We introduce a new task -- language-driven video inpainting, which uses natural language instructions to guide the inpainting process.
RAP-SAM: Towards Real-Time All-Purpose Segment Anything
Segment Anything Model (SAM) is one remarkable model that can achieve generalized segmentation.
OMG-Seg: Is One Model Good Enough For All Segmentation?
In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models.
HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance
In this work, we present HuixiangDou, a technical assistant powered by Large Language Models (LLM).
STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering
However, most models can only handle simple videos in terms of temporal reasoning, and their performance tends to drop when answering temporal-reasoning questions on long and informative videos.
Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively
The CLIP and Segment Anything Model (SAM) are remarkable vision foundation models (VFMs).
Any-point Trajectory Modeling for Policy Learning
Learning from demonstration is a powerful method for teaching robots new skills, and having more demonstration data often improves policy learning.
LLM Factoscope: Uncovering LLMs' Factual Discernment through Inner States Analysis
In this paper, we introduce the LLM factoscope, a novel Siamese network-based model that leverages the inner states of LLMs for factual detection.
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
In the realm of computer vision and robotics, embodied agents are expected to explore their environment and carry out human instructions.
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models
Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles.
T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step
Based on that, we further introduce T-Eval to evaluate the tool utilization capability step by step.
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
Instruction tuning of Large Vision-language Models (LVLMs) has revolutionized the development of versatile models with zero-shot generalization across a wide range of downstream vision-language tasks.
DataElixir: Purifying Poisoned Dataset to Mitigate Backdoor Attacks via Diffusion Models
Dataset sanitization is a widely adopted proactive defense against poisoning-based backdoor attacks, aimed at filtering out and removing poisoned samples from training datasets.
RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation
Real-time multi-person pose estimation presents significant challenges in balancing speed and precision.
Ranked #1 on
Multi-Person Pose Estimation
on CrowdPose
(using extra training data)
Mixed Pseudo Labels for Semi-Supervised Object Detection
While the pseudo-label method has demonstrated considerable success in semi-supervised object detection tasks, this paper uncovers notable limitations within this approach.
Ranked #1 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting
This enables PowerPaint to accomplish various inpainting tasks by utilizing different task prompts, resulting in state-of-the-art performance.
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
Despite remarkable achievements in video synthesis, achieving granular control over complex dynamics, such as nuanced movement among multiple interacting objects, still presents a significant hurdle for dynamic world modeling, compounded by the necessity to manage appearance and disappearance, drastic scale changes, and ensure consistency for instances across frames.
Safer-Instruct: Aligning Language Models with Automated Preference Data
To verify the effectiveness of Safer-Instruct, we apply the pipeline to construct a safety preference dataset as a case study.
SAMIHS: Adaptation of Segment Anything Model for Intracranial Hemorrhage Segmentation
Segment Anything Model (SAM), a vision foundation model trained on large-scale annotations, has recently continued raising awareness within medical image segmentation.
BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues
In contrast, other LLMs struggle to generate multi-turn dialogues of satisfactory quality due to poor instruction-following capability, tendency to generate lengthy utterances, or limited general capability.
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis
The rapid development of large language models (LLMs) has not only provided numerous opportunities but also presented significant challenges.
Implicit Concept Removal of Diffusion Models
To address this, we utilize the intrinsic geometric characteristics of implicit concepts and present the Geom-Erasing, a novel concept removal method based on geometric-driven control.
Evaluating Hallucinations in Chinese Large Language Models
We analyze the primary types of hallucinations in different types of models and their causes.
MagicDrive: Street View Generation with Diverse 3D Geometry Control
Recent advancements in diffusion models have significantly enhanced the data synthesis with 2D control.
LawBench: Benchmarking Legal Knowledge of Large Language Models
We hope this benchmark provides in-depth understanding of the LLMs' domain-specified capabilities and speed up the development of LLMs in the legal domain.
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition.
Ranked #9 on
Visual Question Answering (VQA)
on InfiMM-Eval
Object2Scene: Putting Objects in Context for Open-Vocabulary 3D Detection
Instead of leveraging 2D images, we propose Object2Scene, the first approach that leverages large-scale large-vocabulary 3D object datasets to augment existing 3D scene datasets for open-vocabulary 3D object detection.
Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing
While enjoying the great achievements brought by deep learning (DL), people are also worried about the decision made by DL models, since the high degree of non-linearity of DL models makes the decision extremely difficult to understand.
Visual-Kinematics Graph Learning for Procedure-agnostic Instrument Tip Segmentation in Robotic Surgeries
However, this task is very challenging due to the small sizes of surgical instrument tips, and significant variance of surgical scenes across different procedures.
A Survey for Federated Learning Evaluations: Goals and Measures
Evaluation is a systematic approach to assessing how well a system achieves its intended purpose.
Self-Deception: Reverse Penetrating the Semantic Firewall of Large Language Models
Inspired by the attack that penetrates traditional firewalls through reverse tunnels, we introduce a "self-deception" attack that can bypass the semantic firewall by inducing LLM to generate prompts that facilitate jailbreak.
Deep Fusion Transformer Network with Weighted Vector-Wise Keypoints Voting for Robust 6D Object Pose Estimation
One critical challenge in 6D object pose estimation from a single RGBD image is efficient integration of two different modalities, i. e., color and depth.
Unfolding Once is Enough: A Deployment-Friendly Transformer Unit for Super-Resolution
Since the high resolution of intermediate features in SISR models increases memory and computational requirements, efficient SISR transformers are more favored.
Transferable Graph Structure Learning for Graph-based Traffic Forecasting Across Cities
To address the problem, we propose TransGTR, a transferable structure learning framework for traffic forecasting that jointly learns and transfers the graph structures and forecasting models across cities.
Learning Referring Video Object Segmentation from Weak Annotation
Based on this scheme, we develop a novel RVOS method that exploits weak annotations effectively.
Relational Contrastive Learning for Scene Text Recognition
We argue that such prior contextual information can be interpreted as the relations of textual primitives due to the heterogeneous text and background, which can provide effective self-supervised labels for representation learning.
Improving Pixel-based MIM by Reducing Wasted Modeling Capability
There has been significant progress in Masked Image Modeling (MIM).
Value-Informed Skill Chaining for Policy Learning of Long-Horizon Tasks with Surgical Robot
Based on this value function, a chaining policy is learned to instruct subtask policies to terminate at the state with the highest value so that all subsequent policies are more likely to be connected for accomplishing the task.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
ConFL: Constraint-guided Fuzzing for Machine Learning Framework
Guided by the constraints, ConFL is able to generate valid inputs that can pass the verification and explore deeper paths of kernel codes.
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Before sending to LLM, the reference is replaced by RoI features and interleaved with language embeddings as a sequence.
Ranked #1 on
Visual Question Answering (VQA)
on VCR (Q-AR) test
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Recent advancements in vision foundation models (VFMs) have opened up new possibilities for versatile and efficient visual perception.
GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation
Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification.
Improving Handwritten OCR with Training Samples Generated by Glyph Conditional Denoising Diffusion Probabilistic Model
This model conditions on a printed glyph image and creates mappings between printed characters and handwritten images, thus enabling the generation of photo-realistic handwritten samples with diverse styles and unseen text contents.
GlyphControl: Glyph Conditional Control for Visual Text Generation
We evaluate the effectiveness of our approach by measuring OCR-based metrics, CLIP score, and FID of the generated visual text.
A Meta-learning Framework for Tuning Parameters of Protection Mechanisms in Trustworthy Federated Learning
Motivated by this requirement, we propose a framework that (1) formulates TFL as a problem of finding a protection mechanism to optimize the tradeoff between privacy leakage, utility loss, and efficiency reduction and (2) formally defines bounded measurements of the three factors.
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank
In this paper, we propose a novel approach, RankCSE, for unsupervised sentence representation learning, which incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework.
Zero-shot Generation of Training Data with Denoising Diffusion Probabilistic Model for Handwritten Chinese Character Recognition
Training from handwritten samples of a small character set, the DDPM is capable of mapping printed strokes to handwritten ones, which makes it possible to generate photo-realistic and diverse style handwritten samples of unseen character categories.
Theoretically Principled Federated Learning for Balancing Privacy and Utility
We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters, which facilitates the trade-off between privacy and utility.
TG-VQA: Ternary Game of Video Question Answering
Video question answering aims at answering a question about the video content by reasoning the alignment semantics within them.
MultiModal-GPT: A Vision and Language Model for Dialogue with Humans
To further enhance the ability to chat with humans of the MultiModal-GPT, we utilize language-only instruction-following data to train the MultiModal-GPT jointly.
Towards Achieving Near-optimal Utility for Privacy-Preserving Federated Learning via Data Generation and Parameter Distortion
The nature of the widely-adopted protection mechanisms including \textit{Randomization Mechanism} and \textit{Compression Mechanism} is to protect privacy via distorting model parameter.
Optimizing Privacy, Utility and Efficiency in Constrained Multi-Objective Federated Learning
Different from existing CMOFL works focusing on utility, efficiency, fairness, and robustness, we consider optimizing privacy leakage along with utility loss and training cost, the three primary objectives of a TFL system.
Transformer-Based Visual Segmentation: A Survey
Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks.
RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
Our experiments further demonstrate that pre-training and depth-free BEV transformation has the potential to enhance out-of-distribution robustness.
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
A Game-theoretic Framework for Privacy-preserving Federated Learning
To address this, we propose the first game-theoretic framework that considers both FL defenders and attackers in terms of their respective payoffs, which include computational costs, FL model utilities, and privacy leakage risks.
Probably Approximately Correct Federated Learning
However, existing multi-objective optimization frameworks are very time-consuming, and do not guarantee the existence of the Pareto frontier, this motivates us to seek a solution to transform the multi-objective problem into a single-objective problem because it is more efficient and easier to be solved.
A Survey on Vertical Federated Learning: From a Layered Perspective
From the hardware layer to the vertical federated system layer, researchers contribute to various aspects of VFL.
Mixed Autoencoder for Self-supervised Visual Representation Learning
Specifically, our MixedAE outperforms MAE by +0. 3% accuracy, +1. 7 mIoU and +0. 9 AP on ImageNet-1K, ADE20K and COCO respectively with a standard ViT-Base.
Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications.
Federated Learning without Full Labels: A Survey
Therefore, the problem of federated learning without full labels is important in real-world FL applications.
UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework
To verify our algorithm, we conducted experiments on the V2X-Sim and OPV2V datasets.
Dense Distinct Query for End-to-End Object Detection
Concretely, we first lay dense queries like traditional detectors and then select distinct ones for one-to-one assignments.
Ranked #3 on
Object Detection
on CrowdHuman (full body)
RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose
Recent studies on 2D pose estimation have achieved excellent performance on public benchmarks, yet its application in the industrial community still suffers from heavy model parameters and high latency.
Ranked #3 on
Pose Estimation
on OCHuman
(using extra training data)
Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation
Referring image segmentation segments an image from a language expression.
PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling
Masked Image Modeling (MIM) has achieved promising progress with the advent of Masked Autoencoders (MAE) and BEiT.
Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot
Task automation of surgical robot has the potentials to improve surgical efficiency.
Boosting Neural Networks to Decompile Optimized Binaries
They formulate the decompilation process as a translation problem between LPL and HPL, aiming to reduce the human cost required to develop decompilation tools and improve their generalizability.
Learning Shape Primitives via Implicit Convexity Regularization
In this work, a novel regularization term named Implicit Convexity Regularization (ICR) imposed on implicit primitive learning is proposed to tackle this problem.
RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer
Extensive experiments and ablative analysis also demonstrate that the inductive bias of network architecture, can be incorporated into simple network structure with appropriate optimization strategy.
RTMDet: An Empirical Study of Designing Real-Time Object Detectors
In this paper, we aim to design an efficient real-time object detector that exceeds the YOLO series and is easily extensible for many object recognition tasks such as instance segmentation and rotated object detection.
Ranked #1 on
Oriented Object Detection
on DOTA 1.5
Anger Breeds Controversy: Analyzing Controversy and Emotions on Reddit
We collect discussions from a wide variety of topical forums and use emotion detection to recognize a range of emotions from text, including anger, fear, joy, admiration, etc.
StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS
To further improve the performance of the stereo framework, StereoPose is equipped with a parallax attention module for stereo feature fusion and an epipolar loss for improving the stereo-view consistency of network predictions.
Boosting Point Clouds Rendering via Radiance Mapping
In this work, we focus on boosting the image quality of point clouds rendering with a compact model design.
A Novel Membership Inference Attack against Dynamic Neural Networks by Utilizing Policy Networks Information
Although traditional static DNNs are vulnerable to the membership inference attack (MIA) , which aims to infer whether a particular point was used to train the model, little is known about how such an attack performs on the dynamic NNs.
DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition
Graph convolution networks (GCN) have been widely used in skeleton-based action recognition.
Ranked #7 on
Skeleton Based Action Recognition
on NTU RGB+D
Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks
Deep learning models have achieved excellent recognition results on large-scale video benchmarks.
Deliberated Domain Bridging for Domain Adaptive Semantic Segmentation
In this work, we resort to data mixing to establish a deliberated domain bridging (DDB) for DASS, through which the joint distributions of source and target domains are aligned and interacted with each in the intermediate space.
Ranked #1 on
Domain Adaptation
on GTAV+Synscapes to Cityscapes
SSL-WM: A Black-Box Watermarking Approach for Encoders Pre-trained by Self-supervised Learning
Recent years have witnessed tremendous success in Self-Supervised Learning (SSL), which has been widely utilized to facilitate various downstream tasks in Computer Vision (CV) and Natural Language Processing (NLP) domains.
Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detection
In this study, we dive deep into the inconsistency of pseudo targets in semi-supervised object detection (SSOD).
Trading Off Privacy, Utility and Efficiency in Federated Learning
In addition, it is a mandate for a federated learning system to achieve high \textit{efficiency} in order to enable large-scale model training and deployment.
Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain
However, most of the triggers used in the current study are fixed patterns patched on a small fraction of an image and are often clearly mislabeled, which is easily detected by humans or defense methods such as Neural Cleanse and SentiNet.
Semi-blind source separation using convolutive transfer function for nonlinear acoustic echo cancellation
The recently proposed semi-blind source separation (SBSS) method for nonlinear acoustic echo cancellation (NAEC) outperforms adaptive NAEC in attenuating the nonlinear acoustic echo.
Secure Forward Aggregation for Vertical Federated Neural Networks
In this paper, we focus on SplitNN, a well-known neural network framework in VFL, and identify a trade-off between data security and model performance in SplitNN.
A Novel Multi-Agent Scheduling Mechanism for Adaptation of Production Plans in Case of Supply Chain Disruptions
The system has been implemented as proof of concept and is currently reimplemented and transferred to a production system based on the Jadex agent platform.
What Are Expected Queries in End-to-End Object Detection?
As both sparse and dense queries are imperfect, then \emph{what are expected queries in end-to-end object detection}?
Task-Customized Self-Supervised Pre-training with Scalable Dynamic Routing
On the other hand, for existing SSL methods, it is burdensome and infeasible to use different downstream-task-customized datasets in pre-training for different tasks.
PYSKL: Towards Good Practices for Skeleton Action Recognition
The toolbox supports a wide variety of skeleton action recognition algorithms, including approaches based on GCN and CNN.
Ranked #19 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Group R-CNN for Weakly Semi-supervised Object Detection with Points
The core of this task is to train a point-to-box regressor on well-labeled images that can be used to predict credible bounding boxes for each point annotation.
TransRank: Self-supervised Video Representation Learning via Ranking-based Transformation Recognition
To mitigate this problem, we developed TransRank, a unified framework for recognizing Transformations in a Ranking formulation.
MMRotate: A Rotated Object Detection Benchmark using PyTorch
We present an open-source toolbox, named MMRotate, which provides a coherent algorithm framework of training, inferring, and evaluation for the popular rotated object detection algorithm based on deep learning.
ROMA: Cross-Domain Region Similarity Matching for Unpaired Nighttime Infrared to Daytime Visible Video Translation
To be specific, ROMA could efficiently translate the unpaired nighttime infrared videos into fine-grained daytime visible ones, meanwhile maintain the spatiotemporal consistency via matching the cross-domain region similarity.
NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results
This challenge includes three tracks.
Sim-to-Real 6D Object Pose Estimation via Iterative Self-training for Robotic Bin Picking
To continuously improve the quality of pseudo labels, we iterate the above steps by taking the trained student model as a new teacher and re-label real data using the refined teacher model.
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
We hope this simple, yet effective method can serve as a new, flexible baseline in unified video segmentation design.
Ranked #1 on
Video Panoptic Segmentation
on KITTI-STEP
(using extra training data)
StructToken : Rethinking Semantic Segmentation with Structural Prior
In previous deep-learning-based methods, semantic segmentation has been regarded as a static or dynamic per-pixel classification task, \textit{i. e.,} classify each pixel representation to a specific category.
Dense Siamese Network for Dense Unsupervised Learning
It also extracts a batch of region embeddings that correspond to some sub-regions in the overlapped area to be contrasted for region consistency.
Ranked #2 on
Unsupervised Semantic Segmentation
on COCO-All
(mIoU metric)
APRNet: Attention-based Pixel-wise Rendering Network for Photo-Realistic Text Image Generation
Style-guided text image generation tries to synthesize text image by imitating reference image's appearance while keeping text content unaltered.
CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving
One main reason that impedes the development of truly reliably self-driving systems is the lack of public datasets for evaluating the performance of object detectors on corner cases.
RotateQVS: Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion
Temporal factors are tied to the growth of facts in realistic applications, such as the progress of diseases and the development of political situation, therefore, research on Temporal Knowledge Graph (TKG) attracks much attention.
Ranked #3 on
Link Prediction
on GDELT
GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors
The style modulation aims to generate realistic face details and the feature modulation dynamically fuses the multi-level encoded features and the generated ones conditioned on the upscaling factor.
No Free Lunch Theorem for Security and Utility in Federated Learning
In a federated learning scenario where multiple parties jointly learn a model from their respective data, there exist two conflicting goals for the choice of appropriate algorithms.
Towards Robust Part-aware Instance Segmentation for Industrial Bin Picking
Industrial bin picking is a challenging task that requires accurate and robust segmentation of individual object instances.
PPA: Preference Profiling Attack Against Federated Learning
By observing a user model's gradient sensitivity to a class, PPA can profile the sample proportion of the class in the user's local dataset, and thus the user's preference of the class is exposed.
Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing
To explore this, we first introduce the notion of training plasticity to quantify the training progress of internal DNN layers.
Efficient Data-Plane Memory Scheduling for In-Network Aggregation
However, switch memory is scarce compared to the volume of gradients transmitted in distributed training.
OCSampler: Compressing Videos to One Clip with Single-step Sampling
Recent works prefer to formulate frame sampling as a sequential decision task by selecting frames one by one according to their importance, while we present a new paradigm of learning instance-specific video condensation policies to select informative frames for representing the entire video only in a single step.
TAGPerson: A Target-Aware Generation Pipeline for Person Re-identification
In TAGPerson, we extract information from target scenes and use them to control our parameterized rendering process to generate target-aware synthetic images, which would hold a smaller gap to the real images in the target domain.
Automatic Configuration for Optimal Communication Scheduling in DNN Training
ByteScheduler partitions and rearranges tensor transmissions to improve the communication efficiency of distributed Deep Neural Network (DNN) training.
AI-Lancet: Locating Error-inducing Neurons to Optimize Neural Networks
With the knowledge of error-inducing neurons, we propose two methods to fix the errors: the neuron-flip and the neuron-fine-tuning.
LAVT: Language-Aware Vision Transformer for Referring Image Segmentation
Referring image segmentation is a fundamental vision-language task that aims to segment out an object referred to by a natural language expression from an image.
Generalized Referring Expression Segmentation
Image Segmentation
+2
Few-Shot Object Detection via Association and DIscrimination
1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space.
DBIA: Data-free Backdoor Injection Attack against Transformer Networks
In this paper, we propose DBIA, a novel data-free backdoor attack against the CV-oriented transformer networks, leveraging the inherent attention mechanism of transformers to generate triggers and injecting the backdoor using the poisoned surrogate dataset.
Attacking Video Recognition Models with Bullet-Screen Comments
On both UCF-101 and HMDB-51 datasets, our BSC attack method can achieve about 90\% fooling rate when attacking three mainstream video recognition models, while only occluding \textless 8\% areas in the video.
Adversarial Attack
Adversarial Attack on Video Classification
+2
The Nuts and Bolts of Adopting Transformer in GANs
Transformer becomes prevalent in computer vision, especially for high-level vision tasks.
Quantitative relations among causality measures with applications to nonlinear pulse-output network reconstruction
However, the interpretation of causal connectivity remains to be fully clarified, in particular, how causal connectivity depends on causality measures and how causal connectivity relates to structural connectivity.
Foreground-attention in neural decoding: Guiding Loop-Enc-Dec to reconstruct visual stimulus images from fMRI
Inspired by the mechanism of human visual attention, in this paper, we propose a novel method of reconstructing visual stimulus images, which first decodes the distribution of visual attention from fMRI, and then reconstructs the visual images guided by visual attention.
Temporal RoI Align for Video Object Recognition
In this work, considering the features of the same object instance are highly similar among frames in a video, a novel Temporal RoI Align operator is proposed to extract features from other frames feature maps for current frame proposals by utilizing feature similarity.
Ranked #1 on
Video Instance Segmentation
on YouTube-VIS
MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Driving
By pre-training on SODA10M, a large-scale autonomous driving dataset, MultiSiam exceeds the ImageNet pre-trained MoCo-v2, demonstrating the potential of domain-specific pre-training.
Towards Balanced Learning for Instance Recognition
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks
Furthermore, we design a recurrent reconstruction network for iterative residual refinement to progressively improve the reconstruction and correspondence estimations from coarse to fine.
Practical and Secure Federated Recommendation with Personalized Masks
In this paper, we proposed federated masked matrix factorization (FedMMF) to protect the data privacy in federated recommender systems without sacrificing efficiency and effectiveness.
Aegis: A Trusted, Automatic and Accurate Verification Framework for Vertical Federated Learning
In this paper, we present Aegis, a trusted, automatic, and accurate verification framework to verify the security of VFL jobs.
MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding
We present MMOCR-an open-source toolbox which provides a comprehensive pipeline for text detection and recognition, as well as their downstream tasks such as named entity recognition and key information extraction.
I2V-GAN: Unpaired Infrared-to-Visible Video Translation
Human vision is often adversely affected by complex environmental factors, especially in night vision scenarios.
HAFLO: GPU-Based Acceleration for Federated Logistic Regression
In recent years, federated learning (FL) has been widely applied for supporting decentralized collaborative learning scenarios.
K-Net: Towards Unified Image Segmentation
The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class.
Ranked #7 on
Panoptic Segmentation
on COCO test-dev
SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Driving
Experiments show that SODA10M can serve as a promising pre-training dataset for different self-supervised learning methods, which gives superior performance when fine-tuning with different downstream tasks (i. e., detection, semantic/instance segmentation) in autonomous driving domain.
Learning To Identify Correct 2D-2D Line Correspondences on Sphere
In contrast, we propose the first approach suitable for both structured and unstructured scenes.
WSSOD: A New Pipeline for Weakly- and Semi-Supervised Object Detection
The performance of object detection, to a great extent, depends on the availability of large annotated datasets.
DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
Compared to the method of retraining from scratch, our approach can achieve 99. 0%, 95. 0%, 91. 9%, 96. 7%, 74. 1% accuracy rates and 66. 7$\times$, 75. 0$\times$, 33. 3$\times$, 29. 4$\times$, 13. 7$\times$ speedups on the MNIST, SVHN, CIFAR-10, Purchase, and ImageNet datasets, respectively.
Revisiting Skeleton-based Action Recognition
In this work, we propose PoseC3D, a new approach to skeleton-based action recognition, which relies on a 3D heatmap stack instead of a graph sequence as the base representation of human skeletons.
Ranked #1 on
Action Recognition
on NTU RGB+D
HufuNet: Embedding the Left Piece as Watermark and Keeping the Right Piece for Ownership Verification in Deep Neural Networks
Most existing solutions embed backdoors in DNN model training such that DNN ownership can be verified by triggering distinguishable model behaviors with a set of secret inputs.
SoK: A Modularized Approach to Study the Security of Automatic Speech Recognition Systems
In this paper, we present our systematization of knowledge for ASR security and provide a comprehensive taxonomy for existing work based on a modularized workflow.
Recent Advances in Data-Driven Wireless Communication Using Gaussian Processes: A Comprehensive Survey
Empowered by big data and machine learning, next-generation data-driven communication systems will be intelligent with the characteristics of expressiveness, scalability, interpretability, and especially uncertainty modeling, which can confidently involve diversified latent demands and personalized services in the foreseeable future.
ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction
In this competition, we set up three tasks, namely, Scanned Receipt Text Localisation (Task 1), Scanned Receipt OCR (Task 2) and Key Information Extraction from Scanned Receipts (Task 3).
Key Information Extraction
Optical Character Recognition (OCR)
+1
NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting
Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge.
Fast and Reliable Probabilistic Face Embeddings in the Wild
In addition, an identification preserving loss is proposed to improve the discriminative of the MLS metric, and a multi-layer feature fusion module is proposed to improve the neural network's uncertainty estimation ability.
Learning Icosahedral Spherical Probability Map Based on Bingham Mixture Model for Vanishing Point Estimation
Based on this map, we can detect all the VPs.
SGPA: Structure-Guided Prior Adaptation for Category-Level 6D Object Pose Estimation
The prior adaptation intrinsically associates the adopted prior with different objects, from which we can accurately reconstruct the 3D canonical model of the specific object for pose estimation.
UMLE: Unsupervised Multi-discriminator Network for Low Light Enhancement
To address this problem, we propose a real-time unsupervised generative adversarial network (GAN) containing multiple discriminators, i. e. a multi-scale discriminator, a texture discriminator, and a color discriminator.
A Hierarchical Reasoning Graph Neural Network for The Automatic Scoring of Answer Transcriptions in Video Job Interviews
In this work, we propose a Hierarchical Reasoning Graph Neural Network (HRGNN) for the automatic assessment of question-answer pairs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Positional Encoding as Spatial Inductive Bias in GANs
In this work, taking SinGAN and StyleGAN2 as examples, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
RRCN: A Reinforced Random Convolutional Network based Reciprocal Recommendation Approach for Online Dating
In particular, we technically propose a novel random CNN component that can randomly convolute non-adjacent features to capture their interaction information and learn feature embeddings of key attributes to make the final recommendation.
Neural-iLQR: A Learning-Aided Shooting Method for Trajectory Optimization
Iterative linear quadratic regulator (iLQR) has gained wide popularity in addressing trajectory optimization problems with nonlinear system models.
FedEval: A Holistic Evaluation Framework for Federated Learning
In this paper, we propose a holistic evaluation framework for FL called FedEval, and present a benchmarking study on seven state-of-the-art FL algorithms.
Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for Long-term Forecasting
The heavy tail and skewness characteristics of such distributions in the spectral domain allow to capture long-range covariance of the signal in the time domain.
Possible multi-orbital ground state in CeCu$_2$Si$_2$
The crystal-field ground state wave function of CeCu$_2$Si$_2$ has been investigated with linear polarized $M$-edge x-ray absorption spectroscopy from 250mK to 250K, thus covering the superconducting ($T_{\text{c}}$=0. 6K), the Kondo ($T_{\text{K}}$$\approx$20K) as well as the Curie-Weiss regime.
Strongly Correlated Electrons
Exploring the Generalizability of Spatio-Temporal Traffic Prediction: Meta-Modeling and an Analytic Framework
The Spatio-Temporal Traffic Prediction (STTP) problem is a classical problem with plenty of prior research efforts that benefit from traditional statistical learning and recent deep learning approaches.
Seesaw Loss for Long-Tailed Instance Segmentation
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
Domain-specific Communication Optimization for Distributed DNN Training
Communication overhead poses an important obstacle to distributed DNN training and draws increasing attention in recent years.
FPGA-Based Hardware Accelerator of Homomorphic Encryption for Efficient Federated Learning
Our framework implements the representative Paillier homomorphic cryptosystem with high level synthesis for flexibility and portability, with careful optimization on the modular multiplication operation in terms of processing clock cycle, resource usage and clock frequency.
DS-Sync: Addressing Network Bottlenecks with Divide-and-Shuffle Synchronization for Distributed DNN Training
However, due to the global synchronization nature, its performance can be significantly influenced by network bottlenecks caused by either static topology heterogeneity or dynamic bandwidth contentions.
Target Speech Extraction Based on Blind Source Separation and X-vector-based Speaker Selection Trained with Data Augmentation
Extracting the desired speech from a mixture is a meaningful and challenging task.
Nonlinear Residual Echo Suppression Based on Multi-stream Conv-TasNet
Acoustic echo cannot be entirely removed by linear adaptive filters due to the nonlinear relationship between the echo and far-end signal.
Neural Data-to-Text Generation with Dynamic Content Planning
We further design a reconstruction mechanism with a novel objective function that can reconstruct the whole entry of the used data sequentially from the hidden states of the decoder, which aids the accuracy of the generated text.
Feature Pyramid Grids
Feature pyramid networks have been widely adopted in the object detection literature to improve feature representations for better handling of variations in scale.
Vanishing Point Guided Natural Image Stitching
Recently, works on improving the naturalness of stitching images gain more and more extensive attention.
Quantifying the Performance of Federated Transfer Learning
The scarcity of data and isolated data islands encourage different organizations to share data with each other to train machine learning models.
Unified Approach to Witness Nonentanglement-Breaking Quantum Channels
The ability of quantum devices to preserve or distribute entanglement is essential in employing quantum technologies.
Quantum Physics
Side-Aware Boundary Localization for More Precise Object Detection
To tackle the difficulty of precise localization in the presence of displacements with large variance, we further propose a two-step localization scheme, which first predicts a range of movement through bucket prediction and then pinpoints the precise position within the predicted bucket.
Towards Security Threats of Deep Learning Systems: A Survey
In order to unveil the security weaknesses and aid in the development of a robust deep learning system, we undertake an investigation on attacks towards deep learning, and analyze these attacks to conclude some findings in multiple views.
An End-to-End Audio Classification System based on Raw Waveforms and Mix-Training Strategy
This paper introduces an end-to-end audio classification system based on raw waveforms and mix-training strategy.
Gliding vertex on the horizontal bounding box for multi-oriented object detection
Yet, the widely adopted horizontal bounding box representation is not appropriate for ubiquitous oriented objects such as objects in aerial images and scene texts.
Ranked #42 on
Object Detection In Aerial Images
on DOTA
(using extra training data)
Real-time Scene Text Detection with Differentiable Binarization
Recently, segmentation-based methods are quite popular in scene text detection, as the segmentation results can more accurately describe scene text of various shapes such as curve text.
Ranked #6 on
Scene Text Detection
on MSRA-TD500
Interpretable Encrypted Searchable Neural Networks
In cloud security, traditional searchable encryption (SE) requires high computation and communication overhead for dynamic search and update.
Multi-owner Secure Encrypted Search Using Searching Adversarial Networks
To address this, this paper proposes secure and efficient multi-keyword ranked search over encrypted cloud data for multi-owner model based on searching adversarial networks.
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
Secure Federated Matrix Factorization
The key principle of federated learning is training a machine learning model without needing to know each user's personal raw private data.
Extracting Symptoms and their Status from Clinical Conversations
This paper describes novel models tailored for a new application, that of extracting the symptoms mentioned in clinical conversations along with their status.
CARAFE: Content-Aware ReAssembly of FEatures
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Ranked #3 on
Feature Upsampling
on ImageNet
Prime Sample Attention in Object Detection
Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector.
Libra R-CNN: Towards Balanced Learning for Object Detection
In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels - sample level, feature level, and objective level.
Ranked #149 on
Object Detection
on COCO test-dev
Hybrid Task Cascade for Instance Segmentation
In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.
Ranked #32 on
Object Detection
on COCO-O
