Search Results for author:
Found 179 papers, 135 papers with code
Fixing Localization Errors to Improve Image Classification
This way, in an effort to fix localization errors, our loss provides an extra supervisory signal that helps the model to better discriminate between similar classes.
Count- and Similarity-aware R-CNN for Pedestrian Detection
We further introduce a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity.
Cross-Modal Self-Training: Aligning Images and Pointclouds to Learn Classification without Labels
Thereby we demonstrate that 2D vision language models such as CLIP can be used to complement 3D representation learning to improve classification performance without the need for expensive class annotations.
Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning
ZSLViT mainly considers two properties in the whole network: i) discover the semantic-related visual representations explicitly, and ii) discard the semantic-unrelated visual information.
Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration
All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation.
Language Guided Domain Generalized Medical Image Segmentation
Incorporating text features alongside visual features is a potential solution to enhance the model's understanding of the data, as it goes beyond pixel-level information to provide valuable context.
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation.
ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection
Deep learning has shown remarkable success in remote sensing change detection (CD), aiming to identify semantic change regions between co-registered satellite image pairs acquired at distinct time stamps.
Composed Video Retrieval via Enriched Context and Discriminative Embeddings
Composed video retrieval (CoVR) is a challenging problem in computer vision which has recently highlighted the integration of modification text with visual queries for more sophisticated video search in large databases.
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding
In contrast, this paper introduces a Video Understanding and Reasoning Framework (VURF) based on the reasoning power of LLMs.
AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation
Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task.
Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn better representations.
Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery
Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks by pre-training on large amount of unlabelled data.
Effectiveness Assessment of Recent Large Vision-Language Models
The advent of large vision-language models (LVLMs) represents a noteworthy advancement towards the pursuit of artificial general intelligence.
ObjectCompose: Evaluating Resilience of Vision-Based Models on Object-to-Background Compositional Changes
We produce various versions of standard vision datasets (ImageNet, COCO), incorporating either diverse and realistic backgrounds into the images or introducing color, texture, and adversarial changes in the background.
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT
"Bigger the better" has been the predominant trend in recent Large Language Models (LLMs) development.
Semi-supervised Open-World Object Detection
We demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations.
BiMediX: Bilingual Medical Mixture of Experts LLM
In this paper, we introduce BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic.
Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Models
However, these models struggle to effectively suppress the generation of undesired content, which is explicitly requested to be omitted from the generated image in the prompt.
Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding
Our contributions include a novel spatio-temporal video grounding model, surpassing state-of-the-art results in closed-set evaluations on multiple datasets and demonstrating superior performance in open-vocabulary scenarios.
Faster Diffusion: Rethinking the Role of UNet Encoder in Diffusion Models
This finding inspired us to omit the encoder at certain adjacent time-steps and reuse cyclically the encoder features in the previous time-steps for the decoder.
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM
To this end, we propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation
In this paper, we propose a simple encoder-decoder, named SED, for open-vocabulary semantic segmentation, which comprises a hierarchical encoder-based cost map generation and a gradual fusion decoder with category early rejection.
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Furthermore, the lack of domain-specific multimodal instruction following data as well as strong backbone models for RS make it hard for the models to align their behavior with user queries.
Enhancing Novel Object Detection via Cooperative Foundational Models
We present a novel approach to transform existing closed-set detectors into open-set detectors.
Ranked #1 on
Novel Object Detection
on LVIS v1.0 val
Cal-DETR: Calibrated Detection Transformer
Third, we develop a logit mixing approach that acts as a regularizer with detection-specific losses and is also complementary to the uncertainty-guided logit modulation technique to further improve the calibration performance.
Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization
The promising zero-shot generalization of vision-language models such as CLIP has led to their adoption using prompt learning for numerous downstream tasks.
Videoprompter: an ensemble of foundational models for zero-shot video understanding
Consistent improvements across multiple benchmarks and with various VLMs demonstrate the effectiveness of our proposed framework.
Ranked #2 on
Video-Text Retrieval
on Test-of-Time
Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation
We decompose the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance, respectively.
Unsupervised Landmark Discovery Using Consistency Guided Bottleneck
In the current work, we introduce a consistency-guided bottleneck in an image reconstruction-based pipeline that leverages landmark consistency, a measure of compatibility score with the pseudo-ground truth to generate adaptive heatmaps.
A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos
Existing video-based breast lesion detection approaches typically perform temporal feature aggregation of deep backbone features based on the self-attention operation.
Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement
The method has resulted in a significant performance improvement, of up to 5. 0% AUC, of state-of-the-art (SOTA) visual trackers.
How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges
Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational AI.
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
Frequency Domain Adversarial Training for Robust Volumetric Medical Segmentation
While recent advances in deep learning have improved the performance of volumetric medical image segmentation models, these models cannot be deployed for real-world applications immediately due to their vulnerability to adversarial attacks.
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Video transformer designs are based on self-attention that can model global context at a high computational cost.
Ranked #1 on
Action Recognition
on Diving-48
Self-regulating Prompts: Foundational Model Adaptation without Forgetting
To the best of our knowledge, this is the first regularization framework for prompt learning that avoids overfitting by jointly attending to pre-trained model features, the training trajectory during prompting, and the textual diversity.
Ranked #2 on
Prompt Engineering
on ImageNet V2
PromptIR: Prompting for All-in-One Blind Image Restoration
We present a prompt-based learning approach, PromptIR, for All-In-One image restoration that can effectively restore images from various types and levels of degradation.
Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors
We propose an efficient abnormal event detection model based on a lightweight masked auto-encoder (AE) applied at the video frame level.
Ranked #13 on
Anomaly Detection
on CUHK Avenue
Learnable Weight Initialization for Volumetric Medical Image Segmentation
Hybrid volumetric medical image segmentation models, combining the advantages of local convolution and global attention, have recently received considerable attention.
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks.
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Conversation agents fueled by Large Language Models (LLMs) are providing a new way to interact with visual data.
Ranked #5 on
Zero-Shot Video Question Answer
on TGIF-QA
Video-based Generative Performance Benchmarking (Consistency)
Video-based Generative Performance Benchmarking (Contextual Understanding)
+5
DFormer: Diffusion-guided Transformer for Universal Image Segmentation
Our DFormer outperforms the recent diffusion-based panoptic segmentation method Pix2Seq-D with a gain of 3. 6% on MS COCO val2017 set.
Modulate Your Spectrum in Self-Supervised Learning
In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse.
Salient Mask-Guided Vision Transformer for Fine-Grained Classification
Fine-grained visual classification (FGVC) is a challenging computer vision problem, where the task is to automatically recognise objects from subordinate categories.
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Then, we use two types of pre-defined tokens to mine co-saliency and background information via our proposed contrast-induced pixel-to-token correlation and co-saliency token-to-token correlation modules.
Ranked #1 on
Co-Salient Object Detection
on CoSal2015
Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement
Burst image processing is becoming increasingly popular in recent years.
Remote Sensing Change Detection With Transformers Trained from Scratch
Current transformer-based change detection (CD) approaches either employ a pre-trained model trained on large-scale image classification ImageNet dataset or rely on first pre-training on another CD dataset and then fine-tuning on the target benchmark.
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Through this prompting scheme, we can achieve state-of-the-art zero-shot performance on Kinetics-600, HMDB51 and UCF101 while remaining competitive in the supervised setting.
Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification
In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues.
Video Instance Segmentation in an Open-World
Open-world formulation relaxes the close-world static-learning assumption as follows: (a) first, it distinguishes a set of known categories as well as labels an unknown object as `unknown' and then (b) it incrementally learns the class of an unknown as and when the corresponding semantic labels become available.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
Burstormer: Burst Image Restoration and Enhancement Transformer
Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions.
StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing
A significant research effort is focused on exploiting the amazing capacities of pretrained diffusion models for the editing of images.
Ranked #7 on
Text-based Image Editing
on PIE-Bench
3D-Aware Multi-Class Image-to-Image Translation with NeRFs
In the second step, based on the well-trained multi-class 3D-aware GAN architecture, that preserves view-consistency, we construct a 3D-aware I2I translation system.
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Using our proposed efficient additive attention, we build a series of models called "SwiftFormer" which achieves state-of-the-art performance in terms of both accuracy and mobile inference speed.
Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection
Since the original formulation of our loss depends on the counts of true positives and false positives in a minibatch, we develop a differentiable proxy of our loss that can be used during training with other application-specific loss functions.
3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers
Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morphology.
3D Instance Segmentation via Enhanced Spatial and Semantic Supervision
Specifically, we provide the transformer block with spatial features to facilitate differentiation between similar object queries and incorporate semantic supervision to enhance prediction accuracy based on object class.
Guidance Through Surrogate: Towards a Generic Diagnostic Attack
Recently, different adversarial training defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well studied adversarial attacks such as PGD.
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks.
Fine-tuned CLIP Models are Efficient Video Learners
Since training on a similar scale for videos is infeasible, recent approaches focus on the effective transfer of image-based CLIP to the video domain.
Lightning Fast Video Anomaly Detection via Adversarial Knowledge Distillation
We propose a very fast frame-level model for anomaly detection in video, which learns to detect anomalies by distilling knowledge from multiple highly accurate object-level teacher models.
Ranked #16 on
Anomaly Detection
on CUHK Avenue
Person Image Synthesis via Denoising Diffusion Model
In this work, we show how denoising diffusion models can be applied for high-fidelity person image synthesis with strong sample diversity and enhanced mode coverage of the learnt data distribution.
An Investigation into Whitening Loss for Self-supervised Learning
A desirable objective in self-supervised learning (SSL) is to avoid feature collapse.
PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search
Our PS-ARM achieves state-of-the-art performance on both datasets.
MaPLe: Multi-modal Prompt Learning
Pre-trained vision-language (V-L) models such as CLIP have shown excellent generalization ability to downstream tasks.
Ranked #2 on
Prompt Engineering
on ImageNet-A
Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection
In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss.
Ranked #4 on
Anomaly Detection
on CUHK Avenue
CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection
In this work, we propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework, which takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene at multiple levels to predict a set of object bounding-boxes along with their corresponding semantic labels.
Transformers in Remote Sensing: A Survey
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade.
AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility
While being greatly benefiting to the tracking research, existing benchmarks do not pose the same difficulty as before with recent trackers achieving higher performance mainly due to (i) the introduction of more sophisticated transformers-based methods and (ii) the lack of diverse scenarios with adverse visibility such as, severe weather conditions, camouflage and imaging effects.
Multi-scale Feature Aggregation for Crowd Counting
The SkipAgg module directly propagates features with small receptive fields to features with much larger receptive fields.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
Self-Distilled Vision Transformer for Domain Generalization
Similar to CNNs, ViTs also struggle in out-of-distribution scenarios and the main culprit is overfitting to source domains.
Adversarial Pixel Restoration as a Pretext Task for Transferable Perturbations
Our training approach is based on a min-max scheme which reduces overfitting via an adversarial objective and thus optimizes for a more generalizable surrogate model.
SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection
A self-supervised multi-task learning (SSMTL) framework for video anomaly detection was recently introduced in literature.
Ranked #2 on
Anomaly Detection
on CUHK Avenue
Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection
Two popular forms of weak-supervision used in open-vocabulary detection (OVD) include pretrained CLIP model and image-level supervision.
Ranked #1 on
Open Vocabulary Object Detection
on OpenImages-v4
OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning
In the open-world SSL problem, the objective is to recognize samples of known classes, and simultaneously detect and cluster samples belonging to novel classes present in unlabeled data.
Ranked #1 on
Open-World Semi-Supervised Learning
on CIFAR-10
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
Our EdgeNeXt model with 1. 3M parameters achieves 71. 2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2. 2% with 28% reduction in FLOPs.
Ranked #29 on
Semantic Segmentation
on PASCAL VOC 2012 test
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging
We propose a novel self-supervised Video Object Segmentation (VOS) approach that strives to achieve better object-background discriminability for accurate object segmentation.
Learning Enriched Features for Fast Image Restoration and Enhancement
In the former case, spatial details are preserved but the contextual information cannot be precisely encoded.
Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution
Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple concatenated input tensors, where the kernel (receptive field) size controls the reduction rate of the spatial attention, and the number of convolutional filters controls the reduction rate of the channel attention, respectively.
Ranked #1 on
Image Super-Resolution
on IXI
PSTR: End-to-End One-Step Person Search With Transformers
We propose a novel one-step transformer-based person search framework, PSTR, that jointly performs person detection and re-identification (re-id) in a single architecture.
Energy-based Latent Aligner for Incremental Learning
Deep learning models tend to forget their earlier knowledge while incrementally learning new tasks.
Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer
When using the ResNet50 backbone, our MS-STS achieves a mask AP of 50. 1 %, outperforming the best reported results in literature by 2. 7 % and by 4. 8 % at higher overlap threshold of AP_75, while being comparable in model size and speed on Youtube-VIS 2019 val.
SepTr: Separable Transformer for Audio Spectrogram Processing
Following the successful application of vision transformers in multiple computer vision tasks, these models have drawn the attention of the signal processing community.
Ranked #1 on
Time Series Analysis
on Speech Commands
Transformers in Medical Imaging: A Survey
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators.
Spatio-temporal Relation Modeling for Few-shot Action Recognition
Experiments are performed on four few-shot action recognition benchmarks: Kinetics, SSv2, HMDB51 and UCF101.
Ranked #1 on
Few Shot Action Recognition
on UCF101
(using extra training data)
Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook
Accurate and robust visual object tracking is one of the most challenging and fundamental computer vision problems.
DoodleFormer: Creative Sketch Drawing with Transformers
Creative sketch image generation is a challenging vision problem, where the task is to generate diverse, yet realistic creative sketches possessing the unseen composition of the visual-world objects.
Self-supervised Video Transformer
To the best of our knowledge, the proposed approach is the first to alleviate the dependency on negative samples or dedicated memory banks in Self-supervised Video Transformer (SVT).
Ranked #55 on
Action Recognition
on UCF101
OW-DETR: Open-world Detection Transformer
In the case of incremental object detection, OW-DETR outperforms the state-of-the-art for all settings on PASCAL VOC.
Class-agnostic Object Detection with Multi-modal Transformer
This has been a long-standing question in computer vision.
Restormer: Efficient Transformer for High-Resolution Image Restoration
Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks.
Ranked #1 on
Grayscale Image Denoising
on Urban100 sigma15
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection
Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field.
Ranked #1 on
Anomaly Detection
on CUHK Avenue
(TBDC metric)
UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection
This is a closed-set scenario that fails to test the capability of systems at detecting new anomaly types.
Ranked #5 on
Anomaly Detection
on CUHK Avenue
(using extra training data)
CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation
Our neural model can be trained on unpaired images, due to the integration of a multi-level cycle-consistency loss.
Dense Gaussian Processes for Few-Shot Segmentation
Given the support set, our dense GP learns the mapping from local deep image features to mask values, capable of capturing complex appearance distributions.
Ranked #1 on
Few-Shot Semantic Segmentation
on COCO-20i (10-shot)
Burst Image Restoration and Enhancement
Our central idea is to create a set of pseudo-burst features that combine complementary information from all the input burst frames to seamlessly exchange information.
Ranked #2 on
Burst Image Super-Resolution
on BurstSR
Discriminative Region-based Multi-Label Zero-Shot Learning
We note that the best existing multi-label ZSL method takes a shared approach towards attending to region features with a common set of attention maps for all the classes.
Ranked #2 on
Multi-label zero-shot learning
on Open Images V4
Context-Conditional Adaptation for Recognizing Unseen Classes in Unseen Domains
Recent progress towards designing models that can generalize to unseen domains (i. e domain generalization) or unseen classes (i. e zero-shot learning) has embarked interest towards building models that can tackle both domain-shift and semantic shift simultaneously (i. e zero-shot domain generalization).
Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
The need to address the scarcity of task-specific annotated data has resulted in concerted efforts in recent years for specific settings such as zero-shot learning (ZSL) and domain generalization (DG), to separately address the issues of semantic shift and domain shift, respectively.
On Improving Adversarial Transferability of Vision Transformers
(ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT.
Intriguing Properties of Vision Transformers
We show and analyze the following intriguing properties of ViT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content.
MineGAN++: Mining Generative Models for Efficient Knowledge Transfer to Limited Data Domains
Therefore, we propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs.
Rich Semantics Improve Few-shot Learning
Human learning benefits from multi-modal inputs that often appear as rich semantics (e. g., description of an object's attributes while learning about it).
Ranked #1 on
Few-Shot Image Classification
on Oxford 102 Flower
(using extra training data)
Handwriting Transformers
We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns.
Deep Gaussian Processes for Few-Shot Segmentation
Through the expressivity of the GP, our approach is capable of modeling complex appearance distributions in the deep feature space.
On Generating Transferable Targeted Perturbations
To this end, we propose a new objective function that not only aligns the global distributions of source and target images, but also matches the local neighbourhood structure between the two domains.
Orthogonal Projection Loss
The CE loss encourages features of a class to have a higher projection score on the true class-vector compared to the negative classes.
Towards Open World Object Detection
Humans have a natural instinct to identify unknown object instances in their environments.
Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning
Equivariance or invariance has been employed standalone in the previous works; however, to the best of our knowledge, they have not been used jointly.
Multi-Stage Progressive Image Restoration
At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features.
Ranked #3 on
Spectral Reconstruction
on ARAD-1K
Generative Multi-Label Zero-Shot Learning
Nevertheless, computing reliable attention maps for unseen classes during inference in a multi-label setting is still a challenge.
Ranked #8 on
Multi-label zero-shot learning
on NUS-WIDE
Transformers in Vision: A Survey
Astounding results from Transformer models on natural language tasks have intrigued the vision community to study their application to computer vision problems.
Low Light Image Enhancement via Global and Local Context Modeling
These contexts can be crucial towards inferring several image enhancement tasks, e. g., local and global contrast, brightness and color corrections; which requires cues from both local and global spatial extent.
D2-Net: Weakly-Supervised Action Localization via Discriminative Embeddings and Denoised Activations
The proposed formulation comprises a discriminative and a denoising loss term for enhancing temporal action localization.
Ranked #3 on
Weakly Supervised Action Localization
on THUMOS’14
Learning to Fuse Asymmetric Feature Maps in Siamese Trackers
We propose a learnable module, called the asymmetric convolution (ACM), which learns to better capture the semantic correlation information in offline training on large-scale data.
Ranked #20 on
Visual Object Tracking
on TrackingNet
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture.
Ranked #2 on
Anomaly Detection
on UCSD Peds2
Abnormal Event Detection In Video
Anomaly Detection In Surveillance Videos
+4
Meta-learning the Learning Trends Shared Across Tasks
This demonstrates their ability to acquire transferable knowledge, a capability that is central to human learning.
Synthesizing the Unseen for Zero-shot Object Detection
The existing zero-shot detection approaches project visual features to the semantic domain for seen objects, hoping to map unseen objects to their corresponding semantics during inference.
Ranked #1 on
Zero-Shot Object Detection
on ImageNet Detection
From Handcrafted to Deep Features for Pedestrian Detection: A Survey
In addition to single-spectral pedestrian detection, we also review multi-spectral pedestrian detection, which provides more robust features for illumination variance.
AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results
This paper introduces the real image Super-Resolution (SR) challenge that was part of the Advances in Image Manipulation (AIM) workshop, held in conjunction with ECCV 2020.
A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video
Following the standard formulation of abnormal event detection as outlier detection, we propose a background-agnostic framework that learns from training videos containing only normal events.
Abnormal Event Detection In Video
Anomaly Detection In Surveillance Videos
+2
Image Colorization: A Survey and Dataset
Image colorization is the process of estimating RGB colors for grayscale images or video frames to improve their aesthetic and perceptual quality.
Stylized Adversarial Defense
In contrast to existing adversarial training methods that only use class-boundary information (e. g., using a cross-entropy loss), we propose to exploit additional information from the feature space to craft stronger adversaries that are in turn used to learn a robust model.
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3. 0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp.
Ranked #12 on
Real-time Instance Segmentation
on MSCOCO
Self-supervised Knowledge Distillation for Few-shot Learning
Our experiments show that, even in the first stage, self-supervision can outperform current state-of-the-art methods, with further gains achieved by our second stage distillation process.
Ranked #12 on
Few-Shot Image Classification
on FC100 5-way (5-shot)
A Self-supervised Approach for Adversarial Robustness
Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems e. g., for classification, segmentation and object detection.
Learning Human-Object Interaction Detection using Interaction Points
Human-object interaction (HOI) detection strives to localize both the human and an object as well as the identification of complex interactions between them.
Semi-supervised Learning for Few-shot Image-to-Image Translation
In this work, we go one step further and reduce the amount of required labeled data also from the source domain during training.
iTAML: An Incremental Task-Agnostic Meta-learning Approach
In this paper, we hypothesize this problem can be avoided by learning a set of generalized parameters, that are neither specific to old nor new tasks.
Incremental Object Detection via Meta-Learning
In a real-world setting, object instances from new classes can be continuously encountered by object detectors.
Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification
We propose to enforce semantic consistency at all stages of (generalized) zero-shot learning: training, feature synthesis and classification.
Ranked #2 on
Generalized Zero-Shot Learning
on CUB-200-2011
CycleISP: Real Image Restoration via Improved Data Synthesis
This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline.
Ranked #9 on
Image Denoising
on DND
(using extra training data)
Any-Shot Object Detection
Any-shot detection offers unique challenges compared to conventional novel object detection such as, a high imbalance between unseen, few-shot and seen object classes, susceptibility to forget base-training while learning novel classes and distinguishing novel classes from the background.
Learning Enriched Features for Real Image Restoration and Enhancement
With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing.
Ranked #4 on
Image Denoising
on DND
Learning Fast and Robust Target Models for Video Object Segmentation
The target appearance model consists of a light-weight module, which is learned during the inference stage using fast optimization techniques to predict a coarse but robust target segmentation.
PSC-Net: Learning Part Spatial Co-occurrence for Occluded Pedestrian Detection
On the heavy occluded (\textbf{HO}) set of CityPerosns test set, our PSC-Net obtains an absolute gain of 4. 0\% in terms of log-average miss rate over the state-of-the-art with same backbone, input scale and without using additional VBB supervision.
Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes
The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other.
Ranked #15 on
Fine-Grained Image Classification
on Stanford Dogs
Towards Partial Supervision for Generic Object Counting in Natural Scenes
Our RLC framework further reduces the annotation cost arising from large numbers of object categories in a dataset by only using lower-count supervision for a subset of categories and class-labels for the remaining ones.
Image Classification
Image-level Supervised Instance Segmentation
+3
MineGAN: effective knowledge transfer from GANs to target domains with few images
We propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs.
Random Path Selection for Continual Learning
In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity.
Ranked #7 on
Continual Learning
on F-CelebA (10 tasks)
Deep Contextual Attention for Human-Object Interaction Detection
Our approach outperforms the state-of-the-art on all datasets.
Mask-Guided Attention Network for Occluded Pedestrian Detection
Our approach obtains an absolute gain of 9. 5% in log-average miss rate, compared to the best reported results on the heavily occluded (HO) pedestrian set of CityPersons test set.
AnimalWeb: A Large-Scale Hierarchical Dataset of Annotated Animal Faces
Several studies show that animal needs are often expressed through their faces.
Multi-Modal Fusion for End-to-End RGB-T Tracking
Our tracker is trained in an end-to-end manner, enabling the components to learn how to fuse the information from both modalities.
Ranked #7 on
Rgb-T Tracking
on RGBT210
3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization
Our joint formulation has three terms: a classification term to ensure the separability of learned action features, an adapted multi-label center loss term to enhance the action feature discriminability and a counting loss term to delineate adjacent action sequences, leading to improved localization.
Ranked #1 on
Action Classification
on THUMOS'14
Action Classification
Weakly Supervised Action Localization
+2
Learning the Model Update for Siamese Trackers
In general, this template is linearly combined with the accumulated template from the previous frame, resulting in an exponential decay of information over time.
Distilled Siamese Networks for Visual Tracking
This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools.
An Adaptive Random Path Selection Approach for Incremental Learning
In a conventional supervised learning setting, a machine learning model has access to examples of all object classes that are desired to be recognized during the inference stage.
iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images
Compared to existing small-scale aerial image based instance segmentation datasets, iSAID contains 15$\times$ the number of object categories and 5$\times$ the number of instances.
Ranked #1 on
Object Detection
on iSAID
Cross-Domain Transferability of Adversarial Perturbations
To this end, we propose a framework capable of launching highly transferable attacks that crafts adversarial patterns to mislead networks trained on wholly different domains.
Discriminative Online Learning for Fast Video Object Segmentation
We propose a novel approach, based on a dedicated target appearance model that is exclusively learned online to discriminate between the target and background image regions.
Out-of-Distribution Detection for Generalized Zero-Shot Action Recognition
We introduce an out-of-distribution detector that determines whether the video features belong to a seen or unseen action category.
Action Recognition In Videos
Out-of-Distribution Detection
+2
Learning Digital Camera Pipeline for Extreme Low-Light Imaging
In low-light conditions, a conventional camera imaging pipeline produces sub-optimal images that are usually dark and noisy due to a low photon count and low signal-to-noise ratio (SNR).
Object Counting and Instance Segmentation with Image-level Supervision
Moreover, our approach improves state-of-the-art image-level supervised instance segmentation with a relative gain of 17. 8% in terms of average best overlap, on the PASCAL VOC 2012 dataset.
Ranked #1 on
Object Counting
on COCO count-test
Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Video
Most existing approaches formulate abnormal event detection as an outlier detection task, due to the scarcity of anomalous data during training.
Ranked #14 on
Anomaly Detection
on ShanghaiTech
A Generative Appearance Model for End-to-end Video Object Segmentation
One of the fundamental challenges in video object segmentation is to find an effective representation of the target and background appearance.
ATOM: Accurate Tracking by Overlap Maximization
We argue that this approach is fundamentally limited since target estimation is a complex task, requiring high-level knowledge about the object.
Ranked #7 on
Object Tracking
on FE108
Confidence Propagation through CNNs for Guided Sparse Depth Regression
In this paper, we propose an algebraically-constrained normalized convolution layer for CNNs with highly sparse input that has a smaller number of network parameters compared to related work.
Ranked #7 on
Depth Completion
on KITTI Depth Completion
Synthetic data generation for end-to-end thermal infrared tracking
These methods provide us with a large labeled dataset of synthetic TIR sequences, on which we can train end-to-end optimal features for tracking.
Propagating Confidences through CNNs for Sparse Data Regression
To tackle this challenging problem, we introduce an algebraically-constrained convolution layer for CNNs with sparse input and demonstrate its capabilities for the scene depth completion task.
Unveiling the Power of Deep Tracking
In the field of generic object tracking numerous attempts have been made to exploit deep features.
Density Adaptive Point Set Registration
Contrary to previous works, we model the underlying structure of the scene as a latent probability distribution, and thereby induce invariance to point set density changes.
DCCO: Towards Deformable Continuous Convolution Operators
Generally, DCF based trackers learn a rigid appearance model of the target.
Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification
To the best of our knowledge, we are the first to investigate Binary Patterns encoded CNNs and different deep network fusion architectures for texture recognition and remote sensing scene classification.
Ranked #12 on
Aerial Scene Classification
on AID (20% as trainset)
Deep Projective 3D Semantic Segmentation
Recent attempts, based on 3D deep learning approaches (3D-CNNs), have achieved below-expected results.
Ranked #15 on
Semantic Segmentation
on Semantic3D
Deep Motion Features for Visual Tracking
To the best of our knowledge, we are the first to propose fusing appearance information with deep motion features for visual tracking.
Scale Coding Bag of Deep Features for Human Attribute and Action Recognition
Most approaches to human attribute and action recognition in still images are based on image representation in which multi-scale local features are pooled across scale into a single, scale-invariant encoding.
ECO: Efficient Convolution Operators for Tracking
Moreover, our fast variant, using hand-crafted features, operates at 60 Hz on a single CPU, while obtaining 65. 0% AUC on OTB-2015.
Ranked #13 on
Visual Object Tracking
on VOT2017/18
Discriminative Scale Space Tracking
Compared to the standard exhaustive scale search, our approach achieves a gain of 2. 5% in average overlap precision on the OTB dataset.
Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking
We propose a novel generic approach for alleviating the problem of corrupted training samples in tracking-by-detection frameworks.
Learning Spatially Regularized Correlation Filters for Visual Tracking
These methods utilize a periodic assumption of the training samples to efficiently learn a classifier on all patches in the target neighborhood.
Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking
We also demonstrate the effectiveness of our learning formulation in extensive feature point tracking experiments.
A Probabilistic Framework for Color-Based Point Set Registration
On the Stanford Lounge dataset, our approach achieves a relative reduction of the failure rate by 78% compared to the baseline.
Adaptive Color Attributes for Real-Time Visual Tracking
This paper investigates the contribution of color in a tracking-by-detection framework.
Discriminative Color Descriptors
This results in a drop of discriminative power of the color description.
