Search Results for author:
Found 6 papers, 5 papers with code
Bootstrap Masked Visual Modeling via Hard Patches Mining
To empower the model as a teacher, we propose Hard Patches Mining (HPM), predicting patch-wise losses and subsequently determining where to mask.
Semantic-Aware Autoregressive Image Modeling for Visual Representation Learning
In this study, inspired by human beings' way of grasping an image, i. e., focusing on the main object first, we present a semantic-aware autoregressive image modeling (SemAIM) method to tackle this challenge.
DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions
As it is empirically observed that Vision Transformers (ViTs) are quite insensitive to the order of input tokens, the need for an appropriate self-supervised pretext task that enhances the location awareness of ViTs is becoming evident.
Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning
Different from existing SSL-KD methods that transfer knowledge from a static pre-trained teacher to a student, in MOKD, two different models learn collaboratively in a self-supervised manner.
Hard Patches Mining for Masked Image Modeling
We observe that the reconstruction loss can naturally be the metric of the difficulty of the pre-training task.
Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning
In contrastive self-supervised learning, the common way to learn discriminative representation is to pull different augmented "views" of the same image closer while pushing all other images further apart, which has been proven to be effective.
