Search Results for author:
Found 5 papers, 3 papers with code
Agent Attention: On the Integration of Softmax and Linear Attention
Specifically, the Agent Attention, denoted as a quadruple $(Q, A, K, V)$, introduces an additional set of agent tokens $A$ into the conventional attention module.
FAAC: Facial Animation Generation with Anchor Frame and Conditional Control for Superior Fidelity and Editability
Our method's efficacy has been validated on multiple representative DreamBooth and LoRA models, delivering substantial improvements over the original outcomes in terms of facial fidelity, text-to-image editability, and video motion.
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
Experiments on various transformers demonstrate the effectiveness of our method, while analysis experiments prove our higher robustness to the errors of the token pruning policy.
Ranked #1 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Self-attention mechanism has been a key factor in the recent progress of Vision Transformer (ViT), which enables adaptive feature extraction from global contexts.
Contrastive Language-Image Pre-Training with Knowledge Graphs
Recent years have witnessed the fast development of large-scale pre-training frameworks that can extract multi-modal representations in a unified form and achieve promising performances when transferred to downstream tasks.
