Search Results for author:
Found 8 papers, 5 papers with code
MILAN: Masked Image Pretraining on Language Assisted Representation
When the masked autoencoder is pretrained and finetuned on ImageNet-1K dataset with an input resolution of 224x224, MILAN achieves a top-1 accuracy of 85. 4% on ViT-Base, surpassing previous state-of-the-arts by 1%.
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation
Large pretrained language models (PLMs) are often domain- or task-adapted via fine-tuning or prompting.
CHEX: CHannel EXploration for CNN Model Compression
However, conventional pruning methods have limitations in that: they are restricted to pruning process only, and they require a fully pre-trained large model.
Multi-Dimensional Model Compression of Vision Transformer
In contrast, we advocate a multi-dimensional ViT compression paradigm, and propose to harness the redundancy reduction from attention head, neuron and sequence dimensions jointly.
Few-shot Learning via Dependency Maximization and Instance Discriminant Analysis
We study the few-shot learning (FSL) problem, where a model learns to recognize new objects with extremely few labeled training data per category.
Effective Model Sparsification by Scheduled Grow-and-Prune Methods
It addresses the shortcomings of the previous works by repeatedly growing a subset of layers to dense and then pruning them back to sparse after some training.
A Feature-map Discriminant Perspective for Pruning Deep Neural Networks
Network pruning has become the de facto tool to accelerate deep neural networks for mobile and edge applications.
Scalable Kernel Learning via the Discriminant Information
Kernel approximation methods create explicit, low-dimensional kernel feature maps to deal with the high computational and memory complexity of standard techniques.
