Distilling the Knowledge in Data Pruning

no code implementations • 12 Mar 2024 • Emanuel Ben-Baruch, Adam Botach, Igor Kviatkovsky, Manoj Aggarwal, Gérard Medioni

In this paper we explore the application of data pruning while incorporating knowledge distillation (KD) when training on a pruned subset.

Knowledge Distillation

Solving ImageNet: a Unified Scheme for Training any Backbone to Top Results

2 code implementations • 7 Apr 2022 • Tal Ridnik, Hussam Lawen, Emanuel Ben-Baruch, Asaf Noy

The scheme, named USI (Unified Scheme for ImageNet), is based on knowledge distillation and modern tricks.

Image Classification Knowledge Distillation

It's All in the Head: Representation Knowledge Distillation through Classifier Sharing

1 code implementation • 18 Jan 2022 • Emanuel Ben-Baruch, Matan Karklinsky, Yossi Biton, Avi Ben-Cohen, Hussam Lawen, Nadav Zamir

Such direct methods may be limited in transferring high-order dependencies embedded in the representation vectors, or in handling the capacity gap between the teacher and student models.

Classification Face Verification +3

ML-Decoder: Scalable and Versatile Classification Head

1 code implementation • 25 Nov 2021 • Tal Ridnik, Gilad Sharir, Avi Ben-Cohen, Emanuel Ben-Baruch, Asaf Noy

In this paper, we introduce ML-Decoder, a new attention-based classification head.

Classification Fine-Grained Image Classification +2

Multi-label Classification with Partial Annotations using Class-aware Selective Loss

1 code implementation • CVPR 2022 • Emanuel Ben-Baruch, Tal Ridnik, Itamar Friedman, Avi Ben-Cohen, Nadav Zamir, Asaf Noy, Lihi Zelnik-Manor

We propose to estimate the class distribution using a dedicated temporary model, and we show its improved efficiency over a naive estimation computed using the dataset's partial annotations.

Missing Labels

PETA: Photo Albums Event Recognition using Transformers Attention

1 code implementation • 26 Sep 2021 • Tamar Glaser, Emanuel Ben-Baruch, Gilad Sharir, Nadav Zamir, Asaf Noy, Lihi Zelnik-Manor

We address this gap with a tailor-made solution, combining the power of CNNs for image representation and transformers for album representation to perform global reasoning on image collection, offering a practical and efficient solution for photo albums event recognition.

ImageNet-21K Pretraining for the Masses

4 code implementations • 22 Apr 2021 • Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, Lihi Zelnik-Manor

ImageNet-1K serves as the primary dataset for pretraining deep learning models for computer vision tasks.

Action Recognition Fine-Grained Image Classification +1

Asymmetric Loss For Multi-Label Classification

5 code implementations • ICCV 2021 • Emanuel Ben-Baruch, Tal Ridnik, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, Lihi Zelnik-Manor

In this paper, we introduce a novel asymmetric loss ("ASL"), which operates differently on positive and negative samples.

Classification General Classification +4