Search Results for author:
Found 7 papers, 3 papers with code
Knowledge Distillation Based on Transformed Teacher Matching
As a technique to bridge logit matching and probability distribution matching, temperature scaling plays a pivotal role in knowledge distillation (KD).
Ranked #12 on
Knowledge Distillation
on ImageNet
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information
To improve this estimate for KD, in this paper we introduce the concept of conditional mutual information (CMI) into the estimation of BCPD and propose a novel estimator called the maximum CMI (MCMI) method.
AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
AdaFed adaptively tunes this common direction based on the values of local gradients and loss functions.
Conditional Mutual Information Constrained Deep Learning for Classification
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intra-class concentration and inter-class separation of the DNN, respectively.
Deep Selector-JPEG: Adaptive JPEG Image Compression for Computer Vision in Image classification with Human Vision Criteria
Experimental results show that in comparison with JPEG at the same CR, Deep Selector-JPEG achieves better Rate-Accuracy performance over the ImageNet validation set for all tested DNN classifiers with gains in classification accuracy between 0. 2% and 1% at the same CRs while satisfying HV constraints.
High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networks
Our offline selections yield CNN inference time savings up to 9% and CR up to 10x.
Targeted Attack for Deep Hashing based Retrieval
In this paper, we propose a novel method, dubbed deep hashing targeted attack (DHTA), to study the targeted attack on such retrieval.
