Search Results for author:
Found 43 papers, 17 papers with code
T-VSL: Text-Guided Visual Sound Source Localization in Mixtures
Visual sound source localization poses a significant challenge in identifying the semantic region of each sounding source within a video.
Weakly-supervised Audio Separation via Bi-modal Semantic Similarity
Conditional sound separation in multi-source audio mixtures without having access to single source sound data during training is a long standing challenge.
PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference
Using this insight, we introduce PaPr, a method for substantially pruning redundant patches with minimal accuracy loss using lightweight ConvNets across a variety of deep learning architectures, including ViTs, ConvNets, and hybrid transformers, without any re-training.
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames.
FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer
With our method, a more fine-grained rank configuration can be generated automatically and yield up to 33% extra FLOPs reduction compared to a simple uniform configuration.
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Evol-Q improves the top-1 accuracy of a fully quantized ViT-Base by $10. 30\%$, $0. 78\%$, and $0. 15\%$ for $3$-bit, $4$-bit, and $8$-bit weight quantization levels.
MobileTL: On-device Transfer Learning with Inverted Residual Blocks
MobileTL trains the shifts for internal normalization layers to avoid storing activation maps for the backward pass.
CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers
Borrowing the idea of contrastive loss from self-supervised learning, we find a robust way to jointly minimize a loss function using just 1, 000 calibration images.
AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization
An audio-visual event (AVE) is denoted by the correspondence of the visual and auditory signals in a video segment.
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
To address this, we propose to finetune CLIP on a collection of masked image regions and their corresponding text descriptions.
QUIDAM: A Framework for Quantization-Aware DNN Accelerator and Model Co-Exploration
As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied precision or quantization levels, and model compression techniques, there is a need for design space exploration frameworks that incorporate quantization-aware processing elements into the accelerator design space while having accurate and fast power, performance, and area models.
Efficient Deep Learning Using Non-Volatile Memory Technology
DeepNVM++ is demonstrated on STT-/SOT-MRAM technologies and can be used for the characterization, modeling, and analysis of any NVM technology for last-level caches in GPUs for DL applications.
Play It Cool: Dynamic Shifting Prevents Thermal Throttling
Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices.
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners
The proposed Supervised MAE (SupMAE) only exploits a visible subset of image patches for classification, unlike the standard supervised pre-training where all image patches are used.
QADAM: Quantization-Aware DNN Accelerator Modeling for Pareto-Optimality
We also show that the proposed lightweight processing elements (LightPEs) consistently achieve Pareto-optimal results in terms of accuracy and hardware-efficiency.
QAPPA: Quantization-Aware Power, Performance, and Area Modeling of DNN Accelerators
As the machine learning and systems community strives to achieve higher energy-efficiency through custom DNN accelerators and model compression techniques, there is a need for a design space exploration framework that incorporates quantization-aware processing elements into the accelerator design space while having accurate and fast power, performance, and area models.
Width Transfer: On the (In)variance of Width Optimization
In this work, we propose width transfer, a technique that harnesses the assumptions that the optimized widths (or channel counts) are regular across sizes and depths.
DeepNVM++: Cross-Layer Modeling and Optimization Framework of Non-Volatile Memories for Deep Learning
Under iso-area assumptions, STT-MRAM and SOT-MRAM provide up to 2x and 2. 3x EDP reduction and accommodate 2. 3x and 3. 3x cache capacity when compared to SRAM, respectively.
The Architectural Implications of Distributed Reinforcement Learning on CPU-GPU Systems
With deep reinforcement learning (RL) methods achieving results that exceed human capabilities in games, robotics, and simulated environments, continued scaling of RL training is crucial to its deployment in solving complex real-world problems.
PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks
Slimmable neural networks provide a flexible trade-off front between prediction error and computational cost (such as the number of floating-point operations or FLOPs) with the same storage cost as a single model.
One Weight Bitwidth to Rule Them All
Weight quantization for deep ConvNets has shown promising results for applications such as image classification and semantic segmentation and is especially important for applications where memory storage is limited.
Joslim: Joint Widths and Weights Optimization for Slimmable Neural Networks
In this work, we propose a general framework to enable joint optimization for both width configurations and weights of slimmable networks.
Renofeation: A Simple Transfer Learning Method for Improved Adversarial Robustness
Fine-tuning through knowledge transfer from a pre-trained model on a large-scale dataset is a widely spread approach to effectively build models on small-scale datasets.
On the Pareto Efficiency of Quantized CNN
Weight Quantization for deep convolutional neural networks (CNNs) has shown promising results in compressing and accelerating CNN-powered applications such as semantic segmentation, gesture recognition, and scene understanding.
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization
In this work, we alleviate the NAS search cost down to less than 3 hours, while achieving state-of-the-art image classification results under mobile latency constraints.
ViP: Virtual Pooling for Accelerating CNN-based Image Classification and Object Detection
In this paper, we propose Virtual Pooling (ViP), a model-level approach to improve speed and energy consumption of CNN-based image classification and object detection tasks, with a provable error bound.
Single-Path NAS: Device-Aware Efficient ConvNet Design
Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the latency constraint of a mobile device?
Towards Efficient Model Compression via Learned Global Ranking
First, both the accuracy and the speed of ConvNets can affect the performance of the application.
FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference
Over 46 FPGA-design experiments involving eight configurations and four data sets reveal that lightweight neural networks with a flexible $k$ value (dubbed FLightNNs) fully utilize the hardware resources on Field Programmable Gate Arrays (FPGAs), our experimental results show that FLightNNs can achieve 2$\times$ speedup when compared to lightweight NNs with $k=2$, with only 0. 1\% accuracy degradation.
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours
Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the runtime constraint of a mobile device?
Ranked #891 on
Image Classification
on ImageNet
Regularizing Activation Distribution for Training Binarized Deep Networks
Binarized Neural Networks (BNNs) can significantly reduce the inference latency and energy consumption in resource-constrained devices due to their pure-logical computation and fewer memory accesses.
AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling
In vision-enabled autonomous systems such as robots and autonomous cars, video object detection plays a crucial role, and both its speed and accuracy are important factors to provide reliable operation.
Understanding the Impact of Label Granularity on CNN-based Image Classification
In this paper, we conduct extensive experiments using various datasets to demonstrate and analyze how and why training based on fine-grain labeling, such as "Persian cat" can improve CNN accuracy on classifying coarse-grain classes, in this case "cat."
Learning-based Application-Agnostic 3D NoC Design for Heterogeneous Manycore Systems
Our results show that these generalized 3D NoCs only incur a 1. 8% (36-tile system) and 1. 1% (64-tile system) average performance loss compared to application-specific NoCs.
Differentiable Training for Hardware Efficient LightNNs
To reduce runtime and resource utilization of Deep Neural Networks (DNNs) on customized hardware, LightNN has been proposed by constraining the weights of DNNs to be a sum of a limited number (denoted as $k\in\{1, 2\}$) of powers of 2.
Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks
Resource-efficient convolution neural networks enable not only the intelligence on edge devices but also opportunities in system-level optimization such as scheduling.
Hardware-Aware Machine Learning: Modeling and Optimization
What is the latency or energy cost for an inference made by a Deep Neural Network (DNN)?
Designing Adaptive Neural Networks for Energy-Constrained Image Classification
We cast the design of adaptive CNNs as a hyper-parameter optimization problem with respect to energy, accuracy, and communication constraints imposed by the mobile device.
Tractable Learning and Inference for Large-Scale Probabilistic Boolean Networks
Probabilistic Boolean Networks (PBNs) have been previously proposed so as to gain insights into complex dy- namical systems.
HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks
While selecting the hyper-parameters of Neural Networks (NNs) has been so far treated as an art, the emergence of more complex, deeper architectures poses increasingly more challenges to designers and Machine Learning (ML) practitioners, especially when power and memory constraints need to be considered.
LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks
For a fixed DNN configuration, LightNNs have better accuracy at a slight energy increase than BNNs, yet are more energy efficient with only slightly less accuracy than conventional DNNs.
Machine Learning and Manycore Systems Design: A Serendipitous Symbiosis
Tight collaboration between experts of machine learning and manycore system design is necessary to create a data-driven manycore design framework that integrates both learning and expert knowledge.
NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks
We also propose the "energy-precision ratio" (EPR) metric to guide machine learners in selecting an energy-efficient CNN architecture that better trades off the energy consumption and prediction accuracy.
