Search Results for author:
Found 59 papers, 31 papers with code
LongVLM: Efficient Long Video Understanding via Large Language Models
In this way, we encode video representations that incorporate both local and global information, enabling the LLM to generate comprehensive responses for long-term videos.
MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images
We propose MVSplat, an efficient feed-forward 3D Gaussian Splatting model learned from sparse multi-view images.
Ranked #1 on
Generalizable Novel View Synthesis
on ACID
Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM
Human motion generation stands as a significant pursuit in generative computer vision, while achieving long-sequence and efficient motion generation remains challenging.
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model.
ModaVerse: Efficiently Transforming Modalities with LLMs
This alignment process, which synchronizes a language model trained on textual data with encoders and decoders trained on multi-modal data, often necessitates extensive training of several projection layers in multiple stages.
Efficient Stitchable Task Adaptation
In this work, we present a novel framework, Efficient Stitchable Task Adaptation (ESTA), to efficiently produce a palette of fine-tuned models that adhere to diverse resource constraints.
Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction
Then, we adapt the SR model by implementing feature-level reconstruction learning from the initial test image to its second-order degraded counterparts, which helps the SR model generate plausible HR images.
Mask Propagation for Efficient Video Semantic Segmentation
By reusing predictions from key frames, we circumvent the need to process a large volume of video frames individually with resource-intensive segmentors, alleviating temporal redundancy and significantly reducing computational costs.
Object-aware Inversion and Reassembly for Image Editing
Subsequently, we propose an additional reassembly step to seamlessly integrate the respective editing results and the non-editing region to obtain the final edited image.
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Additionally, an adaptive strategy is designed to autonomously determine the optimal number of sub-channels for channel disassembly.
EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models
In this paper, we introduce a data-free and parameter-efficient fine-tuning framework for low-bit diffusion models, dubbed EfficientDM, to achieve QAT-level performance with PTQ-like efficiency.
SwitchGPT: Adapting Large Language Models for Non-Text Outputs
To bridge this gap, we propose a novel approach, \methodname, from a modality conversion perspective that evolves a text-based LLM into a multi-modal one.
Stitched ViTs are Flexible Vision Backbones
With extensive experiments on ImageNet-1K, ADE20K, COCO-Stuff-10K and NYUv2, SN-Netv2 demonstrates superior performance over SN-Netv1 on downstream dense predictions and shows strong ability as a flexible vision backbone, achieving great advantages in both training efficiency and deployment flexibility.
LoRAPrune: Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
This is due to their utilization of unstructured pruning on LPMs, impeding the merging of LoRA weights, or their dependence on the gradients of pre-trained weights to guide pruning, which can impose significant memory overhead.
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease storage burden and optimization difficulty.
Stitchable Neural Networks
As each model family consists of pretrained models with diverse scales (e. g., DeiT-Ti/S/B), it naturally arises a fundamental question of how to efficiently assemble these readily available models in a family for dynamic accuracy-efficiency trade-offs at runtime.
A Survey on Efficient Training of Transformers
Recent advances in Transformers have come with a huge requirement on computing resources, highlighting the importance of developing efficient training techniques to make Transformer training faster, at lower cost, and to higher accuracy by the efficient use of computation and memory resources.
BiViT: Extremely Compressed Binary Vision Transformers
To solve this, we propose Softmax-aware Binarization, which dynamically adapts to the data distribution and reduces the error caused by binarization.
BiViT: Extremely Compressed Binary Vision Transformer
To solve this, we propose Softmax-aware Binarization, which dynamically adapts to the data distribution and reduces the error caused by binarization.
EcoFormer: Energy-Saving Attention with Linear Complexity
To this end, we propose a new binarization paradigm customized to high-dimensional softmax attention via kernelized hashing, called EcoFormer, to map the original queries and keys into low-dimensional binary codes in Hamming space.
FocusFormer: Focusing on What We Need via Architecture Sampler
During architecture search, these methods focus on finding architectures on the Pareto frontier of performance and resource consumption, which forms a gap between training and deployment.
An Efficient Spatio-Temporal Pyramid Transformer for Action Detection
The task of action detection aims at deducing both the action category and localization of the start and end moment for each action instance in a long, untrimmed video.
Fast Vision Transformers with HiLo Attention
Therefore, we propose to disentangle the high/low frequency patterns in an attention layer by separating the heads into two groups, where one group encodes high frequencies via self-attention within each local window, and another group encodes low frequencies by performing global attention between the average-pooled low-frequency keys and values from each window and each query position in the input feature map.
Ranked #281 on
Image Classification
on ImageNet
Dynamic Focus-aware Positional Queries for Semantic Segmentation
In this paper, we propose a simple yet effective query design for semantic segmentation termed Dynamic Focus-aware Positional Queries (DFPQ), which dynamically generates positional queries conditioned on the cross-attention scores from the preceding decoder block and the positional encodings for the corresponding image features, simultaneously.
Ranked #21 on
Semantic Segmentation
on ADE20K
Automated Progressive Learning for Efficient Training of Vision Transformers
First, we develop a strong manual baseline for progressive learning of ViTs, by introducing momentum growth (MoGrow) to bridge the gap brought by model growth.
Sharpness-aware Quantization for Deep Neural Networks
However, the abrupt changes in quantized weights during training often lead to severe loss fluctuations and result in a sharp loss landscape, making the gradients unstable and thus degrading the performance.
Ranked #196 on
Image Classification
on CIFAR-100
Pruning Self-attentions into Convolutional Layers in Single Path
Relying on the single-path space, we introduce learnable binary gates to encode the operation choices in MSA layers.
Ranked #18 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Mesa: A Memory-saving Training Framework for Transformers
While Transformers have delivered significant performance improvements, training such networks is extremely memory intensive owing to storing all intermediate activations that are needed for gradient computation during backpropagation, especially for long sequences.
Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints
More critically, EAS is able to find compact architectures within 0. 1 second for 50 deployment scenarios.
Less is More: Pay Less Attention in Vision Transformers
Transformers have become one of the dominant architectures in deep learning, particularly as a powerful alternative to convolutional neural networks (CNNs) in computer vision.
End-to-end One-shot Human Parsing
To address three main challenges in OSHP, i. e., small sizes, testing bias, and similar parts, we devise an End-to-end One-shot human Parsing Network (EOP-Net).
Scalable Vision Transformers with Hierarchical Pooling
However, the routine of the current ViT model is to maintain a full-length patch sequence during inference, which is redundant and lacks hierarchical representation.
Ranked #22 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Single-path Bit Sharing for Automatic Loss-aware Model Compression
By jointly training the binary gates in conjunction with network parameters, the compression configurations of each layer can be automatically determined.
Fully Quantized Image Super-Resolution Networks
With the rising popularity of intelligent mobile devices, it is of great practical significance to develop accurate, realtime and energy-efficient image Super-Resolution (SR) inference methods.
FATNN: Fast and Accurate Ternary Neural Networks
Ternary Neural Networks (TNNs) have received much attention due to being potentially orders of magnitude faster in inference, as well as more power efficient, than full-precision counterparts.
AQD: Towards Accurate Fully-Quantized Object Detection
Network quantization allows inference to be conducted using low-precision arithmetic for improved inference efficiency of deep neural networks on edge devices.
Role-Wise Data Augmentation for Knowledge Distillation
To our knowledge, existing methods overlook the fact that although the student absorbs extra knowledge from the teacher, both models share the same input data -- and this data is the only medium by which the teacher's knowledge can be demonstrated.
Generative Low-bitwidth Data Free Quantization
More critically, our method achieves much higher accuracy on 4-bit quantization than the existing data free quantization method.
Ranked #2 on
Data Free Quantization
on CIFAR-100
Switchable Precision Neural Networks
Instantaneous and on demand accuracy-efficiency trade-off has been recently explored in the context of neural networks slimming.
Automatic Pruning for Quantized Neural Networks
In particular, for ResNet-18 on ImageNet, we prune 26. 12% of the model size with Binarized Neural Network quantization, achieving a top-1 classification accuracy of 47. 32% in a model of 2. 47 MB and 59. 30% with a 2-bit DoReFa-Net in 4. 36 MB.
Discrimination-aware Network Pruning for Deep Model Compression
In this paper, we propose a simple-yet-effective method called discrimination-aware channel pruning (DCP) to choose the channels that actually contribute to the discriminative power.
Structured Binary Neural Networks for Image Recognition
Experiments on both classification, semantic segmentation and object detection tasks demonstrate the superior performance of the proposed methods over various quantized networks in the literature.
Auxiliary Learning for Deep Multi-task Learning
The most current methods can be categorized as either: (i) hard parameter sharing where a subset of the parameters is shared among tasks while other parameters are task-specific; or (ii) soft parameter sharing where all parameters are task-specific but they are jointly regularized.
RBCN: Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs
Binarized convolutional neural networks (BCNNs) are widely used to improve memory and computation efficiency of deep convolutional neural networks (DCNNs) for mobile and AI chips based applications.
Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
Furthermore, we propose a second progressive quantization scheme which gradually decreases the bit-width from high-precision to low-precision during training.
Training Quantized Neural Networks with a Full-precision Auxiliary Module
In this paper, we seek to tackle a challenge in training low-precision networks: the notorious difficulty in propagating gradient through a low-precision network due to the non-differentiable quantization function.
Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
In this paper, we propose to train convolutional neural networks (CNNs) with both binarized weights and activations, leading to quantized models specifically} for mobile devices with limited power capacity and computation resources.
Discrimination-aware Channel Pruning for Deep Neural Networks
Channel pruning is one of the predominant approaches for deep model compression.
Training Compact Neural Networks with Binary Weights and Low Precision Activations
In this paper, we propose to train a network with binary weights and low-bitwidth activations, designed especially for mobile devices with limited power consumption.
Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queries
To this end we propose a unified framework, the ParalleL AttentioN (PLAN) network, to discover the object in an image that is being referred to in variable length natural expression descriptions, from short phrases query to long multi-round dialogs.
Towards Effective Low-bitwidth Convolutional Neural Networks
This paper tackles the problem of training a deep convolutional neural network with both low-precision weights and low-bitwidth activations.
Towards Context-Aware Interaction Recognition for Visual Relationship Detection
The proposed method still builds one classifier for one interaction (as per type (ii) above), but the classifier built is adaptive to context via weights which are context dependent.
TasselNet: Counting maize tassels in the wild via local counts regression network
To our knowledge, this is the first time that a plant-related counting problem is considered using computer vision technologies under unconstrained field-based environment.
Care about you: towards large-scale human-centric visual relationship detection
In addressing this problem we first construct a large-scale human-centric visual relationship detection dataset (HCVRD), which provides many more types of relationship annotation (nearly 10K categories) than the previous released datasets.
Human-Object Interaction Detection
Relationship Detection
+1
Towards Context-aware Interaction Recognition
Recognizing how objects interact with each other is a crucial task in visual recognition.
Sequential Person Recognition in Photo Albums with a Recurrent Network
In this work, we propose to model the relational information between people as a sequence prediction task.
Attend in groups: a weakly-supervised deep learning framework for learning from web data
Large-scale datasets have driven the rapid development of deep neural networks for visual recognition.
Visual Tracking via Shallow and Deep Collaborative Model
In the discriminative model, we exploit the advances of deep learning architectures to learn generic features which are robust to both background clutters and foreground appearance variations.
Fast Training of Triplet-based Deep Binary Embedding Networks
To solve the first stage, we design a large-scale high-order binary codes inference algorithm to reduce the high-order objective to a standard binary quadratic problem such that graph cuts can be used to efficiently infer the binary code which serve as the label of each training datum.
