Search Results for author:
Found 44 papers, 15 papers with code
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
Kullback-Leiber divergence has been widely used in Knowledge Distillation (KD) to compress Large Language Models (LLMs).
Taming Lookup Tables for Efficient Image Retouching
Existing enhancement models often optimize for high performance while falling short of reducing hardware inference time and power consumption, especially on edge devices with constrained computing and storage resources.
APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Models
Large Language Models (LLMs) have greatly advanced the natural language processing paradigm.
LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
Various parameter-efficient fine-tuning (PEFT) techniques have been proposed to enable computationally efficient fine-tuning while maintaining model performance.
Learning Spatially Collaged Fourier Bases for Implicit Neural Representation
Existing approaches to Implicit Neural Representation (INR) can be interpreted as a global scene representation via a linear combination of Fourier bases of different frequencies.
A Unifying Tensor View for Lightweight CNNs
Despite the decomposition of convolutional kernels for lightweight CNNs being well studied, existing works that rely on tensor network diagrams or hyperdimensional abstraction lack geometry intuition.
Hundred-Kilobyte Lookup Tables for Efficient Single-Image Super-Resolution
Conventional super-resolution (SR) schemes make heavy use of convolutional neural networks (CNNs), which involve intensive multiply-accumulate (MAC) operations, and require specialized hardware such as graphics processing units.
Lite it fly: An All-Deformable-Butterfly Network
Most deep neural networks (DNNs) consist fundamentally of convolutional and/or fully connected layers, wherein the linear transform can be cast as the product between a filter matrix and a data matrix obtained by arranging feature tensors into columns.
A Spectral Perspective towards Understanding and Improving Adversarial Robustness
Deep neural networks (DNNs) are incredibly vulnerable to crafted, imperceptible adversarial perturbations.
Overcoming Beam Squint in Dual-Wideband mmWave MIMO Channel Estimation: A Bayesian Multi-Band Sparsity Approach
A probabilistic model is built to induce the common sparsity in the spatial domain, and the first-order Taylor expansion is adopted to get rid of the grid mismatch in the dictionaries.
To Fold or Not to Fold: Graph Regularized Tensor Train for Visual Data Completion
Tensor train (TT) representation has achieved tremendous success in visual data completion tasks, especially when it is combined with tensor folding.
Weight-Inherited Distillation for Task-Agnostic BERT Compression
Knowledge Distillation (KD) is a predominant approach for BERT compression.
Context-Aware Transformer for 3D Point Cloud Automatic Annotation
3D automatic annotation has received increased attention since manually annotating 3D point clouds is laborious.
Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Distilling high-accuracy Graph Neural Networks~(GNNs) to low-latency multilayer perceptrons~(MLPs) on graph tasks has become a hot research topic.
DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference
To accelerate the inference of deep neural networks (DNNs), quantization with low-bitwidth numbers is actively researched.
Frequency Regularization for Improving Adversarial Robustness
Deep neural networks are incredibly vulnerable to crafted, human-imperceptible adversarial perturbations.
ODG-Q: Robust Quantization via Online Domain Generalization
To our best knowledge, this work is the first work that trains both quantized and binary neural networks on ImageNet that consistently improve robustness under different attacks.
PECAN: A Product-Quantized Content Addressable Memory Network
A novel deep neural network (DNN) architecture is proposed wherein the filtering and linear transform are realized solely with product quantization (PQ).
Multimodal Transformer for Automatic 3D Annotation and Object Detection
By enriching the sparse point clouds, our method achieves 4. 48\% and 4. 03\% better 3D AP on KITTI moderate and hard samples, respectively, versus the state-of-the-art autolabeler.
Multilayer Perceptron Based Stress Evolution Analysis under DC Current Stressing for Multi-segment Wires
Electromigration (EM) is one of the major concerns in the reliability analysis of very large scale integration (VLSI) systems due to the continuous technology scaling.
MAP-Gen: An Automated 3D-Box Annotation Flow with Multimodal Attention Point Generator
While a few previous studies tried to automatically generate 3D bounding boxes from weak labels such as 2D boxes, the quality is sub-optimal compared to human annotators.
Coarse to Fine: Image Restoration Boosted by Multi-Scale Low-Rank Tensor Completion
Existing low-rank tensor completion (LRTC) approaches aim at restoring a partially observed tensor by imposing a global low-rank constraint on the underlying completed tensor.
A Space-Time Neural Network for Analysis of Stress Evolution under DC Current Stressing
The electromigration (EM)-induced reliability issues in very large scale integration (VLSI) circuits have attracted increased attention due to the continuous technology scaling.
Deformable Butterfly: A Highly Structured and Sparse Linear Transform
We introduce a new kind of linear transform named Deformable Butterfly (DeBut) that generalizes the conventional butterfly matrices and can be adapted to various input-output dimensions.
Compression of Generative Pre-trained Language Models via Quantization
We find that previous quantization methods fail on generative tasks due to the \textit{homogeneous word embeddings} caused by reduced capacity, and \textit{varied distribution of weights}.
What Do Adversarially trained Neural Networks Focus: A Fourier Domain-based Study
We are hopeful this work can shed light on the design of more robust neural networks.
Exploiting Elasticity in Tensor Ranks for Compressing Neural Networks
Elasticities in depth, width, kernel size and resolution have been explored in compressing deep neural networks (DNNs).
EZCrop: Energy-Zoned Channels for Robust Output Pruning
Recent results have revealed an interesting observation in a trained convolutional neural network (CNN), namely, the rank of a feature map channel matrix remains surprisingly constant despite the input images.
AET-EFN: A Versatile Design for Static and Dynamic Event-Based Vision
The neuromorphic event cameras, which capture the optical changes of a scene, have drawn increasing attention due to their high speed and low power consumption.
FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation
Prior arts often discretize the network weights by carefully tuning hyper-parameters of quantization (e. g. non-uniform stepsize and layer-wise bitwidths), which are complicated and sub-optimal because the full-precision and low-precision models have a large discrepancy.
S3-Net: A Fast and Lightweight Video Scene Understanding Network by Single-shot Segmentation
Real-time understanding in video is crucial in various AI applications such as autonomous driving.
Tensor Train Factorization and Completion under Noisy Data with Prior Analysis and Rank Estimation
Tensor train (TT) decomposition, a powerful tool for analyzing multidimensional data, exhibits superior performance in many machine learning tasks.
HOTCAKE: Higher Order Tucker Articulated Kernels for Deeper CNN Compression
The emerging edge computing has promoted immense interests in compacting a neural network without sacrificing much accuracy.
Kernelized Support Tensor Train Machines
In this paper, we propose a tensor train (TT)-based kernel technique for the first time, and apply it to the conventional support vector machine (SVM) for image classification.
Fastened CROWN: Tightened Neural Network Robustness Certificates
We draw inspiration from such work and further demonstrate the optimality of deterministic CROWN (Zhang et al. 2018) solutions in a given linear programming problem under mild constraints.
MiSC: Mixed Strategies Crowdsourcing
Popular crowdsourcing techniques mostly focus on evaluating workers' labeling quality before adjusting their weights during label aggregation.
POPQORN: Quantifying Robustness of Recurrent Neural Networks
The vulnerability to adversarial attacks has been a critical issue for deep neural networks.
Matrix Product Operator Restricted Boltzmann Machines
This work presents the matrix product operator RBM (MPORBM) that utilizes a tensor network generalization of Mv/TvRBM, preserves input formats in both the visible and hidden layers, and results in higher expressive power.
Deep Compression of Sum-Product Networks on Tensor Networks
Sum-product networks (SPNs) represent an emerging class of neural networks with clear probabilistic semantics and superior inference speed over graphical models.
A Support Tensor Train Machine
There has been growing interest in extending traditional vector-based machine learning techniques to their tensor forms.
Fast and Accurate Tensor Completion with Total Variation Regularized Tensor Trains
We propose a new tensor completion method based on tensor trains.
Parallelized Tensor Train Learning of Polynomial Classifiers
In pattern classification, polynomial classifiers are well-studied methods as they are capable of generating complex decision surfaces.
A Tensor Network Kalman filter with an application in recursive MIMO Volterra system identification
This article introduces a Tensor Network Kalman filter, which can estimate state vectors that are exponentially large without ever having to explicitly construct them.
Systems and Control
A Constructive Algorithm for Decomposing a Tensor into a Finite Sum of Orthonormal Rank-1 Terms
We propose a constructive algorithm that decomposes an arbitrary real tensor into a finite sum of orthonormal rank-1 outer products.
Numerical Analysis Numerical Analysis
