Search Results for author:
Found 24 papers, 7 papers with code
TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models
To guide video generation with the additional image input, we propose a "repeat-and-slide" strategy that modulates the reverse denoising process, allowing the frozen diffusion model to synthesize a video frame-by-frame starting from the provided image.
Multimodal 3D Object Detection on Unseen Domains
To this end, we propose CLIX$^\text{3D}$, a multimodal fusion and supervised contrastive learning framework for 3D object detection that performs alignment of object features from same-class samples of different domains while pushing the features from different classes apart.
Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection
This can enable improved performance in downstream tasks that are equivariant to such transformations.
G-RepsNet: A Fast and General Construction of Equivariant Networks for Arbitrary Matrix Groups
Recent work by Finzi et al. (2021) directly solves the equivariance constraint for arbitrary matrix groups to obtain equivariant MLPs (EMLPs).
Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis
To this end, and capitalizing on the powerful fine-grained generative control offered by the recent diffusion-based generative models, we introduce Steered Diffusion, a generalized framework for photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation.
Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection
We first pretrain these factor matrices on the RGB modality, for which plenty of training data are assumed to exist and then augment only a few trainable parameters for training on the IR modality to avoid over-fitting, while encouraging them to capture complementary cues from those trained only on the RGB modality.
Pixel-Grounded Prototypical Part Networks
Prototypical part neural networks (ProtoPartNNs), namely PROTOPNET and its derivatives, are an intrinsically interpretable approach to machine learning.
Are Deep Neural Networks SMARTer than Second Graders?
To answer this question, we propose SMART: a Simple Multimodal Algorithmic Reasoning Task and the associated SMART-101 dataset, for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed specifically for children in the 6--8 age group.
Cross-Modal Knowledge Transfer Without Task-Relevant Source Data
In such cases, transferring knowledge from a neural network trained on a well-labeled large dataset in the source modality (RGB) to a neural network that works on a target modality (depth, infrared, etc.)
Learning Partial Equivariances from Data
Frequently, transformations occurring in data can be better represented by a subset of a group than by a group as a whole, e. g., rotations in $[-90^{\circ}, 90^{\circ}]$.
Understanding the Success of Knowledge Distillation -- A Data Augmentation Perspective
We achieve new state-of-the-art accuracy by using the original KD loss armed with stronger augmentation schemes, compared to existing state-of-the-art methods that employ more advanced distillation losses.
Rotation-Invariant Autoencoders for Signals on Spheres
These newly proposed convolutional layers naturally extend the notion of convolution to functions on the unit sphere $S^2$ and the group of rotations $SO(3)$ and these layers are equivariant to 3D rotations.
Model Compression Using Optimal Transport
Model compression methods are important to allow for easier deployment of deep learning models in compute, memory and energy-constrained environments such as mobile phones.
Multi-head Knowledge Distillation for Model Compression
We add loss terms for training the student that measure the dissimilarity between student and teacher outputs of the auxiliary classifiers.
What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective
What makes a "good" DA in KD?
Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization
Motion capture (mocap) and time-of-flight based sensing of human actions are becoming increasingly popular modalities to perform robust activity analysis.
Generative Patch Priors for Practical Compressive Image Recovery
In this paper, we propose the generative patch prior (GPP) that defines a generative prior for compressive image recovery, based on patch-manifold models.
Temporal Transformer Networks: Joint Learning of Invariant and Discriminative Time Warping
We call this a temporal transformer network (TTN).
Rate-Adaptive Neural Networks for Spatial Multiplexers
Using standard datasets, we demonstrate that, when tested over a range of MRs, a rate-adaptive network can provide high quality reconstruction over a the entire range, resulting in up to about 15 dB improvement over previous methods, where the network is valid for only one MR. We demonstrate the effectiveness of our approach for sample-efficient object tracking where video frames are acquired at dynamically varying MRs. We also extend this algorithm to learn the measurement operator in conjunction with image recognition networks.
CS-VQA: Visual Question Answering with Compressively Sensed Images
Visual Question Answering (VQA) is a complex semantic task requiring both natural language processing and visual recognition.
Learning Invariant Riemannian Geometric Representations Using Deep Nets
Non-Euclidean constraints are inherent in many kinds of data in computer vision and machine learning, typically as a result of specific invariance requirements that need to be respected during high-level inference.
Convolutional Neural Networks for Non-iterative Reconstruction of Compressively Sensed Images
We show empirically that our algorithm yields reconstructions with higher PSNRs compared to iterative algorithms at low measurement rates and in presence of measurement noise.
ReconNet: Non-Iterative Reconstruction of Images From Compressively Sensed Measurements
The intermediate reconstruction is fed into an off-the-shelf denoiser to obtain the final reconstructed image.
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements
The intermediate reconstruction is fed into an off-the-shelf denoiser to obtain the final reconstructed image.
