Search Results for author: Ayan Chakrabarti

Found 38 papers, 18 papers with code

SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection

no code implementations24 Jan 2024 Ke Ye, Heinrich Jiang, Afshin Rostamizadeh, Ayan Chakrabarti, Giulia Desalvo, Jean-François Kagy, Lazaros Karydas, Gui Citovsky, Sanjiv Kumar

In this paper, we present SpacTor, a new training procedure consisting of (1) a hybrid objective combining span corruption (SC) and token replacement detection (RTD), and (2) a two-stage curriculum that optimizes the hybrid objective over the initial $\tau$ iterations, then transitions to standard SC loss.

Rethinking FID: Towards a Better Evaluation Metric for Image Generation

2 code implementations30 Nov 2023 Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, Sanjiv Kumar

It is an unbiased estimator that does not make any assumptions on the probability distribution of the embeddings and is sample efficient.

Image Generation

Substance or Style: What Does Your Image Embedding Know?

no code implementations10 Jul 2023 Cyrus Rashtchian, Charles Herrmann, Chun-Sung Ferng, Ayan Chakrabarti, Dilip Krishnan, Deqing Sun, Da-Cheng Juan, Andrew Tomkins

We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).

Style Transfer

Benchmarking Robustness to Adversarial Image Obfuscations

1 code implementation NeurIPS 2023 Florian Stimberg, Ayan Chakrabarti, Chun-Ta Lu, Hussein Hazimeh, Otilia Stretcu, Wei Qiao, Yintao Liu, Merve Kaya, Cyrus Rashtchian, Ariel Fuxman, Mehmet Tek, Sven Gowal

We evaluate 33 pretrained models on the benchmark and train models with different augmentations, architectures and training methods on subsets of the obfuscations to measure generalization.

Benchmarking

Adaptive Edge Offloading for Image Classification Under Rate Limit

1 code implementation31 Jul 2022 Jiaming Qiu, Ruiqi Wang, Ayan Chakrabarti, Roch Guerin, Chenyang Lu

Because of limited computing capacity, embedded devices rely on a parsimonious classification model with uneven accuracy.

Classification Image Classification

PROVES: Establishing Image Provenance using Semantic Signatures

1 code implementation21 Oct 2021 Mingyang Xie, Manav Kulshrestha, Shaojie Wang, Jinghan Yang, Ayan Chakrabarti, Ning Zhang, Yevgeniy Vorobeychik

Modern AI tools, such as generative adversarial networks, have transformed our ability to create and modify visual data with photorealistic results.

Face Verification

Leveraging redundancy in attention with Reuse Transformers

1 code implementation13 Oct 2021 Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit, Michal Lukasik, Himanshu Jain, Frederick Liu, Yin-Wen Chang, Sanjiv Kumar

Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications in language and vision.

Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation

no code implementations16 Jun 2021 Srinadh Bhojanapalli, Ayan Chakrabarti, Himanshu Jain, Sanjiv Kumar, Michal Lukasik, Andreas Veit

State-of-the-art transformer models use pairwise dot-product based self-attention, which comes at a computational cost quadratic in the input sequence length.

Understanding Robustness of Transformers for Image Classification

no code implementations ICCV 2021 Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, Andreas Veit

We find that when pre-trained with a sufficient amount of data, ViT models are at least as robust as the ResNet counterparts on a broad range of perturbations.

Classification General Classification +1

Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments

no code implementations CVPR 2021 Zhihao Xia, Michaël Gharbi, Federico Perazzi, Kalyan Sunkavalli, Ayan Chakrabarti

We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments.

Denoising

Real-Time Edge Classification: Optimal Offloading under Token Bucket Constraints

1 code implementation26 Oct 2020 Ayan Chakrabarti, Roch Guérin, Chenyang Lu, Jiangnan Liu

To deploy machine learning-based algorithms for real-time applications with strict latency constraints, we consider an edge-computing setting where a subset of inputs are offloaded to the edge for processing by an accurate but resource-intensive model, and the rest are processed only by a less-accurate model on the device itself.

Classification Edge Classification +3

Finding Physical Adversarial Examples for Autonomous Driving with Fast and Differentiable Image Compositing

1 code implementation17 Oct 2020 Jinghan Yang, Adith Boloor, Ayan Chakrabarti, Xuan Zhang, Yevgeniy Vorobeychik

We propose a scalable approach for finding adversarial modifications of a simulated autonomous driving environment using a differentiable approximation for the mapping from environmental modifications (rectangles on the road) to the corresponding video inputs to the controller neural network.

Autonomous Driving Bayesian Optimization +2

Adversarial Robustness of Deep Sensor Fusion Models

no code implementations23 Jun 2020 Shaojie Wang, Tong Wu, Ayan Chakrabarti, Yevgeniy Vorobeychik

First, we find that the fusion model is usually both more accurate, and more robust against single-source attacks than single-sensor deep neural networks.

Adversarial Robustness Autonomous Driving +4

Towards a MEMS-based Adaptive LIDAR

no code implementations21 Mar 2020 Francesco Pittaluga, Zaid Tasneem, Justin Folden, Brevin Tilmon, Ayan Chakrabarti, Sanjeev J. Koppal

We present a proof-of-concept LIDAR design that allows adaptive real-time measurements according to dynamically specified measurement patterns.

Protecting Geolocation Privacy of Photo Collections

1 code implementation4 Dec 2019 Jinghan Yang, Ayan Chakrabarti, Yevgeniy Vorobeychik

We study this problem formally as a combinatorial optimization problem in the context of geolocation prediction facilitated by deep learning.

Combinatorial Optimization

Training Image Estimators without Image Ground Truth

1 code implementation NeurIPS 2019 Zhihao Xia, Ayan Chakrabarti

Deep neural networks have been very successful in compressive-sensing and image restoration applications, as a means to estimate images from partial, blurry, or otherwise degraded measurements.

Compressive Sensing Image Restoration

Neural Network-Inspired Analog-to-Digital Conversion to Achieve Super-Resolution with Low-Precision RRAM Devices

no code implementations28 Nov 2019 Weidong Cao, Liu Ke, Ayan Chakrabarti, Xuan Zhang

Recent works propose neural network- (NN-) inspired analog-to-digital converters (NNADCs) and demonstrate their great potentials in many emerging applications.

Quantization Robust Design +1

Generating and Exploiting Probabilistic Monocular Depth Estimates

1 code implementation CVPR 2020 Zhihao Xia, Patrick Sullivan, Ayan Chakrabarti

Beyond depth estimation from a single image, the monocular cue is useful in a broader range of depth inference applications and settings---such as when one can leverage other available depth cues for improved accuracy.

Depth Completion Monocular Depth Estimation

Training Image Estimators without Image Ground-Truth

1 code implementation13 Jun 2019 Zhihao Xia, Ayan Chakrabarti

We evaluate our method for training networks for compressive-sensing and blind deconvolution, considering both non-blind and blind training for the latter.

Compressive Sensing Image Restoration

Fast Deep Stereo with 2D Convolutional Processing of Cost Signatures

1 code implementation8 Mar 2019 Kyle Yee, Ayan Chakrabarti

Modern neural network-based algorithms are able to produce highly accurate depth estimates from stereo image pairs, nearly matching the reliability of measurements from more expensive depth sensors.

Autonomous Vehicles

Backprop with Approximate Activations for Memory-efficient Network Training

1 code implementation ICLR 2019 Ayan Chakrabarti, Benjamin Moseley

Training convolutional neural network models is memory intensive since back-propagation requires storing activations of all intermediate layers.

Learning to Separate Multiple Illuminants in a Single Image

no code implementations CVPR 2019 Zhuo Hui, Ayan Chakrabarti, Kalyan Sunkavalli, Aswin C. Sankaranarayanan

We present a method to separate a single image captured under two illuminants, with different spectra, into the two images corresponding to the appearance of the scene under each individual illuminant.

Identifying Recurring Patterns with Deep Neural Networks for Natural Image Denoising

1 code implementation13 Jun 2018 Zhihao Xia, Ayan Chakrabarti

In this work, we propose a new method for natural image denoising that trains a deep neural network to determine whether patches in a noisy image input share common underlying patterns.

Color Image Denoising Image Denoising +1

Learning Privacy Preserving Encodings through Adversarial Training

no code implementations14 Feb 2018 Francesco Pittaluga, Sanjeev J. Koppal, Ayan Chakrabarti

We present a framework to learn privacy-preserving encodings of images that inhibit inference of chosen private attributes, while allowing recovery of other desirable information.

Attribute Privacy Preserving

Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning

3 code implementations ICLR 2018 Charles Schaff, David Yunis, Ayan Chakrabarti, Matthew R. Walter

The physical design of a robot and the policy that controls its motion are inherently coupled, and should be determined according to the task and environment.

reinforcement-learning Reinforcement Learning (RL)

Stabilizing GAN Training with Multiple Random Projections

2 code implementations ICLR 2018 Behnam Neyshabur, Srinadh Bhojanapalli, Ayan Chakrabarti

Training generative adversarial networks is unstable in high-dimensions as the true data distribution tends to be concentrated in a small fraction of the ambient space.

Jointly Optimizing Placement and Inference for Beacon-based Localization

1 code implementation24 Mar 2017 Charles Schaff, David Yunis, Ayan Chakrabarti, Matthew R. Walter

The accuracy of such a beacon-based localization system depends both on how beacons are distributed in the environment, and how the robot's location is inferred based on noisy and potentially ambiguous measurements.

Examining the Impact of Blur on Recognition by Convolutional Networks

no code implementations17 Nov 2016 Igor Vasiljevic, Ayan Chakrabarti, Gregory Shakhnarovich

We investigate the extent to which this degradation is due to the mismatch between training and input image statistics.

Single-image RGB Photometric Stereo With Spatially-varying Albedo

no code implementations14 Sep 2016 Ayan Chakrabarti, Kalyan Sunkavalli

We present a single-shot system to recover surface geometry of objects with spatially-varying albedos, from images captured under a calibrated RGB photometric stereo setup---with three light directions multiplexed across different color channels in the observed RGB image.

Learning Sensor Multiplexing Design through Back-propagation

no code implementations NeurIPS 2016 Ayan Chakrabarti

In this paper, we specifically consider the design and inference problems in a typical color camera---where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image.

A Neural Approach to Blind Motion Deblurring

2 code implementations15 Mar 2016 Ayan Chakrabarti

We present a new method for blind motion deblurring that uses a neural network trained to compute estimates of sharp image patches from observations that are blurred by an unknown motion kernel.

Deblurring

Color Constancy by Learning to Predict Chromaticity from Luminance

no code implementations NeurIPS 2015 Ayan Chakrabarti

Specifically, we describe an illuminant estimation method that is built around a "classifier" for identifying the true chromaticity of a pixel given its luminance (absolute brightness across color channels).

Color Constancy

Low-level Vision by Consensus in a Spatial Hierarchy of Regions

no code implementations CVPR 2015 Ayan Chakrabarti, Ying Xiong, Steven J. Gortler, Todd Zickler

We introduce a multi-scale framework for low-level vision, where the goal is estimating physical scene values from image data---such as depth from stereo image pairs.

valid

Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images

no code implementations27 Nov 2013 Ayan Chakrabarti, Ying Xiong, Baochen Sun, Trevor Darrell, Daniel Scharstein, Todd Zickler, Kate Saenko

To produce images that are suitable for display, tone-mapping is widely used in digital cameras to map linear color measurements into narrow gamuts with limited dynamic range.

Tone Mapping

From Shading to Local Shape

no code implementations10 Oct 2013 Ying Xiong, Ayan Chakrabarti, Ronen Basri, Steven J. Gortler, David W. Jacobs, Todd Zickler

We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch.

Surface Reconstruction

Cannot find the paper you are looking for? You can Submit a new open access paper.