no code implementations • 24 Feb 2024 • Neal Mangaokar, Ashish Hooda, Jihye Choi, Shreyas Chandrashekaran, Kassem Fawaz, Somesh Jha, Atul Prakash
More recent LLMs often incorporate an additional layer of defense, a Guard Model, which is a second LLM that is designed to check and moderate the output response of the primary LLM.
no code implementations • 19 Feb 2024 • Shuowei Jin, Yongji Wu, Haizhong Zheng, Qingzhao Zhang, Matthew Lentz, Z. Morley Mao, Atul Prakash, Feng Qian, Danyang Zhuo
Large language models (LLMs) have seen significant adoption for natural language tasks, owing their success to massive numbers of model parameters (e. g., 70B+); however, LLM inference incurs significant computation and memory costs.
no code implementations • 9 Feb 2024 • Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash
The emergence of activation sparsity in LLMs provides a natural approach to reduce this cost by involving only parts of the parameters for inference.
no code implementations • 11 Oct 2023 • Haizhong Zheng, Jiachen Sun, Shutong Wu, Bhavya Kailkhura, Zhuoqing Mao, Chaowei Xiao, Atul Prakash
In this paper, we recognize that images share common features in a hierarchical way due to the inherent hierarchical structure of the classification system, which is overlooked by current data parameterization methods.
no code implementations • 30 Jul 2023 • Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash
This work aims to address this gap by offering a theoretical characterization of the trade-off between detection and false positive rates for stateful defenses.
no code implementations • 1 Jun 2023 • Jiachen Sun, Haizhong Zheng, Qingzhao Zhang, Atul Prakash, Z. Morley Mao, Chaowei Xiao
CALICO's efficacy is substantiated by extensive evaluations on 3D object detection and BEV map segmentation tasks, where it delivers significant performance improvements.
1 code implementation • 11 Mar 2023 • Ryan Feng, Ashish Hooda, Neal Mangaokar, Kassem Fawaz, Somesh Jha, Atul Prakash
Such stateful defenses aim to defend against black-box attacks by tracking the query history and detecting and rejecting queries that are "similar" and thus preventing black-box attacks from finding useful gradients and making progress towards finding adversarial attacks within a reasonable query budget.
1 code implementation • 28 Oct 2022 • Haizhong Zheng, Rui Liu, Fan Lai, Atul Prakash
We then propose a novel one-shot coreset selection method, Coverage-centric Coreset Selection (CCS), that jointly considers overall data coverage upon a distribution as well as the importance of each example.
no code implementations • 18 May 2022 • Ryan Feng, Somesh Jha, Atul Prakash
Preprocessing and outlier detection techniques have both been applied to neural networks to increase robustness with varying degrees of success.
1 code implementation • 4 Mar 2022 • Jihye Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, Atul Prakash
Based on these metrics, we propose an unsupervised framework for learning a set of concepts that satisfy the desired properties of high detection completeness and concept separability, and demonstrate its effectiveness in providing concept-based explanations for diverse off-the-shelf OOD detectors.
no code implementations • 11 Feb 2022 • Ashish Hooda, Neal Mangaokar, Ryan Feng, Kassem Fawaz, Somesh Jha, Atul Prakash
D4 uses an ensemble of models over disjoint subsets of the frequency spectrum to significantly improve adversarial robustness.
no code implementations • 29 Sep 2021 • Tianji Cong, Atul Prakash
The problem of detecting out-of-distribution (OOD) examples in neural networks has been widely studied in the literature, with state-of-the-art techniques being supervised in that they require fine-tuning on OOD data to achieve high-quality OOD detection.
Out-of-Distribution Detection Out of Distribution (OOD) Detection
no code implementations • ICML Workshop AML 2021 • Nelson Manohar-Alers, Ryan Feng, Sahib Singh, Jiguo Song, Atul Prakash
We present DeClaW, a system for detecting, classifying, and warning of adversarial inputs presented to a classification neural network.
no code implementations • ICLR 2021 • Sanjay Kariyappa, Atul Prakash, Moinuddin K Qureshi
EDM is made up of models that are trained to produce dissimilar predictions for OOD inputs.
no code implementations • 3 Dec 2020 • Ryan Feng, Wu-chi Feng, Atul Prakash
We first formally prove that adaptive codebooks can provide stronger robustness guarantees than fixed codebooks as a preprocessing defense on some datasets.
no code implementations • 17 Jul 2020 • Haizhong Zheng, Ziqi Zhang, Honglak Lee, Atul Prakash
Moreover, we design the first diagnostic method to quantify the vulnerability contributed by each layer, which can be used to identify vulnerable parts of model architectures.
no code implementations • 8 May 2020 • Liang Tong, Minzhe Guo, Atul Prakash, Yevgeniy Vorobeychik
We then experimentally demonstrate that our attacks indeed do not significantly change perceptual salience of the background, but are highly effective against classifiers robust to conventional attacks.
1 code implementation • CVPR 2021 • Sanjay Kariyappa, Atul Prakash, Moinuddin Qureshi
The effectiveness of such attacks relies heavily on the availability of data necessary to query the target model.
1 code implementation • 17 Feb 2020 • Ryan Feng, Neal Mangaokar, Jiefeng Chen, Earlence Fernandes, Somesh Jha, Atul Prakash
We address three key requirements for practical attacks for the real-world: 1) automatically constraining the size and shape of the attack so it can be applied with stickers, 2) transform-robustness, i. e., robustness of a attack to environmental physical variations such as viewpoint and lighting changes, and 3) supporting attacks in not only white-box, but also black-box hard-label scenarios, so that the adversary can attack proprietary models.
2 code implementations • CVPR 2020 • Haizhong Zheng, Ziqi Zhang, Juncheng Gu, Honglak Lee, Atul Prakash
Adversarial training is an effective defense method to protect classification models against adversarial attacks.
no code implementations • 27 Nov 2019 • Pratik Vaishnavi, Tianji Cong, Kevin Eykholt, Atul Prakash, Amir Rahmati
Focusing on the observation that discrete pixelization in MNIST makes the background completely black and foreground completely white, we hypothesize that the important property for increasing robustness is the elimination of image background using attention masks before classifying an object.
no code implementations • 12 Sep 2019 • Pratik Vaishnavi, Kevin Eykholt, Atul Prakash, Amir Rahmati
Numerous techniques have been proposed to harden machine learning algorithms and mitigate the effect of adversarial attacks.
no code implementations • 27 May 2019 • Haizhong Zheng, Earlence Fernandes, Atul Prakash
Recently, interpretable models called self-explaining models (SEMs) have been proposed with the goal of providing interpretability robustness.
no code implementations • 26 May 2019 • Kevin Eykholt, Swati Gupta, Atul Prakash, Amir Rahmati, Pratik Vaishnavi, Haizhong Zheng
Existing deep neural networks, say for image classification, have been shown to be vulnerable to adversarial images that can cause a DNN misclassification, without any perceptible change to an image.
no code implementations • 17 Dec 2018 • Kevin Eykholt, Atul Prakash
We provide a methodology, resilient feature engineering, for creating adversarially resilient classifiers.
no code implementations • 20 Jul 2018 • Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, Dawn Song
In this work, we extend physical attacks to more challenging object detection models, a broader class of deep learning algorithms widely used to detect and label multiple objects within a scene.
no code implementations • CVPR 2018 • Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song
Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input.
no code implementations • 14 Jan 2018 • Amir Rahmati, Earlence Fernandes, Kevin Eykholt, Atul Prakash
When using risk-based permissions, device operations are grouped into units of similar risk, and users grant apps access to devices at that risk-based granularity.
Cryptography and Security
no code implementations • 21 Dec 2017 • Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Dawn Song, Tadayoshi Kohno, Amir Rahmati, Atul Prakash, Florian Tramer
Given the fact that state-of-the-art objection detection algorithms are harder to be fooled by the same set of adversarial examples, here we show that these detectors can also be attacked by physical adversarial examples.
1 code implementation • 27 Jul 2017 • Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song
We propose a general attack algorithm, Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions.
no code implementations • 23 May 2017 • Earlence Fernandes, Amir Rahmati, Kevin Eykholt, Atul Prakash
The Internet of Things (IoT) is a new computing paradigm that spans wearable devices, homes, hospitals, cities, transportation, and critical infrastructure.
Cryptography and Security