no code implementations • ICML 2020 • Yihan Wang, huan zhang, Hongge Chen, Duane Boning, Cho-Jui Hsieh
In this paper, we study the robustness verification and defense with respect to general $\ell_p$ norm perturbation for ensemble trees and stumps.
no code implementations • Findings (ACL) 2022 • Jianhan Xu, Cenyuan Zhang, Xiaoqing Zheng, Linyang Li, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
no code implementations • EMNLP 2021 • Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
no code implementations • 26 Apr 2024 • Yue Kang, Cho-Jui Hsieh, Thomas C. M. Lee
By utilizing the truncation on observed payoffs and the dynamic exploration, we propose a novel algorithm called LOTUS attaining the regret bound of order $\tilde O(d^\frac{3}{2}r^\frac{1}{2}T^\frac{1}{1+\delta}/\tilde{D}_{rr})$ without knowing $T$, which matches the state-of-the-art regret bound under sub-Gaussian noises~\citep{lu2021low, kang2022efficient} with $\delta = 1$.
1 code implementation • 11 Apr 2024 • Lujie Yang, Hongkai Dai, Zhouxing Shi, Cho-Jui Hsieh, Russ Tedrake, huan zhang
The flexibility and efficiency of our framework allow us to demonstrate Lyapunov-stable output feedback control with synthesized NN-based controllers and NN-based observers with formal stability guarantees, for the first time in literature.
1 code implementation • 26 Feb 2024 • Yihan Wang, Zhouxing Shi, Andrew Bai, Cho-Jui Hsieh
The inferred prompt is called the backtranslated prompt which tends to reveal the actual intent of the original prompt, since it is generated based on the LLM's response and is not directly manipulated by the attacker.
1 code implementation • 25 Feb 2024 • Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, Cho-Jui Hsieh
DrAttack includes three key components: (a) `Decomposition' of the original prompt into sub-prompts, (b) `Reconstruction' of these sub-prompts implicitly by in-context learning with semantically similar but harmless reassembling demo, and (c) a `Synonym Search' of sub-prompts, aiming to find sub-prompts' synonyms that maintain the original intent while jailbreaking LLMs.
no code implementations • 24 Feb 2024 • Yong liu, Zirui Zhu, Chaoyu Gong, Minhao Cheng, Cho-Jui Hsieh, Yang You
While fine-tuning large language models (LLMs) for specific tasks often yields impressive results, it comes at the cost of memory inefficiency due to back-propagation in gradient-based training.
1 code implementation • 20 Feb 2024 • Sen Li, Ruochen Wang, Cho-Jui Hsieh, Minhao Cheng, Tianyi Zhou
Moreover, MuLan adopts a vision-language model (VLM) to provide feedback to the image generated in each sub-task and control the diffusion model to re-generate the image if it violates the original prompt.
no code implementations • 12 Feb 2024 • Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh, Ankur Taly
We propose a novel sampling scheme, mix-cd, that identifies and prioritizes samples that actually face forgetting, which we call collateral damage.
1 code implementation • 1 Feb 2024 • Chia-Cheng Chiang, Li-Cheng Lan, Wei-Fang Sun, Chien Feng, Cho-Jui Hsieh, Chun-Yi Lee
In this paper, we focus on single-demonstration imitation learning (IL), a practical approach for real-world applications where obtaining numerous expert demonstrations is costly or infeasible.
no code implementations • 17 Jan 2024 • Tong Xie, Haoyu Li, Andrew Bai, Cho-Jui Hsieh
Data attribution methods trace model behavior back to its training dataset, offering an effective approach to better understand ''black-box'' neural networks.
no code implementations • 14 Jan 2024 • Yue Kang, Cho-Jui Hsieh, Thomas C. M. Lee
In the stochastic contextual low-rank matrix bandit problem, the expected reward of an action is given by the inner product between the action's feature matrix and some fixed, but initially unknown $d_1$ by $d_2$ matrix $\Theta^*$ with rank $r \ll \{d_1, d_2\}$, and an agent sequentially takes actions based on past experience to maximize the cumulative reward.
no code implementations • 22 Dec 2023 • Tiejin Chen, Yuanpu Cao, Yujia Wang, Cho-Jui Hsieh, Jinghui Chen
Specifically, FedPTR allows local clients or the server to optimize an auxiliary (synthetic) dataset that mimics the learning dynamics of the recent model update and utilizes it to project the next-step model trajectory for local training regularization.
1 code implementation • 5 Dec 2023 • Wei-Cheng Chang, Jyun-Yu Jiang, Jiong Zhang, Mutasem Al-Darabsah, Choon Hui Teo, Cho-Jui Hsieh, Hsiang-Fu Yu, S. V. N. Vishwanathan
For product search, PEFA improves the Recall@100 of the fine-tuned ERMs by an average of 5. 3% and 14. 5%, for PEFA-XS and PEFA-XL, respectively.
no code implementations • 16 Nov 2023 • Cho-Jui Hsieh, Si Si, Felix X. Yu, Inderjit S. Dhillon
Large language models (LLMs) have demonstrated remarkable capabilities in solving complex open-domain tasks, guided by comprehensive instructions and demonstrations provided in the form of prompts.
no code implementations • 16 Nov 2023 • Yuhang Li, Yihan Wang, Zhouxing Shi, Cho-Jui Hsieh
In this work, we propose to improve the quality of texts generated by a watermarked language model by Watermarking with Importance Scoring (WIS).
no code implementations • 18 Oct 2023 • Liu Liu, Xuanqing Liu, Cho-Jui Hsieh, DaCheng Tao
In this paper, we explore a family of stochastic TR and ARC methods that can simultaneously provide inexact computations of the Hessian matrix, gradient, and function values.
1 code implementation • 14 Oct 2023 • Lucas Tecot, Cho-Jui Hsieh
To address this concern, we compare the performance of classical optimizers across a series of partially-randomized tasks to more broadly sample the space of quantum optimization problems.
no code implementations • 8 Oct 2023 • Xiusi Chen, Jyun-Yu Jiang, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Wei Wang
Few-shot question answering (QA) aims at achieving satisfactory results on machine question answering when only a few training samples are available.
2 code implementations • 31 May 2023 • Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, Cho-Jui Hsieh
The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users.
no code implementations • 31 May 2023 • Che-Ping Tsai, Jiong Zhang, Eli Chien, Hsiang-Fu Yu, Cho-Jui Hsieh, Pradeep Ravikumar
We introduce a novel class of sample-based explanations we term high-dimensional representers, that can be used to explain the predictions of a regularized high-dimensional model in terms of importance weights for each of the training samples.
1 code implementation • 21 May 2023 • Eli Chien, Jiong Zhang, Cho-Jui Hsieh, Jyun-Yu Jiang, Wei-Cheng Chang, Olgica Milenkovic, Hsiang-Fu Yu
Unlike most existing XMC frameworks that treat labels and input instances as featureless indicators and independent entries, PINA extracts information from the label metadata and the correlations among training instances.
no code implementations • 26 Apr 2023 • Li-Cheng Lan, huan zhang, Cho-Jui Hsieh
With extensive experimental evaluation, we show the prevalence of \emph{generalization failure} on controllable states from stranger agents.
no code implementations • 28 Mar 2023 • Yuanhao Xiong, Long Zhao, Boqing Gong, Ming-Hsuan Yang, Florian Schroff, Ting Liu, Cho-Jui Hsieh, Liangzhe Yuan
Existing video-language pre-training methods primarily focus on instance-level alignment between video clips and captions via global contrastive learning but neglect rich fine-grained local information in both videos and text, which is of importance to downstream tasks requiring temporal localization and semantic reasoning.
no code implementations • 18 Feb 2023 • Yue Kang, Cho-Jui Hsieh, Thomas C. M. Lee
In stochastic contextual bandits, an agent sequentially makes actions from a time-dependent action set based on past experience to minimize the cumulative regret.
no code implementations • NeurIPS 2023 • Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, Quoc V. Le
On diffusion models, Lion outperforms Adam by achieving a better FID score and reducing the training compute by up to 2. 3x.
2 code implementations • 19 Nov 2022 • Justin Cui, Ruochen Wang, Si Si, Cho-Jui Hsieh
The resulting algorithm sets new SOTA on ImageNet-1K: we can scale up to 50 IPCs (Image Per Class) on ImageNet-1K on a single GPU (all previous methods can only scale to 2 IPCs on ImageNet-1K), leading to the best accuracy (only 5. 9% accuracy drop against full dataset training) while utilizing only 4. 2% of the number of data points - an 18. 2% absolute gain over prior SOTA.
1 code implementation • 7 Nov 2022 • Li-Cheng Lan, huan zhang, Ti-Rong Wu, Meng-Yu Tsai, I-Chen Wu, Cho-Jui Hsieh
Given that the state space of Go is extremely large and a human player can play the game from any legal state, we ask whether adversarial states exist for Go AIs that may lead them to play surprisingly wrong actions.
no code implementations • 4 Nov 2022 • Anaelia Ovalle, Evan Czyzycki, Cho-Jui Hsieh
Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks.
no code implementations • 1 Nov 2022 • Yihan Wang, Si Si, Daliang Li, Michal Lukasik, Felix Yu, Cho-Jui Hsieh, Inderjit S Dhillon, Sanjiv Kumar
Pretrained large language models (LLMs) are general purpose problem solvers applicable to a diverse set of tasks with prompts.
1 code implementation • 22 Oct 2022 • Fan Yin, Yao Li, Cho-Jui Hsieh, Kai-Wei Chang
Finally, our analysis shows that the two types of uncertainty provided by \textbf{ADDMU} can be leveraged to characterize adversarial examples and identify the ones that contribute most to model's robustness in adversarial training.
1 code implementation • 21 Oct 2022 • Andrew Bai, Cho-Jui Hsieh, Wendy Kan, Hsuan-Tien Lin
In this paper, we propose memorization rejection, a training scheme that rejects generated samples that are near-duplicates of training samples during training.
no code implementations • 18 Oct 2022 • Jyun-Yu Jiang, Wei-Cheng Chang, Jiong Zhong, Cho-Jui Hsieh, Hsiang-Fu Yu
Uncertainty quantification is one of the most crucial tasks to obtain trustworthy and reliable machine learning models for decision making.
1 code implementation • 16 Oct 2022 • Nilesh Gupta, Patrick H. Chen, Hsiang-Fu Yu, Cho-Jui Hsieh, Inderjit S Dhillon
A popular approach for dealing with the large label space is to arrange the labels into a shallow tree-based index and then learn an ML model to efficiently search this index via beam search.
no code implementations • 14 Oct 2022 • Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh
Large pre-trained language models (PLMs) have proven to be a crucial component of modern natural language processing systems.
2 code implementations • 13 Oct 2022 • Zhouxing Shi, Yihan Wang, huan zhang, Zico Kolter, Cho-Jui Hsieh
In this paper, we develop an efficient framework for computing the $\ell_\infty$ local Lipschitz constant of a neural network by tightly upper bounding the norm of Clarke Jacobian via linear bound propagation.
1 code implementation • 27 Sep 2022 • Ruochen Wang, Yuanhao Xiong, Minhao Cheng, Cho-Jui Hsieh
Efficient and automated design of optimizers plays a crucial role in full-stack AutoML systems.
1 code implementation • 31 Aug 2022 • Andrew Bai, Chih-Kuan Yeh, Pradeep Ravikumar, Neil Y. C. Lin, Cho-Jui Hsieh
We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space.
2 code implementations • 11 Aug 2022 • huan zhang, Shiqi Wang, Kaidi Xu, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, J. Zico Kolter
Our generalized bound propagation method, GCP-CROWN, opens up the opportunity to apply general cutting plane methods for neural network verification while benefiting from the efficiency and GPU acceleration of bound propagation methods.
2 code implementations • 20 Jul 2022 • Justin Cui, Ruochen Wang, Si Si, Cho-Jui Hsieh
Dataset Condensation is a newly emerging technique aiming at learning a tiny dataset that captures the rich information encoded in the original dataset.
1 code implementation • CVPR 2023 • Yuanhao Xiong, Ruochen Wang, Minhao Cheng, Felix Yu, Cho-Jui Hsieh
Federated learning~(FL) has recently attracted increasing attention from academia and industry, with the ultimate goal of achieving collaborative training under privacy and communication constraints.
1 code implementation • Findings (ACL) 2022 • Cenyuan Zhang, Xiang Zhou, Yixin Wan, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh
Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models.
1 code implementation • ICLR 2022 • Shoukang Hu, Ruochen Wang, Lanqing Hong, Zhenguo Li, Cho-Jui Hsieh, Jiashi Feng
Efficient performance estimation of architectures drawn from large search spaces is essential to Neural Architecture Search.
no code implementations • ICLR 2022 • Yihan Wang, Zhouxing Shi, Quanquan Gu, Cho-Jui Hsieh
Interval Bound Propagation (IBP) is so far the base of state-of-the-art methods for training neural networks with certifiable robustness guarantees when potential adversarial perturbations present, while the convergence of IBP training remains unknown in existing literature.
2 code implementations • CVPR 2022 • Yong liu, Siqi Mai, Xiangning Chen, Cho-Jui Hsieh, Yang You
Recently, Sharpness-Aware Minimization (SAM), which connects the geometry of the loss landscape and generalization, has demonstrated significant performance boosts on training large-scale models such as vision transformers.
1 code implementation • NAACL 2022 • Yuanhao Xiong, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Inderjit Dhillon
To learn the semantic embeddings of instances and labels with raw text, we propose to pre-train Transformer-based encoders with self-supervised contrastive losses.
Multi Label Text Classification Multi-Label Text Classification +2
1 code implementation • 15 Dec 2021 • Jaehui Hwang, huan zhang, Jun-Ho Choi, Cho-Jui Hsieh, Jong-Seok Lee
Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance.
no code implementations • NeurIPS 2021 • Pei-Hung Chen, Hsiang-Fu Yu, Inderjit Dhillon, Cho-Jui Hsieh
In addition to compressing standard models, out method can also be used on distilled BERT models to further improve compression rate.
1 code implementation • 18 Nov 2021 • Yao Li, Minhao Cheng, Cho-Jui Hsieh, Thomas C. M. Lee
Despite the efficiency and scalability of machine learning systems, recent studies have demonstrated that many classification methods, especially deep neural networks (DNNs), are vulnerable to adversarial examples; i. e., examples that are carefully crafted to fool a well-trained classification model while being indistinguishable from natural data to human.
no code implementations • 2 Nov 2021 • Shanda Li, Xiangning Chen, Di He, Cho-Jui Hsieh
Several recent studies have demonstrated that attention-based networks, such as Vision Transformer (ViT), can outperform Convolutional Neural Networks (CNNs) on several computer vision tasks without using convolutional layers.
4 code implementations • ICLR 2022 • Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon
We also provide a theoretical analysis that justifies the use of XMC over link prediction and motivates integrating XR-Transformers, a powerful method for solving XMC problems, into the GIANT framework.
Ranked #2 on Node Property Prediction on ogbn-papers100M
no code implementations • 22 Oct 2021 • Rulin Shao, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh
Our comprehensive analysis shows several novel insights that (1) With KDIGA, students can preserve or even exceed the adversarial robustness of the teacher model, even when their models have fundamentally different architectures; (2) KDIGA enables robustness to transfer to pre-trained students, such as KD from an adversarially trained ResNet to a pre-trained ViT, without loss of clean accuracy; and (3) Our derived local linearity bounds for characterizing adversarial robustness in KD are consistent with the empirical results.
no code implementations • 13 Oct 2021 • Yunxiao Qin, Yuanhao Xiong, JinFeng Yi, Lihong Cao, Cho-Jui Hsieh
In this paper, we define a Generalized Transferable Attack (GTA) problem where the attacker doesn't know this information and is acquired to attack any randomly encountered images that may come from unknown datasets.
no code implementations • 29 Sep 2021 • Xuanqing Liu, Sara Imboden, Marie Payne, Neil Lin, Cho-Jui Hsieh
In addition, we introduce FastEnsemble, a fast ensemble method which only requires less than $8\%$ of the full-ensemble training time to generate a new ensemble member.
no code implementations • 29 Sep 2021 • huan zhang, Shiqi Wang, Kaidi Xu, Yihan Wang, Suman Jana, Cho-Jui Hsieh, J Zico Kolter
In this work, we formulate an adversarial attack using a branch-and-bound (BaB) procedure on ReLU neural networks and search adversarial examples in the activation space corresponding to binary variables in a mixed integer programming (MIP) formulation.
no code implementations • 29 Sep 2021 • Yong liu, Siqi Mai, Xiangning Chen, Cho-Jui Hsieh, Yang You
Large-batch training is an important direction for distributed machine learning, which can improve the utilization of large-scale clusters and therefore accelerate the training process.
no code implementations • ICLR 2022 • Yuanhao Xiong, Li-Cheng Lan, Xiangning Chen, Ruochen Wang, Cho-Jui Hsieh
By constructing a directed graph for the underlying neural network of the target problem, GNS encodes current dynamics with a graph message passing network and trains an agent to control the learning rate accordingly via reinforcement learning.
2 code implementations • 5 Sep 2021 • Yunxiao Qin, Yuanhao Xiong, JinFeng Yi, Cho-Jui Hsieh
In this paper, we tackle this problem from a novel angle -- instead of using the original surrogate models, can we obtain a Meta-Surrogate Model (MSM) such that attacks to this model can be easier transferred to other models?
1 code implementation • EMNLP 2021 • Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, Cho-Jui Hsieh
Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models.
no code implementations • ICCV 2021 • Ruochen Wang, Xiangning Chen, Minhao Cheng, Xiaocheng Tang, Cho-Jui Hsieh
Predictor-based algorithms have achieved remarkable performance in the Neural Architecture Search (NAS) tasks.
2 code implementations • ICCV 2021 • Yongming Rao, Benlin Liu, Yi Wei, Jiwen Lu, Cho-Jui Hsieh, Jie zhou
In particular, we propose to generate random layouts of a scene by making use of the objects in the synthetic CAD dataset and learn the 3D scene representation by applying object-level contrastive learning on two random scenes generated from the same set of synthetic objects.
1 code implementation • ICLR 2021 • Ruochen Wang, Minhao Cheng, Xiangning Chen, Xiaocheng Tang, Cho-Jui Hsieh
Differentiable Neural Architecture Search is one of the most popular Neural Architecture Search (NAS) methods for its search efficiency and simplicity, accomplished by jointly optimizing the model weight and architecture parameters in a weight-sharing supernet via gradient-based algorithms.
1 code implementation • ACL 2021 • Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.
no code implementations • NeurIPS 2021 • Xuanqing Liu, Wei-Cheng Chang, Hsiang-Fu Yu, Cho-Jui Hsieh, Inderjit S. Dhillon
Partition-based methods are increasingly-used in extreme multi-label classification (XMC) problems due to their scalability to large output spaces (e. g., millions or more).
no code implementations • NeurIPS 2021 • Shiqi Wang, huan zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, J Zico Kolter
We develop $\beta$-CROWN, a new bound propagation based method that can fully encode neuron split constraints in branch-and-bound (BaB) based complete verification via optimizable parameters $\beta$.
1 code implementation • NeurIPS 2021 • Yang Li, Si Si, Gang Li, Cho-Jui Hsieh, Samy Bengio
Attentional mechanisms are order-invariant.
no code implementations • 5 Jun 2021 • Qin Ding, Yue Kang, Yi-Wei Liu, Thomas C. M. Lee, Cho-Jui Hsieh, James Sharpnack
To tackle this problem, we first propose a two-layer bandit structure for auto tuning the exploration parameter and further generalize it to the Syndicated Bandits framework which can learn multiple hyper-parameters dynamically in contextual bandit environment.
no code implementations • 5 Jun 2021 • Qin Ding, Cho-Jui Hsieh, James Sharpnack
We provide theoretical guarantees for our proposed algorithm and show by experiments that our proposed algorithm improves the robustness against various kinds of popular attacks.
2 code implementations • ICLR 2022 • Xiangning Chen, Cho-Jui Hsieh, Boqing Gong
Vision Transformers (ViTs) and MLPs signal further efforts on replacing hand-wired features or inductive biases with general-purpose neural architectures.
1 code implementation • NeurIPS 2021 • Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie zhou, Cho-Jui Hsieh
Based on this observation, we propose a dynamic token sparsification framework to prune redundant tokens progressively and dynamically based on the input.
Ranked #3 on Efficient ViTs on ImageNet-1K (With LV-ViT-S)
no code implementations • ICLR 2022 • Yong liu, Xiangning Chen, Minhao Cheng, Cho-Jui Hsieh, Yang You
Current methods usually use extensive data augmentation to increase the batch size, but we found the performance gain with data augmentation decreases as batch size increases, and data augmentation will become insufficient after certain point.
no code implementations • 19 May 2021 • Seungyeon Kim, Daniel Glasner, Srikumar Ramalingam, Cho-Jui Hsieh, Kishore Papineni, Sanjiv Kumar
It is generally believed that robust training of extremely large networks is critical to their success in real-world applications.
1 code implementation • 18 May 2021 • Yao Li, Tongyi Tang, Cho-Jui Hsieh, Thomas C. M. Lee
In this paper, we propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors and make it easier to simulate the output distribution of a deep neural network.
no code implementations • 30 Apr 2021 • Jun-Ho Choi, huan zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, Jong-Seok Lee
Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated.
1 code implementation • 26 Apr 2021 • Yu-Chuan Su, Soravit Changpinyo, Xiangning Chen, Sathish Thoppay, Cho-Jui Hsieh, Lior Shapira, Radu Soricut, Hartwig Adam, Matthew Brown, Ming-Hsuan Yang, Boqing Gong
To enable progress on this task, we create a new dataset consisting of 220k human-annotated 2. 5D relationships among 512K objects from 11K images.
1 code implementation • ACL 2022 • Fan Yin, Zhouxing Shi, Cho-Jui Hsieh, Kai-Wei Chang
We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria.
1 code implementation • NAACL 2021 • Chong Zhang, Jieyu Zhao, huan zhang, Kai-Wei Chang, Cho-Jui Hsieh
Our method is able to reveal the hidden model biases not directly shown in the test dataset.
2 code implementations • NeurIPS 2021 • Zhouxing Shi, Yihan Wang, huan zhang, JinFeng Yi, Cho-Jui Hsieh
Despite that state-of-the-art (SOTA) methods including interval bound propagation (IBP) and CROWN-IBP have per-batch training complexity similar to standard neural network training, they usually use a long warmup schedule with hundreds or thousands epochs to reach SOTA performance and are thus still costly.
1 code implementation • 29 Mar 2021 • Rulin Shao, Zhouxing Shi, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh
Following the success in advancing natural language processing and understanding, transformers are expected to bring revolutionary changes to computer vision.
1 code implementation • CVPR 2021 • Xiangning Chen, Cihang Xie, Mingxing Tan, Li Zhang, Cho-Jui Hsieh, Boqing Gong
Data augmentation has become a de facto component for training high-performance deep image classifiers, but its potential is under-explored for object detection.
Ranked #17 on Object Detection on COCO-O
4 code implementations • NeurIPS 2021 • Shiqi Wang, huan zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, J. Zico Kolter
Compared to the typically tightest but very costly semidefinite programming (SDP) based incomplete verifiers, we obtain higher verified accuracy with three orders of magnitudes less verification time.
1 code implementation • 3 Feb 2021 • Hojung Lee, Cho-Jui Hsieh, Jong-Seok Lee
We show that the proposed approach successfully decouples the update process of the layer groups for both convolutional neural networks (CNNs) and recurrent neural networks (RNNs).
2 code implementations • ICLR 2021 • huan zhang, Hongge Chen, Duane Boning, Cho-Jui Hsieh
We study the robustness of reinforcement learning (RL) with adversarially perturbed state observations, which aligns with the setting of many adversarial attacks to deep reinforcement learning (DRL) and is also important for rolling out real-world RL agent under unpredictable sensing noise.
no code implementations • 18 Jan 2021 • Seong-Eun Moon, Chun-Jui Chen, Cho-Jui Hsieh, Jane-Ling Wang, Jong-Seok Lee
Convolutional neural networks (CNNs) are widely used to recognize the user's state through electroencephalography (EEG) signals.
no code implementations • 7 Jan 2021 • Rulin Shao, Zhouxing Shi, JinFeng Yi, Pin-Yu Chen, Cho-Jui Hsieh
At the second stage, we design and apply a highly transferable adversarial attack for text CAPTCHAs to better obstruct CAPTCHA solvers.
no code implementations • 1 Jan 2021 • Minhao Cheng, Zhe Gan, Yu Cheng, Shuohang Wang, Cho-Jui Hsieh, Jingjing Liu
By incorporating different feature maps after the masking, we can distill better features to help model generalization.
no code implementations • 1 Jan 2021 • Patrick Chen, Hsiang-Fu Yu, Inderjit S Dhillon, Cho-Jui Hsieh
In this paper, we observe that the learned representation of each layer lies in a low-dimensional space.
no code implementations • 1 Jan 2021 • Yuanhao Xiong, Cho-Jui Hsieh
Recent decades have witnessed great prosperity of deep learning in tackling various problems such as classification and decision making.
no code implementations • ICCV 2021 • Yao Li, Martin Renqiang Min, Thomas Lee, Wenchao Yu, Erik Kruus, Wei Wang, Cho-Jui Hsieh
Recent studies have demonstrated the vulnerability of deep neural networks against adversarial examples.
no code implementations • 1 Jan 2021 • Jing Xu, Zhouxing Shi, huan zhang, JinFeng Yi, Cho-Jui Hsieh, LiWei Wang
We also demonstrate that the perturbation budget generator can produce semantically-meaningful budgets, which implies that the generator can capture contextual information and the sensitivity of different features in a given image.
1 code implementation • 22 Dec 2020 • Minhao Cheng, Pin-Yu Chen, Sijia Liu, Shiyu Chang, Cho-Jui Hsieh, Payel Das
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems.
no code implementations • 14 Dec 2020 • Li-Cheng Lan, Meng-Yu Tsai, Ti-Rong Wu, I-Chen Wu, Cho-Jui Hsieh
This implies that a significant amount of resources can be saved if we are able to stop the searching earlier when we are confident with the current searching result.
no code implementations • 28 Nov 2020 • Devvrit, Minhao Cheng, Cho-Jui Hsieh, Inderjit Dhillon
Several previous attempts tackled this problem by ensembling the soft-label prediction and have been proved vulnerable based on the latest attack methods.
3 code implementations • ICLR 2021 • Kaidi Xu, huan zhang, Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, Cho-Jui Hsieh
Formal verification of neural networks (NNs) is a challenging and important problem.
no code implementations • 17 Nov 2020 • Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang
Based on these studies, we propose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models.
1 code implementation • NeurIPS 2020 • Chong Zhang, huan zhang, Cho-Jui Hsieh
We study the problem of efficient adversarial attacks on tree based ensembles such as gradient boosting decision trees (GBDTs) and random forests (RFs).
no code implementations • 19 Oct 2020 • Yuanhao Xiong, Xuanqing Liu, Li-Cheng Lan, Yang You, Si Si, Cho-Jui Hsieh
For end-to-end efficiency, unlike previous work that assumes random hyperparameter tuning, which over-emphasizes the tuning time, we propose to evaluate with a bandit hyperparameter tuning strategy.
no code implementations • ECCV 2020 • Benlin Liu, Yongming Rao, Jiwen Lu, Jie zhou, Cho-Jui Hsieh
Knowledge Distillation (KD) has been one of the most popu-lar methods to learn a compact model.
1 code implementation • 20 Aug 2020 • Yihan Wang, huan zhang, Hongge Chen, Duane Boning, Cho-Jui Hsieh
In this paper, we study the problem of robustness verification and certified defense with respect to general $\ell_p$ norm perturbations for ensemble decision stumps and trees.
1 code implementation • 7 Aug 2020 • Jiachen Zhong, Xuanqing Liu, Cho-Jui Hsieh
Generative adversarial networks (GAN) have shown remarkable results in image generation tasks.
no code implementations • NeurIPS 2020 • Hongge Chen, Si Si, Yang Li, Ciprian Chelba, Sanjiv Kumar, Duane Boning, Cho-Jui Hsieh
With this score, we can identify the pretraining examples in the pretraining task that contribute most to a prediction in the finetuning task.
no code implementations • ACL 2020 • Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
no code implementations • ACL 2020 • Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, Xuanjing Huang
Despite achieving prominent performance on many important tasks, it has been reported that neural networks are vulnerable to adversarial examples.
1 code implementation • 20 Jun 2020 • Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang
Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.
1 code implementation • ICLR 2021 • Xiangning Chen, Ruochen Wang, Minhao Cheng, Xiaocheng Tang, Cho-Jui Hsieh
This paper proposes a novel differentiable architecture search method by formulating it into a distribution learning problem.
no code implementations • 15 Jun 2020 • Yang You, Yuhui Wang, huan zhang, Zhao Zhang, James Demmel, Cho-Jui Hsieh
For the first time we scale the batch size on ImageNet to at least a magnitude larger than all previous work, and provide detailed studies on the performance of many state-of-the-art optimization schemes under this setting.
2 code implementations • NeurIPS 2020 • Lu Wang, Xuanqing Liu, Jin-Feng Yi, Yuan Jiang, Cho-Jui Hsieh
Metric learning is an important family of algorithms for classification and similarity search, but the robustness of learned metrics against small adversarial perturbations is less studied.
no code implementations • 7 Jun 2020 • Qin Ding, Cho-Jui Hsieh, James Sharpnack
A natural way to resolve this problem is to apply online stochastic gradient descent (SGD) so that the per-step time and memory complexity can be reduced to constant with respect to $t$, but a contextual bandit policy based on online SGD updates that balances exploration and exploitation has remained elusive.
no code implementations • ICLR 2021 • Cheng-Yu Hsieh, Chih-Kuan Yeh, Xuanqing Liu, Pradeep Ravikumar, Seungyeon Kim, Sanjiv Kumar, Cho-Jui Hsieh
In this paper, we establish a novel set of evaluation criteria for such feature based explanations by robustness analysis.
1 code implementation • 11 May 2020 • Lu Wang, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh, Yuan Jiang
By constraining adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset, the spanning attack significantly improves the query efficiency of a wide variety of existing black-box attacks.
no code implementations • ECCV 2020 • Yuanhao Xiong, Cho-Jui Hsieh
Adversarial attack has recently become a tremendous threat to deep learning models.
4 code implementations • NeurIPS 2020 • Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Mingyan Liu, Duane Boning, Cho-Jui Hsieh
Several works have shown this vulnerability via adversarial attacks, but existing approaches on improving the robustness of DRL under this setting have limited success and lack for theoretical principles.
1 code implementation • ICML 2020 • Xuanqing Liu, Hsiang-Fu Yu, Inderjit Dhillon, Cho-Jui Hsieh
The main reason is that position information among input units is not inherently encoded, i. e., the models are permutation equivalent; this problem justifies why all of the existing models are accompanied by a sinusoidal encoding/embedding layer at the input.
Ranked #5 on Semantic Textual Similarity on MRPC
5 code implementations • NeurIPS 2020 • Kaidi Xu, Zhouxing Shi, huan zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, Cho-Jui Hsieh
Linear relaxation based perturbation analysis (LiRPA) for neural networks, which computes provable linear bounds of output neurons given a certain amount of input perturbation, has become a core component in robustness verification and certified defense.
no code implementations • 17 Feb 2020 • Minhao Cheng, Qi Lei, Pin-Yu Chen, Inderjit Dhillon, Cho-Jui Hsieh
Adversarial training has become one of the most effective methods for improving robustness of neural networks.
1 code implementation • ICLR 2020 • Zhouxing Shi, huan zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees.
no code implementations • 13 Feb 2020 • Qin Ding, Cho-Jui Hsieh, James Sharpnack
Classic contextual bandit algorithms for linear models, such as LinUCB, assume that the reward distribution for an arm is modeled by a stationary linear regression.
1 code implementation • ICML 2020 • Xiangning Chen, Cho-Jui Hsieh
Furthermore, we mathematically show that SDARTS implicitly regularizes the Hessian norm of the validation loss, which accounts for a smoother loss landscape and improved performance.
2 code implementations • ICLR 2020 • Runtian Zhai, Chen Dan, Di He, huan zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, Li-Wei Wang
Adversarial training is one of the most popular ways to learn robust models but is usually attack-dependent and time costly.
no code implementations • 3 Dec 2019 • Patrick H. Chen, Wei Wei, Cho-Jui Hsieh, Bo Dai
In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework.
1 code implementation • 11 Nov 2019 • Xiaoyun Wang, Xuanqing Liu, Cho-Jui Hsieh
Inspired by the previous works on adversarial defense for deep neural networks, and especially adversarial training algorithm, we propose a method called GraphDefense to defend against the adversarial perturbations.
no code implementations • IJCNLP 2019 • Yukun Ma, Patrick H. Chen, Cho-Jui Hsieh
For example, input embedding and Softmax matrices in IWSLT-2014 German-to-English data set account for more than 80{\%} of the total model parameters.
no code implementations • 31 Oct 2019 • Huan Zhang, Minhao Cheng, Cho-Jui Hsieh
We propose an algorithm to enhance certified robustness of a deep model ensemble by optimally weighting each base model.
no code implementations • NeurIPS 2019 • Xuanqing Liu, Si Si, Xiaojin Zhu, Yang Li, Cho-Jui Hsieh
In this paper, we proposed a general framework for data poisoning attacks to graph-based semi-supervised learning (G-SSL).
1 code implementation • ICLR 2020 • Yangjun Ruan, Yuanhao Xiong, Sashank Reddi, Sanjiv Kumar, Cho-Jui Hsieh
In the learning to learn (L2L) framework, we cast the design of optimization algorithms as a machine learning problem and use deep neural networks to learn the update rules.
1 code implementation • NeurIPS 2020 • Utkarsh Ojha, Krishna Kumar Singh, Cho-Jui Hsieh, Yong Jae Lee
We propose a novel unsupervised generative model that learns to disentangle object identity from other low-level aspects in class-imbalanced data.
2 code implementations • 25 Sep 2019 • Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James Sharpnack
Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed for better use of the temporal ordering of items that each user has engaged with.
Ranked #1 on Recommendation Systems on MovieLens 1M (nDCG@10 metric)
no code implementations • 25 Sep 2019 • Minhao Cheng, Pin-Yu Chen, Sijia Liu, Shiyu Chang, Cho-Jui Hsieh, Payel Das
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy and reliable machine learning systems.
no code implementations • 25 Sep 2019 • Liu Liu, Ji Liu, Cho-Jui Hsieh, DaCheng Tao
The strategy is also accompanied by a mini-batch version of the proposed method that improves query complexity with respect to the size of the mini-batch.
no code implementations • 25 Sep 2019 • Patrick H. Chen, Sashank Reddi, Sanjiv Kumar, Cho-Jui Hsieh
We consider the learning to learn problem, where the goal is to leverage deeplearning models to automatically learn (iterative) optimization algorithms for training machine learning models.
no code implementations • 25 Sep 2019 • Yao Li, Martin Renqiang Min, Wenchao Yu, Cho-Jui Hsieh, Thomas Lee, Erik Kruus
Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples.
1 code implementation • ICLR 2020 • Minhao Cheng, Simranjit Singh, Patrick Chen, Pin-Yu Chen, Sijia Liu, Cho-Jui Hsieh
We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.
no code implementations • 10 Sep 2019 • Zhenxin Xiao, Puyudi Yang, Yuchen Jiang, Kai-Wei Chang, Cho-Jui Hsieh
Adversarial example generation becomes a viable method for evaluating the robustness of a machine learning model.
1 code implementation • 6 Sep 2019 • Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, Cho-Jui Hsieh
This work proposes a novel algorithm to generate natural language adversarial input for text classification models, in order to investigate the robustness of these models.
3 code implementations • 15 Aug 2019 • Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James Sharpnack
Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed us to make better use of the temporal ordering of items that each user has engaged with.
7 code implementations • 9 Aug 2019 • Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
Ranked #1 on Visual Reasoning on NLVR
no code implementations • ACL 2019 • Yu-Lun Hsieh, Minhao Cheng, Da-Cheng Juan, Wei Wei, Wen-Lian Hsu, Cho-Jui Hsieh
This work examines the robustness of self-attentive neural networks against adversarial input perturbations.
2 code implementations • 28 Jun 2019 • Quanming Yao, Xiangning Chen, James Kwok, Yong Li, Cho-Jui Hsieh
Motivated by the recent success of automated machine learning (AutoML), we propose in this paper the search for simple neural interaction functions (SIF) in CF.
no code implementations • NeurIPS 2019 • Ruiqi Gao, Tianle Cai, Haochuan Li, Li-Wei Wang, Cho-Jui Hsieh, Jason D. Lee
Neural networks are vulnerable to adversarial examples, i. e. inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network.
2 code implementations • ICLR 2020 • Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, Cho-Jui Hsieh
In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass.
1 code implementation • 10 Jun 2019 • Lu Wang, Xuanqing Liu, Jin-Feng Yi, Zhi-Hua Zhou, Cho-Jui Hsieh
Furthermore, we show that dual solutions for these QP problems could give us a valid lower bound of the adversarial perturbation that can be used for formal robustness verification, giving us a nice view of attack/verification for NN models.
2 code implementations • NeurIPS 2019 • Hongge Chen, huan zhang, Si Si, Yang Li, Duane Boning, Cho-Jui Hsieh
We show that there is a simple linear time algorithm for verifying a single tree, and for tree ensembles, the verification problem can be cast as a max-clique problem on a multi-partite graph with bounded boxicity.
no code implementations • 8 Jun 2019 • Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael. I. Jordan
Furthermore, we extend our method to include multi-layer feature attributions in order to tackle the attacks with mixed confidence levels.
1 code implementation • 5 Jun 2019 • Xuanqing Liu, Tesi Xiao, Si Si, Qin Cao, Sanjiv Kumar, Cho-Jui Hsieh
In this paper, we propose a new continuous neural network framework called Neural Stochastic Differential Equation (Neural SDE) network, which naturally incorporates various commonly used regularization mechanisms based on random noise injection.
no code implementations • NAACL 2019 • Minhao Cheng, Wei Wei, Cho-Jui Hsieh
Moreover, we show that with the adversarial training, we are able to improve the robustness of negotiation agents by 1. 5 points on average against all our attacks.
no code implementations • 29 May 2019 • Liwei Wu, Hsiang-Fu Yu, Nikhil Rao, James Sharpnack, Cho-Jui Hsieh
In this paper, we propose using Graph DNA, a novel Deep Neighborhood Aware graph encoding algorithm, for exploiting deeper neighborhood information.
3 code implementations • NeurIPS 2019 • Liwei Wu, Shuqing Li, Cho-Jui Hsieh, James Sharpnack
We find that when used along with widely-used regularization methods such as weight decay and dropout, our proposed SSE can further reduce overfitting, which often leads to more favorable generalization results.
6 code implementations • KDD 2019 • Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh
Furthermore, Cluster-GCN allows us to train much deeper GCN without much time and memory overhead, which leads to improved prediction accuracy---using a 5-layer Cluster-GCN, we achieve state-of-the-art test F1 score 99. 36 on the PPI dataset, while the previous best result was 98. 71 by [16].
Ranked #1 on Node Classification on Amazon2M
no code implementations • ICLR 2019 • Minhao Cheng, Thong Le, Pin-Yu Chen, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh
We study the problem of attacking machine learning models in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
1 code implementation • ICCV 2019 • Jun-Ho Choi, huan zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, Jong-Seok Lee
Single-image super-resolution aims to generate a high-resolution version of a low-resolution image, which serves as an essential component in many computer vision applications.
24 code implementations • ICLR 2020 • Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, Cho-Jui Hsieh
In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches.
Ranked #11 on Question Answering on SQuAD1.1 dev (F1 metric)
no code implementations • TACL 2019 • Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang
Contextual representation models have achieved great success in improving various downstream natural language processing tasks.
no code implementations • 28 Feb 2019 • Liunian Harold Li, Patrick H. Chen, Cho-Jui Hsieh, Kai-Wei Chang
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
3 code implementations • 27 Feb 2019 • Hongge Chen, huan zhang, Duane Boning, Cho-Jui Hsieh
Although adversarial examples and model robustness have been extensively studied in the context of linear models and neural networks, research on this issue in tree-based models and how to make tree-based models robust against adversarial examples is still limited.
3 code implementations • NeurIPS 2019 • Hadi Salman, Greg Yang, huan zhang, Cho-Jui Hsieh, Pengchuan Zhang
This framework works for neural networks with diverse architectures and nonlinearities and covers both primal and dual views of robustness verification.
1 code implementation • 24 Jan 2019 • Yang You, Jonathan Hseu, Chris Ying, James Demmel, Kurt Keutzer, Cho-Jui Hsieh
LEGW enables Sqrt Scaling scheme to be useful in practice and as a result we achieve much better results than the Linear Scaling learning rate scheme.
no code implementations • ICLR 2019 • Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit S. Dhillon, Cho-Jui Hsieh
In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on test set) of adversarial training has a strong correlation with the distance between a test point and the manifold of training data embedded by the network.
no code implementations • NeurIPS 2018 • Yao Li, Minhao Cheng, Kevin Fujii, Fushing Hsieh, Cho-Jui Hsieh
We study the problem of learning from group comparisons, with applications in predicting outcomes of sports and online games.
no code implementations • 19 Nov 2018 • Yao Li, Martin Renqiang Min, Wenchao Yu, Cho-Jui Hsieh, Thomas C. M. Lee, Erik Kruus
Recent studies have demonstrated the vulnerability of deep convolutional neural networks against adversarial examples.
no code implementations • 4 Nov 2018 • Yuefeng Liang, Cho-Jui Hsieh, Thomas C. M. Lee
Extreme multi-label classification aims to learn a classifier that annotates an instance with a relevant subset of labels from an extremely large label set.
14 code implementations • NeurIPS 2018 • Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, Luca Daniel
Finding minimum distortion of adversarial examples and thus certifying robustness in neural network classifiers for given data points is known to be a challenging problem.
no code implementations • ICLR 2019 • Patrick H. Chen, Si Si, Sanjiv Kumar, Yang Li, Cho-Jui Hsieh
The algorithm achieves an order of magnitude faster inference than the original softmax layer for predicting top-$k$ words in various tasks such as beam search in machine translation or next words prediction.
4 code implementations • 28 Oct 2018 • Huan Zhang, Pengchuan Zhang, Cho-Jui Hsieh
The Jacobian matrix (or the gradient for single-output networks) is directly related to many important properties of neural networks, such as the function landscape, stationary points, (local) Lipschitz constants and robustness to adversarial attacks.
no code implementations • ICLR 2019 • Xiaoyun Wang, Minhao Cheng, Joe Eaton, Cho-Jui Hsieh, Felix Wu
In this paper, we propose a new type of "fake node attacks" to attack GCNs by adding malicious fake nodes.
1 code implementation • 19 Oct 2018 • Tsui-Wei Weng, huan zhang, Pin-Yu Chen, Aurelie Lozano, Cho-Jui Hsieh, Luca Daniel
We apply extreme value theory on the new formal robustness guarantee and the estimated robustness is called second-order CLEVER score.
1 code implementation • ICLR 2019 • Xuanqing Liu, Yao Li, Chongruo wu, Cho-Jui Hsieh
Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way.
no code implementations • 26 Sep 2018 • Liu Liu, Xuanqing Liu, Cho-Jui Hsieh, DaCheng Tao
Trust region and cubic regularization methods have demonstrated good performance in small scale non-convex optimization, showing the ability to escape from saddle points.
no code implementations • 6 Sep 2018 • Liu Liu, Ji Liu, Cho-Jui Hsieh, DaCheng Tao
In this paper, we consider the convex and non-convex composition problem with the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where $G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )} $ is the inner function, and $F_i(\cdot)$ is the outer function.
no code implementations • ICLR 2019 • Jiarui Fang, Haohuan Fu, Guangwen Yang, Cho-Jui Hsieh
Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes.
no code implementations • ICML 2018 • Xuanqing Liu, Cho-Jui Hsieh
In this paper we study a family of variance reduction methods with randomized batch size---at each step, the algorithm first randomly chooses the batch size and then selects a batch of samples to conduct a variance-reduced stochastic update.
2 code implementations • CVPR 2019 • Xuanqing Liu, Cho-Jui Hsieh
Adversarial training is the technique used to improve the robustness of discriminator by combining adversarial attacker and discriminator in the training phase.
1 code implementation • 12 Jul 2018 • Minhao Cheng, Thong Le, Pin-Yu Chen, Jin-Feng Yi, huan zhang, Cho-Jui Hsieh
We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions.
no code implementations • ICML 2018 • Minhao Cheng, Ian Davidson, Cho-Jui Hsieh
We consider the setting where we wish to perform ranking for hundreds of thousands of users which is common in recommender systems and web search ranking.
no code implementations • NeurIPS 2018 • Patrick H. Chen, Si Si, Yang Li, Ciprian Chelba, Cho-Jui Hsieh
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses.
no code implementations • NAACL 2018 • Chao Jiang, Hsiang-Fu Yu, Cho-Jui Hsieh, Kai-Wei Chang
In such a situation, the co-occurrence matrix is sparse as the co-occurrences of many word pairs are unobserved.
no code implementations • 31 May 2018 • Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, Michael. I. Jordan
We present a probabilistic framework for studying adversarial attacks on discrete data.
no code implementations • 30 May 2018 • Liu Liu, Minhao Cheng, Cho-Jui Hsieh, DaCheng Tao
However, due to the variance in the search direction, the convergence rates and query complexities of existing methods suffer from a factor of $d$, where $d$ is the problem dimension.
1 code implementation • 30 May 2018 • Chun-Chen Tu, Pai-Shun Ting, Pin-Yu Chen, Sijia Liu, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh, Shin-Ming Cheng
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting.
3 code implementations • 28 May 2018 • Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, huan zhang, Cho-Jui Hsieh, Mani Srivastava
Our experiments on different datasets (MNIST, CIFAR-10, and ImageNet) show that GenAttack can successfully generate visually imperceptible adversarial examples against state-of-the-art image recognition models with orders of magnitude fewer queries than previous approaches.
1 code implementation • 9 May 2018 • Chao Jiang, Hsiang-Fu Yu, Cho-Jui Hsieh, Kai-Wei Chang
In such a situation, the co-occurrence matrix is sparse as the co-occurrences of many word pairs are unobserved.
6 code implementations • ICML 2018 • Tsui-Wei Weng, huan zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Duane Boning, Inderjit S. Dhillon, Luca Daniel
Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17].
1 code implementation • 3 Mar 2018 • Minhao Cheng, Jin-Feng Yi, Pin-Yu Chen, huan zhang, Cho-Jui Hsieh
In this paper, we study the much more challenging problem of crafting adversarial examples for sequence-to-sequence (seq2seq) models, whose inputs are discrete text strings and outputs have an almost infinite number of possibilities.
1 code implementation • ICML 2018 • Liwei Wu, Cho-Jui Hsieh, James Sharpnack
In this paper, we propose a listwise approach for constructing user-specific rankings in recommendation systems in a collaborative fashion.
1 code implementation • 15 Feb 2018 • Puyudi Yang, Cho-Jui Hsieh, Jane-Ling Wang
In this paper we propose a new algorithm for streaming principal component analysis.
1 code implementation • ICLR 2018 • Tsui-Wei Weng, huan zhang, Pin-Yu Chen, Jin-Feng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel
Our analysis yields a novel robustness metric called CLEVER, which is short for Cross Lipschitz Extreme Value for nEtwork Robustness.
no code implementations • ICLR 2018 • Xuanqing Liu, Jason D. Lee, Cho-Jui Hsieh
Solving this subproblem is non-trivial---existing methods have only sub-linear convergence rate.
no code implementations • ICLR 2018 • Patrick H. Chen, Cho-Jui Hsieh
Despite many second-order methods have been proposed to train neural networks, most of the results were done on smaller single layer fully connected networks, so we still cannot conclude whether it's useful in training deep convolutional networks.
2 code implementations • ACL 2018 • Hongge Chen, huan zhang, Pin-Yu Chen, Jin-Feng Yi, Cho-Jui Hsieh
Our extensive experiments show that our algorithm can successfully craft visually-similar adversarial examples with randomly targeted captions or keywords, and the adversarial examples can be made highly transferable to other image captioning systems.
no code implementations • ECCV 2018 • Xuanqing Liu, Minhao Cheng, huan zhang, Cho-Jui Hsieh
In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}.
1 code implementation • 14 Sep 2017 • Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, Kurt Keutzer
If we can make full use of the supercomputer for DNN training, we should be able to finish the 90-epoch ResNet-50 training in one minute.
6 code implementations • 13 Sep 2017 • Pin-Yu Chen, Yash Sharma, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh
Recent studies have highlighted the vulnerability of deep neural networks (DNNs) to adversarial examples - a visually indistinguishable adversarial image can easily be crafted to cause a well-trained model to misclassify.
no code implementations • 28 Aug 2017 • Xuanqing Liu, Cho-Jui Hsieh, Jason D. Lee, Yuekai Sun
We propose a fast proximal Newton-type algorithm for minimizing regularized finite sums that returns an $\epsilon$-suboptimal point in $\tilde{\mathcal{O}}(d(n + \sqrt{\kappa d})\log(\frac{1}{\epsilon}))$ FLOPS, where $n$ is number of samples, $d$ is feature dimension, and $\kappa$ is the condition number.
5 code implementations • 14 Aug 2017 • Pin-Yu Chen, huan zhang, Yash Sharma, Jin-Feng Yi, Cho-Jui Hsieh
However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples.
no code implementations • ICML 2017 • Si Si, huan zhang, S. Sathiya Keerthi, Dhruv Mahajan, Inderjit S. Dhillon, Cho-Jui Hsieh
In this paper, we study the gradient boosted decision trees (GBDT) when the output space is high dimensional and sparse.
3 code implementations • 26 Jun 2017 • Huan Zhang, Si Si, Cho-Jui Hsieh
In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training.
3 code implementations • NeurIPS 2017 • Xiangru Lian, Ce Zhang, huan zhang, Cho-Jui Hsieh, Wei zhang, Ji Liu
On network configurations with low bandwidth or high latency, D-PSGD can be up to one order of magnitude faster than its well-optimized centralized counterparts.
no code implementations • NeurIPS 2017 • Jinfeng Yi, Cho-Jui Hsieh, Kush Varshney, Lijun Zhang, Yao Li
In particular for durable goods, time utility is a function of inter-purchase duration within product category because consumers are unlikely to purchase two items in the same category in close temporal succession.
no code implementations • NeurIPS 2016 • Yang You, Xiangru Lian, Ji Liu, Hsiang-Fu Yu, Inderjit S. Dhillon, James Demmel, Cho-Jui Hsieh
n this paper, we propose and study an Asynchronous parallel Greedy Coordinate Descent (Asy-GCD) algorithm for minimizing a smooth function with bounded constraints.
no code implementations • NeurIPS 2017 • Hsiang-Fu Yu, Cho-Jui Hsieh, Qi Lei, Inderjit S. Dhillon
Maximum Inner Product Search (MIPS) is an important task in many machine learning applications such as the prediction phase of a low-rank matrix factorization model for a recommender system.