Search Results for author:
Found 25 papers, 20 papers with code
Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs
Large language models are aligned to be safe, preventing users from generating harmful content like misinformation or instructions for illegal activities.
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks.
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (2) a jailbreaking dataset comprising 100 behaviors -- both original and sourced from prior work -- which align with OpenAI's usage policies; (3) a standardized evaluation framework that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard that tracks the performance of attacks and defenses for various LLMs.
Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they?
Scaling Compute Is Not All You Need for Adversarial Robustness
Finally, we make our benchmarking framework (built on top of \texttt{timm}~\citep{rw2019timm}) publicly available to facilitate future analysis in efficient robust deep learning.
Analyzing Sharpness-aware Minimization under Overparameterization
Training an overparameterized neural network can yield minimizers of different generalization capabilities despite the same level of training loss.
Why Do We Need Weight Decay in Modern Deep Learning?
In this work, we highlight that the role of weight decay in modern deep learning is different from its regularization effect studied in classical learning theory.
Layer-wise Linear Mode Connectivity
Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models.
Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings
We present a method that allows us to train adversarially robust deep networks for tabular data and to transfer this robustness to other classifiers via universal robust embeddings tailored to categorical data.
A Modern Look at the Relationship between Sharpness and Generalization
Overall, we observe that sharpness does not correlate well with generalization but rather with some training parameters like the learning rate that can be positively or negatively correlated with generalization depending on the setup.
SGD with Large Step Sizes Learns Sparse Features
We present empirical observations that commonly used large step sizes (i) lead the iterates to jump from one side of a valley to the other causing loss stabilization, and (ii) this stabilization induces a hidden stochastic dynamics orthogonal to the bouncing directions that biases it implicitly toward sparse predictors.
Towards Understanding Sharpness-Aware Minimization
We further study the properties of the implicit bias on non-linear networks empirically, where we show that fine-tuning a standard model with SAM can lead to significant generalization improvements.
ARIA: Adversarially Robust Image Attribution for Content Provenance
Finally, we show how to train an adversarially robust image comparator model for detecting editorial changes in matched images.
Understanding Sharpness-Aware Minimization
Next, we discuss why SAM can be helpful in the noisy label setting where we first show that it can help to improve generalization even for linear classifiers.
On the effectiveness of adversarial training against common corruptions
First, we show that, when used with an appropriately selected perturbation radius, $\ell_p$ adversarial training can serve as a strong baseline against common corruptions improving both accuracy and calibration.
RobustBench: a standardized adversarial robustness benchmark
As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models.
Understanding and Improving Fast Adversarial Training
We show that adding a random step to FGSM, as proposed in Wong et al. (2020), does not prevent catastrophic overfitting, and that randomness is not important per se -- its main role being simply to reduce the magnitude of the perturbation.
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting.
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.
Square Attack: a query-efficient black-box adversarial attack via random search
We propose the Square Attack, a score-based black-box $l_2$- and $l_\infty$-adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking.
Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
The problem of adversarial robustness has been studied extensively for neural networks.
Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem
We show that this technique is surprisingly effective in reducing the confidence of predictions far away from the training data while maintaining high confidence predictions and test error on the original classification task compared to standard training.
Logit Pairing Methods Can Fool Gradient-Based Attacks
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.
Provable Robustness of ReLU networks via Maximization of Linear Regions
It has been shown that neural network classifiers are not robust.
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
We show in this paper for the first time formal guarantees on the robustness of a classifier by giving instance-specific lower bounds on the norm of the input manipulation required to change the classifier decision.
