Search Results for author:
Found 26 papers, 21 papers with code
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks.
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models
To address these challenges, we introduce JailbreakBench, an open-sourced benchmark with the following components: (1) a new jailbreaking dataset containing 100 unique behaviors, which we call JBB-Behaviors; (2) an evolving repository of state-of-the-art adversarial prompts, which we refer to as jailbreak artifacts; (3) a standardized evaluation framework that includes a clearly defined threat model, system prompts, chat templates, and scoring functions; and (4) a leaderboard that tracks the performance of attacks and defenses for various LLMs.
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models
The CLIP model, or one of its variants, is used as a frozen vision encoder in many vision-language models (VLMs), e. g. LLaVA and OpenFlamingo.
Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they?
Segment (Almost) Nothing: Prompt-Agnostic Adversarial Attacks on Segmentation Models
General purpose segmentation models are able to generate (semantic) segmentation masks from a variety of prompts, including visual (points, boxed, etc.)
Robust Semantic Segmentation: Strong Adversarial Attacks and Fast Training of Robust Models
While a large amount of work has focused on designing adversarial attacks against image classifiers, only a few methods exist to attack semantic segmentation models.
Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models
While adversarial training has been extensively studied for ResNet architectures and low resolution datasets like CIFAR, much less is known for ImageNet.
Seasoning Model Soups for Robustness to Adversarial and Natural Distribution Shifts
Adversarial training is widely used to make classifiers robust to a specific threat or adversary, such as $\ell_p$-norm bounded perturbations of a given $p$-norm.
A Modern Look at the Relationship between Sharpness and Generalization
Overall, we observe that sharpness does not correlate well with generalization but rather with some training parameters like the learning rate that can be positively or negatively correlated with generalization depending on the setup.
Diffusion Visual Counterfactual Explanations
Two modifications to the diffusion process are key for our DVCEs: first, an adaptive parameterization, whose hyperparameters generalize across images and models, together with distance regularization and late start of the diffusion process, allow us to generate images with minimal semantic changes to the original ones but different classification.
Revisiting adapters with adversarial training
By co-training a neural network on clean and adversarial inputs, it is possible to improve classification accuracy on the clean, non-adversarial inputs.
On the interplay of adversarial robustness and architecture components: patches, convolution and attention
In recent years novel architecture components for image classification have been developed, starting with attention and patches used in transformers.
Sparse Visual Counterfactual Explanations in Image Space
Visual counterfactual explanations (VCEs) in image space are an important tool to understand decisions of image classifiers as they show under which changes of the image the decision of the classifier would change.
Evaluating the Adversarial Robustness of Adaptive Test-time Defenses
Adaptive defenses, which optimize at test time, promise to improve adversarial robustness.
Adversarial Robustness against Multiple and Single $l_p$-Threat Models via Quick Fine-Tuning of Robust Classifiers
In this way we get the first multiple-norm robust model for ImageNet and boost the state-of-the-art for multiple-norm robustness to more than $51\%$ on CIFAR-10.
Mind the box: $l_1$-APGD for sparse adversarial attacks on image classifiers
Finally, we combine $l_1$-APGD and an adaptation of the Square Attack to $l_1$ into $l_1$-AutoAttack, an ensemble of attacks which reliably assesses adversarial robustness for the threat model of $l_1$-ball intersected with $[0, 1]^d$.
RobustBench: a standardized adversarial robustness benchmark
As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models.
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting.
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness.
Square Attack: a query-efficient black-box adversarial attack via random search
We propose the Square Attack, a score-based black-box $l_2$- and $l_\infty$-adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking.
Sparse and Imperceivable Adversarial Attacks
On the other hand the pixelwise perturbations of sparse attacks are typically large and thus can be potentially detected.
Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
The evaluation of robustness against adversarial manipulation of neural networks-based classifiers is mainly tested with empirical attacks as methods for the exact computation, even when available, do not scale to large networks.
Provable robustness against all adversarial $l_p$-perturbations for $p\geq 1$
In recent years several adversarial attacks and defenses have been proposed.
Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks
Modern neural networks are highly non-robust against adversarial manipulation.
A randomized gradient-free attack on ReLU networks
Relatively fast heuristics have been proposed to produce these adversarial inputs but the problem of finding the optimal adversarial input, that is with the minimal change of the input, is NP-hard.
Provable Robustness of ReLU networks via Maximization of Linear Regions
It has been shown that neural network classifiers are not robust.
