Search Results for author:
Found 11 papers, 8 papers with code
Generative AI Security: Challenges and Countermeasures
Generative AI's expanding footprint across numerous industries has led to both excitement and increased scrutiny.
PAL: Proxy-Guided Black-Box Attack on Large Language Models
In this work, we introduce the Proxy-Guided Attack on LLMs (PAL), the first optimization-based attack on LLMs in a black-box query-only setting.
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods.
Mark My Words: Analyzing and Evaluating Language Model Watermarks
The capabilities of large language models have grown significantly in recent years and so too have concerns about their misuse.
Can LLMs Follow Simple Rules?
As Large Language Models (LLMs) are deployed with increasing real-world responsibilities, it is important to be able to specify and constrain the behavior of these systems in a reliable manner.
SLIP: Self-supervision meets Language-Image Pre-training
Across ImageNet and a battery of additional datasets, we find that SLIP improves accuracy by a large margin.
A Rigorous Evaluation of Real-World Distribution Shifts
Motivated by this, we introduce a new data augmentation method which advances the state-of-the-art and outperforms models pretrained with 1000x more labeled data.
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
We find that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work.
Ranked #29 on
Domain Generalization
on ImageNet-R
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
We propose AugMix, a data processing technique that is simple to implement, adds limited computational overhead, and helps models withstand unforeseen corruptions.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
MNIST-C: A Robustness Benchmark for Computer Vision
We introduce the MNIST-C dataset, a comprehensive suite of 15 corruptions applied to the MNIST test set, for benchmarking out-of-distribution robustness in computer vision.
Parameter Re-Initialization through Cyclical Batch Size Schedules
We demonstrate the ability of our method to improve language modeling performance by up to 7. 91 perplexity and reduce training iterations by up to $61\%$, in addition to its flexibility in enabling snapshot ensembling and use with adversarial training.
Ranked #51 on
Natural Language Inference
on SNLI
