Search Results for author:
Found 84 papers, 41 papers with code
SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image Editing
Deep neural networks (DNNs) have achieved great successes in various vision applications due to their strong expressive power.
JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge.
Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Models
Building on the insights from the user study, we also developed a system using AI as the assistant to automate the process of jailbreak prompt generation.
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting
However, with the integration of additional modalities, MLLMs are exposed to new vulnerabilities, rendering them prone to structured-based jailbreak attacks, where semantic content (e. g., "harmful text") has been injected into the images to mislead MLLMs.
Automatic and Universal Prompt Injection Attacks against Large Language Models
Large Language Models (LLMs) excel in processing and generating human language, powered by their ability to interpret and follow instructions.
A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems
Large Language Model (LLM) systems are inherently compositional, with individual LLM serving as the core foundation with additional layers of objects such as plugins, sandbox, and so on.
WIPI: A New Web Threat for LLM-Driven Web Agents
To evaluate the effectiveness of the proposed methodology, we conducted extensive experiments using 7 plugin-based ChatGPT Web Agents, 8 Web GPTs, and 3 different open-source Web Agents.
Mitigating Fine-tuning Jailbreak Attack with Backdoor Enhanced Alignment
Despite the general capabilities of Large Language Models (LLMs) like GPT-4 and Llama-2, these models still request fine-tuning or adaptation with customized data when it comes to meeting the specific business demands and intricacies of tailored use cases.
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model.
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
There is a surprisingly large gap between the speed and scale of their development and deployment and our understanding of their safety risks.
Preference Poisoning Attacks on Reward Model Learning
In addition, we observe that the simpler and more scalable rank-by-distance approaches are often competitive with the best, and on occasion significantly outperform gradient-based methods.
A Learning-based Declarative Privacy-Preserving Framework for Federated Data Management
It is challenging to balance the privacy and accuracy for federated query processing over multiple private data silos.
Instructional Fingerprinting of Large Language Models
The exorbitant cost of training Large language models (LLMs) from scratch makes it essential to fingerprint the models to protect intellectual property via ownership authentication and to ensure downstream users and developers comply with their license terms (e. g. restricting commercial use).
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios
Simulation plays a crucial role in the development of autonomous vehicles (AVs) due to the potential risks associated with real-world testing.
Exploring the Limits of ChatGPT in Software Security Applications
Large language models (LLMs) have undergone rapid evolution and achieved remarkable results in recent times.
DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructions
In this paper, we introduce DeceptPrompt, a novel algorithm that can generate adversarial natural language instructions that drive the Code LLMs to generate functionality correct code with vulnerabilities.
Dolphins: Multimodal Language Model for Driving
The quest for fully autonomous vehicles (AVs) capable of navigating complex real-world scenarios with human-like understanding and responsiveness.
On the Exploitability of Reinforcement Learning with Human Feedback for Large Language Models
Reinforcement Learning with Human Feedback (RLHF) is a methodology designed to align Large Language Models (LLMs) with human preferences, playing an important role in LLMs alignment.
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking
While large language models (LLMs) have demonstrated increasing power, they have also given rise to a wide range of harmful behaviors.
Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations
Existing studies in backdoor defense have predominantly focused on the training phase, overlooking the critical aspect of testing time defense.
HiCL: Hierarchical Contrastive Learning of Unsupervised Sentence Embeddings
In this paper, we propose a hierarchical contrastive learning framework, HiCL, which considers local segment-level and global sequence-level relationships to improve training efficiency and effectiveness.
Leveraging Hierarchical Feature Sharing for Efficient Dataset Condensation
In this paper, we recognize that images share common features in a hierarchical way due to the inherent hierarchical structure of the classification system, which is overlooked by current data parameterization methods.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
CSI: Enhancing the Robustness of 3D Point Cloud Recognition against Corruption
In this study, we harness the inherent set property of point cloud data to introduce a novel critical subset identification (CSI) method, aiming to bolster recognition robustness in the face of data corruption.
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts?
Semantic Adversarial Attacks via Diffusion Models
Then there are two variants of this framework: 1) the Semantic Transformation (ST) approach fine-tunes the latent space of the generated image and/or the diffusion model itself; 2) the Latent Masking (LM) approach masks the latent space with another target image and local backpropagation-based interpretation methods.
Reinforcement Learning with Human Feedback for Realistic Traffic Simulation
This works aims to address this by developing a framework that employs reinforcement learning with human preference (RLHF) to enhance the realism of existing traffic models.
DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing
Diffusion models have been leveraged to perform adversarial purification and thus provide both empirical and certified robustness for a standard model.
On the Exploitability of Instruction Tuning
In this work, we investigate how an adversary can exploit instruction tuning by injecting specific instruction-following examples into the training data that intentionally changes the model's behavior.
Differentially Private Video Activity Recognition
In recent years, differential privacy has seen significant advancements in image classification; however, its application to video activity recognition remains under-explored.
CALICO: Self-Supervised Camera-LiDAR Contrastive Pre-training for BEV Perception
CALICO's efficacy is substantiated by extensive evaluations on 3D object detection and BEV map segmentation tasks, where it delivers significant performance improvements.
ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback
This research sheds light on the potential of ChatGPT and conversational LLMs for drug editing.
Voyager: An Open-Ended Embodied Agent with Large Language Models
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention.
Adversarial Demonstration Attacks on Large Language Models
We propose a novel attack method named advICL, which aims to manipulate only the demonstration without changing the input to mislead the models.
Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models
We investigate security concerns of the emergent instruction tuning paradigm, that models are trained on crowdsourced datasets with task instructions to achieve superior performance.
From Shortcuts to Triggers: Backdoor Defense with Denoised PoE
Language models are often at risk of diverse backdoor attacks, especially data poisoning.
Defending against Insertion-based Textual Backdoor Attacks via Attribution
Textual backdoor attack, as a novel attack model, has been shown to be effective in adding a backdoor to the model during training.
ChatGPT as an Attack Tool: Stealthy Textual Backdoor Attack via Blackbox Generative Model Trigger
Textual backdoor attacks pose a practical threat to existing systems, as they can compromise the model by inserting imperceptible triggers into inputs and manipulating labels in the training dataset.
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Thus, it is still an open question: shall we pretrain large autoregressive LMs with retrieval?
Mole Recruitment: Poisoning of Image Classifiers via Selective Batch Sampling
In this work, we present a data poisoning attack that confounds machine learning models without any manipulation of the image or label.
Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistency
Deep neural networks are proven to be vulnerable to backdoor attacks.
Prismer: A Vision-Language Model with Multi-Task Experts
Recent vision-language models have shown impressive multi-modal generation capabilities.
Ranked #1 on
Image Captioning
on nocaps val
Defending against Adversarial Audio via Diffusion Model
In this paper, we propose an adversarial purification-based defense pipeline, AudioPure, for acoustic systems via off-the-shelf diffusion models.
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion
To enable such capability in AI systems, we propose VoxFormer, a Transformer-based semantic scene completion framework that can output complete 3D volumetric semantics from only 2D images.
3D Semantic Scene Completion from a single RGB image
Depth Estimation
PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees
In this paper, we propose PerAda, a parameter-efficient pFL framework that reduces communication and computational costs and exhibits superior generalization performance, especially under test-time distribution shifts.
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e. g., Flamingo) has obtained the state-of-the-art results in image-to-text generation.
A Text-guided Protein Design Framework
Current AI-assisted protein design mainly utilizes protein sequential and structural information.
Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing
Here we present a multi-modal molecule structure-text model, MoleculeSTM, by jointly learning molecules' chemical structures and textual descriptions via a contrastive learning strategy.
DensePure: Understanding Diffusion Models towards Adversarial Robustness
By using the highest density point in the conditional distribution as the reversed sample, we identify the robust region of a given instance under the diffusion model's reverse process.
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors.
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
In evaluating cross-dataset generalization with unseen categories, TPT performs on par with the state-of-the-art approaches that use additional training data.
Retrieval-based Controllable Molecule Generation
On various tasks ranging from simple design criteria to a challenging real-world scenario for designing lead compounds that bind to the SARS-CoV-2 main protease, we demonstrate our approach extrapolates well beyond the retrieval database, and achieves better performance and wider applicability than previous methods.
PointDP: Diffusion-driven Purification against Adversarial Attacks on 3D Point Cloud Recognition
3D Point cloud is becoming a critical data representation in many real-world applications like autonomous driving, robotics, and medical imaging.
Robust Trajectory Prediction against Adversarial Attacks
We demonstrate that our method is able to improve the performance by 46% on adversarial data and at the cost of only 3% performance degradation on clean data, compared to the model trained with clean data.
SecretGen: Privacy Recovery on Pre-Trained Models via Distribution Discrimination
We first explore different statistical information which can discriminate the private training distribution from other distributions.
Diffusion Models for Adversarial Purification
Adversarial purification refers to a class of defense methods that remove adversarial perturbations using a generative model.
Understanding The Robustness in Vision Transformers
Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.
Ranked #4 on
Domain Generalization
on ImageNet-R
(using extra training data)
RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning
This task remains challenging for current deep learning algorithms since it requires addressing three key technical problems jointly: 1) identifying object entities and their properties, 2) inferring semantic relations between pairs of entities, and 3) generalizing to novel object-relation combinations, i. e., systematic generalization.
Ranked #1 on
Zero-Shot Human-Object Interaction Detection
on HICO
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models.
Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions
Deep neural networks on 3D point cloud data have been widely used in the real world, especially in safety-critical applications.
Ranked #1 on
3D Point Cloud Data Augmentation
on ModelNet40-C
3D Point Cloud Classification
3D Point Cloud Data Augmentation
+2
Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions
In this paper, we systematically study the impact of various self-supervised learning proxy tasks on different architectures and threat models for 3D point clouds with adversarial training.
AugMax: Adversarial Composition of Random Augmentations for Robust Training
Diversity and hardness are two complementary dimensions of data augmentation to achieve robustness.
Auditing AI models for Verified Deployment under Semantic Specifications
We enable such unit tests through variations in a semantically-interpretable latent space of a generative model.
Long-Short Transformer: Efficient Transformers for Language and Vision
For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.
Ranked #1 on
Language Modelling
on enwik8 dev
Taxonomy of Machine Learning Safety: A Survey and Primer
The open-world deployment of Machine Learning (ML) algorithms in safety-critical applications such as autonomous vehicles needs to address a variety of ML vulnerabilities such as interpretability, verifiability, and performance limitations.
Can Shape Structure Features Improve Model Robustness Under Diverse Adversarial Settings?
Specifically, EdgeNetRob and EdgeGANRob first explicitly extract shape structure features from a given image via an edge detection algorithm.
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
Several works have shown this vulnerability via adversarial attacks, but existing approaches on improving the robustness of DRL under this setting have limited success and lack for theoretical principles.
AdvIT: Adversarial Frames Identifier Based on Temporal Consistency in Videos
In particular, we apply optical flow estimation to the target and previous frames to generate pseudo frames and evaluate the consistency of the learner output between these pseudo frames and target.
Characterizing Attacks on Deep Reinforcement Learning
To the best of our knowledge, we are the first to apply adversarial attacks on DRL systems to physical robots.
Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving
In contrast to prior work that concentrates on camera-based perception, in this work we perform the first security study of LiDAR-based perception in AV settings, which is highly important but unexplored.
Adversarial Objects Against LiDAR-Based Autonomous Driving Systems
Deep neural networks (DNNs) are found to be vulnerable against adversarial examples, which are carefully crafted inputs with a small magnitude of perturbation aiming to induce arbitrarily incorrect predictions.
SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples".
Towards Stable and Efficient Training of Verifiably Robust Neural Networks
In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass.
Application-driven Privacy-preserving Data Publishing with Correlated Attributes
To address the privacy concerns of users in this environment, we propose a novel framework called PR-GAN that offers privacy-preserving mechanism using generative adversarial networks.
Data Poisoning Attack against Unsupervised Node Embedding Methods
In this paper, we take the task of link prediction as an example, which is one of the most fundamental problems for graph analysis, and introduce a data positioning attack to node embedding methods.
MeshAdv: Adversarial Meshes for Visual Recognition
Highly expressive models such as deep neural networks (DNNs) have been widely applied to various applications.
Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation
In this paper, we aim to characterize adversarial examples based on spatial context information in semantic segmentation.
Robust Physical-World Attacks on Deep Learning Visual Classification
Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input.
Performing Co-Membership Attacks Against Deep Generative Models
We conduct extensive experiments on a variety of datasets and generative models showing that: our attacker network outperforms prior membership attacks; co-membership attacks can be substantially more powerful than single attacks; and VAEs are more susceptible to membership attacks compared to GANs.
Generating Adversarial Examples with Adversarial Networks
A challenge to explore adversarial robustness of neural networks on MNIST.
Spatially Transformed Adversarial Examples
Perturbations generated through spatial transformation could result in large $\mathcal{L}_p$ distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems.
Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features
A conventional approach to evaluate ML robustness to such attacks, as well as to design robust ML, is by considering simplified feature-space models of attacks, where the attacker changes ML features directly to effect evasion, while minimizing or constraining the magnitude of this change.
Robust Physical-World Attacks on Deep Learning Models
We propose a general attack algorithm, Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions.
