Search Results for author: Yanchao Sun

Found 26 papers, 11 papers with code

Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies

1 code implementation20 Feb 2024 Xiangyu Liu, ChengHao Deng, Yanchao Sun, Yongyuan Liang, Furong Huang

In light of the burgeoning success of reinforcement learning (RL) in diverse real-world applications, considerable focus has been directed towards ensuring RL policies are robust to adversarial attacks during test time.

Adversarial Attack Reinforcement Learning (RL)

Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learning

no code implementations12 Oct 2023 Aakriti Agrawal, Rohith Aralikatti, Yanchao Sun, Furong Huang

This work is the first to formulate the generalised problem of robustness to multi-modal environment uncertainty in MARL.

Multi-agent Reinforcement Learning

COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL

no code implementations11 Oct 2023 Xiyao Wang, Ruijie Zheng, Yanchao Sun, Ruonan Jia, Wichayaporn Wongkamjan, Huazhe Xu, Furong Huang

In this paper, we propose $\texttt{COPlanner}$, a planning-driven framework for model-based methods to address the inaccurately learned dynamics model problem with conservative model rollouts and optimistic environment exploration.

Continuous Control Model-based Reinforcement Learning +1

Learning Generalizable Agents via Saliency-Guided Features Decorrelation

no code implementations NeurIPS 2023 Sili Huang, Yanchao Sun, Jifeng Hu, Siyuan Guo, Hechang Chen, Yi Chang, Lichao Sun, Bo Yang

Our experimental results demonstrate that SGFD can generalize well on a wide range of test environments and significantly outperforms state-of-the-art methods in handling both task-irrelevant variations and task-relevant variations.

Reinforcement Learning (RL)

Equal Long-term Benefit Rate: Adapting Static Fairness Notions to Sequential Decision Making

1 code implementation7 Sep 2023 Yuancheng Xu, ChengHao Deng, Yanchao Sun, Ruijie Zheng, Xiyao Wang, Jieyu Zhao, Furong Huang

Moreover, we show that the policy gradient of Long-term Benefit Rate can be analytically reduced to standard policy gradient.

Decision Making Fairness

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

no code implementations22 Jul 2023 Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Xiangyu Liu, Benjamin Eysenbach, Tuomas Sandholm, Furong Huang, Stephen Mcaleer

To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game.

Continuous Control reinforcement-learning +1

Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training

1 code implementation ICCV 2023 Yao Wei, Yanchao Sun, Ruijie Zheng, Sai Vemprala, Rogerio Bonatti, Shuhang Chen, Ratnesh Madaan, Zhongjie Ba, Ashish Kapoor, Shuang Ma

We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning.

Decision Making

TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning

1 code implementation22 Jun 2023 Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daumé III, Furong Huang

Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle.

Continuous Control Contrastive Learning +3

Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RL

no code implementations27 May 2023 Xiangyu Liu, Souradip Chakraborty, Yanchao Sun, Furong Huang

To address these limitations, we introduce a generalized attack framework that has the flexibility to model to what extent the adversary is able to control the agent, and allows the attacker to regulate the state distribution shift and produce stealthier adversarial policies.

Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness

2 code implementations6 Feb 2023 Yuancheng Xu, Yanchao Sun, Micah Goldblum, Tom Goldstein, Furong Huang

However, it is unclear whether existing robust training methods effectively increase the margin for each vulnerable point during training.

Adversarial Robustness

SMART: Self-supervised Multi-task pretrAining with contRol Transformers

no code implementations24 Jan 2023 Yanchao Sun, Shuang Ma, Ratnesh Madaan, Rogerio Bonatti, Furong Huang, Ashish Kapoor

Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels.

Imitation Learning Reinforcement Learning (RL)

Provable Unrestricted Adversarial Training without Compromise with Generalizability

no code implementations22 Jan 2023 Lilin Zhang, Ning Yang, Yanchao Sun, Philip S. Yu

Second, the existing AT methods often achieve adversarial robustness at the expense of standard generalizability (i. e., the accuracy on natural examples) because they make a tradeoff between them.

Adversarial Robustness

Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach

1 code implementation2 Nov 2022 Kaiwen Yang, Yanchao Sun, Jiahao Su, Fengxiang He, Xinmei Tian, Furong Huang, Tianyi Zhou, DaCheng Tao

In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e. g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly.

Data Augmentation Representation Learning

Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning

1 code implementation12 Oct 2022 Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Furong Huang

Recent studies reveal that a well-trained deep reinforcement learning (RL) policy can be particularly vulnerable to adversarial perturbations on input observations.

reinforcement-learning Reinforcement Learning (RL)

Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems

no code implementations21 Jun 2022 Yanchao Sun, Ruijie Zheng, Parisa Hassanzadeh, Yongyuan Liang, Soheil Feizi, Sumitra Ganesh, Furong Huang

Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions.

Multi-agent Reinforcement Learning

Transfer RL across Observation Feature Spaces via Model-Based Regularization

no code implementations ICLR 2022 Yanchao Sun, Ruijie Zheng, Xiyao Wang, Andrew Cohen, Furong Huang

In many reinforcement learning (RL) applications, the observation space is specified by human developers and restricted by physical realizations, and may thus be subject to dramatic changes over time (e. g. increased number of observable features).

Reinforcement Learning (RL)

Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL

1 code implementation ICLR 2022 Yanchao Sun, Ruijie Zheng, Yongyuan Liang, Furong Huang

Existing works on adversarial RL either use heuristics-based methods that may not find the strongest adversary, or directly train an RL-based adversary by treating the agent as a part of the environment, which can find the optimal adversary but may become intractable in a large state space.

Reinforcement Learning (RL)

Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics

no code implementations ICLR 2021 Yanchao Sun, Da Huo, Furong Huang

Poisoning attacks on Reinforcement Learning (RL) systems could take advantage of RL algorithm's vulnerabilities and cause failure of the learning.

Reinforcement Learning (RL)

TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL

no code implementations16 Feb 2020 Yanchao Sun, Xiangyu Yin, Furong Huang

Transferring knowledge among various environments is important to efficiently learn multiple tasks online.

Understanding Generalization in Deep Learning via Tensor Methods

no code implementations14 Jan 2020 Jingling Li, Yanchao Sun, Jiahao Su, Taiji Suzuki, Furong Huang

Recently proposed complexity measures have provided insights to understanding the generalizability in neural networks from perspectives of PAC-Bayes, robustness, overparametrization, compression and so on.

Can Agents Learn by Analogy? An Inferable Model for PAC Reinforcement Learning

1 code implementation21 Dec 2019 Yanchao Sun, Furong Huang

We propose a new model-based method called Greedy Inference Model (GIM) that infers the unknown dynamics from known dynamics based on the internal spectral properties of the environment.

Model-based Reinforcement Learning reinforcement-learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.