1 code implementation • 20 Feb 2024 • Xiangyu Liu, ChengHao Deng, Yanchao Sun, Yongyuan Liang, Furong Huang
In light of the burgeoning success of reinforcement learning (RL) in diverse real-world applications, considerable focus has been directed towards ensuring RL policies are robust to adversarial attacks during test time.
1 code implementation • 5 Feb 2024 • Yuancheng Xu, Jiarui Yao, Manli Shu, Yanchao Sun, Zichu Wu, Ning Yu, Tom Goldstein, Furong Huang
We show that Shadowcast are highly effective in achieving attacker's intentions using as few as 50 poison samples.
no code implementations • 22 Oct 2023 • Yuchen Xiao, Yanchao Sun, Mengda Xu, Udari Madhushani, Jared Vann, Deepeka Garg, Sumitra Ganesh
Recent advancements in large language models (LLMs) have exhibited promising performance in solving sequential decision-making problems.
no code implementations • 12 Oct 2023 • Aakriti Agrawal, Rohith Aralikatti, Yanchao Sun, Furong Huang
This work is the first to formulate the generalised problem of robustness to multi-modal environment uncertainty in MARL.
no code implementations • 11 Oct 2023 • Xiyao Wang, Ruijie Zheng, Yanchao Sun, Ruonan Jia, Wichayaporn Wongkamjan, Huazhe Xu, Furong Huang
In this paper, we propose $\texttt{COPlanner}$, a planning-driven framework for model-based methods to address the inaccurately learned dynamics model problem with conservative model rollouts and optimistic environment exploration.
no code implementations • NeurIPS 2023 • Sili Huang, Yanchao Sun, Jifeng Hu, Siyuan Guo, Hechang Chen, Yi Chang, Lichao Sun, Bo Yang
Our experimental results demonstrate that SGFD can generalize well on a wide range of test environments and significantly outperforms state-of-the-art methods in handling both task-irrelevant variations and task-relevant variations.
2 code implementations • 7 Sep 2023 • Yuancheng Xu, ChengHao Deng, Yanchao Sun, Ruijie Zheng, Xiyao Wang, Jieyu Zhao, Furong Huang
To address biases in sequential decision-making, we introduce a long-term fairness concept named Equal Long-term Benefit Rate (ELBERT).
no code implementations • 22 Jul 2023 • Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Xiangyu Liu, Benjamin Eysenbach, Tuomas Sandholm, Furong Huang, Stephen Mcaleer
To tackle this challenge, we propose GRAD, a novel game-theoretic approach that treats the temporally-coupled robust RL problem as a partially observable two-player zero-sum game.
1 code implementation • ICCV 2023 • Yao Wei, Yanchao Sun, Ruijie Zheng, Sai Vemprala, Rogerio Bonatti, Shuhang Chen, Ratnesh Madaan, Zhongjie Ba, Ashish Kapoor, Shuang Ma
We introduce DualMind, a generalist agent designed to tackle various decision-making tasks that addresses challenges posed by current methods, such as overfitting behaviors and dependence on task-specific fine-tuning.
1 code implementation • 22 Jun 2023 • Ruijie Zheng, Xiyao Wang, Yanchao Sun, Shuang Ma, Jieyu Zhao, Huazhe Xu, Hal Daumé III, Furong Huang
Despite recent progress in reinforcement learning (RL) from raw pixel data, sample inefficiency continues to present a substantial obstacle.
no code implementations • 13 Jun 2023 • Siyuan Guo, Yanchao Sun, Jifeng Hu, Sili Huang, Hechang Chen, Haiyin Piao, Lichao Sun, Yi Chang
However, constrained by the limited quality of the offline dataset, its performance is often sub-optimal.
no code implementations • 8 Jun 2023 • Jifeng Hu, Yanchao Sun, Sili Huang, Siyuan Guo, Hechang Chen, Li Shen, Lichao Sun, Yi Chang, DaCheng Tao
Recent works have shown the potential of diffusion models in computer vision and natural language processing.
no code implementations • 27 May 2023 • Xiangyu Liu, Souradip Chakraborty, Yanchao Sun, Furong Huang
To address these limitations, we introduce a generalized attack framework that has the flexibility to model to what extent the adversary is able to control the agent, and allows the attacker to regulate the state distribution shift and produce stealthier adversarial policies.
2 code implementations • 6 Feb 2023 • Yuancheng Xu, Yanchao Sun, Micah Goldblum, Tom Goldstein, Furong Huang
However, it is unclear whether existing robust training methods effectively increase the margin for each vulnerable point during training.
no code implementations • 24 Jan 2023 • Yanchao Sun, Shuang Ma, Ratnesh Madaan, Rogerio Bonatti, Furong Huang, Ashish Kapoor
Self-supervised pretraining has been extensively studied in language and vision domains, where a unified model can be easily adapted to various downstream tasks by pretraining representations without explicit labels.
no code implementations • 22 Jan 2023 • Lilin Zhang, Ning Yang, Yanchao Sun, Philip S. Yu
Second, the existing AT methods often achieve adversarial robustness at the expense of standard generalizability (i. e., the accuracy on natural examples) because they make a tradeoff between them.
1 code implementation • 2 Nov 2022 • Kaiwen Yang, Yanchao Sun, Jiahao Su, Fengxiang He, Xinmei Tian, Furong Huang, Tianyi Zhou, DaCheng Tao
In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e. g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly.
1 code implementation • 14 Oct 2022 • Jifeng Hu, Yanchao Sun, Hechang Chen, Sili Huang, Haiyin Piao, Yi Chang, Lichao Sun
Our main idea is to design the multi-action-branch reward estimation and policy-weighted reward aggregation for stabilized training.
Multi-agent Reinforcement Learning reinforcement-learning +1
1 code implementation • 12 Oct 2022 • Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Furong Huang
Recent studies reveal that a well-trained deep reinforcement learning (RL) policy can be particularly vulnerable to adversarial perturbations on input observations.
no code implementations • 21 Jun 2022 • Yanchao Sun, Ruijie Zheng, Parisa Hassanzadeh, Yongyuan Liang, Soheil Feizi, Sumitra Ganesh, Furong Huang
Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions.
no code implementations • ICLR 2022 • Yanchao Sun, Ruijie Zheng, Xiyao Wang, Andrew Cohen, Furong Huang
In many reinforcement learning (RL) applications, the observation space is specified by human developers and restricted by physical realizations, and may thus be subject to dramatic changes over time (e. g. increased number of observable features).
1 code implementation • ICLR 2022 • Yanchao Sun, Ruijie Zheng, Yongyuan Liang, Furong Huang
Existing works on adversarial RL either use heuristics-based methods that may not find the strongest adversary, or directly train an RL-based adversary by treating the agent as a part of the environment, which can find the optimal adversary but may become intractable in a large state space.
no code implementations • ICLR 2021 • Yanchao Sun, Da Huo, Furong Huang
Poisoning attacks on Reinforcement Learning (RL) systems could take advantage of RL algorithm's vulnerabilities and cause failure of the learning.
no code implementations • 16 Feb 2020 • Yanchao Sun, Xiangyu Yin, Furong Huang
Transferring knowledge among various environments is important to efficiently learn multiple tasks online.
no code implementations • 14 Jan 2020 • Jingling Li, Yanchao Sun, Jiahao Su, Taiji Suzuki, Furong Huang
Recently proposed complexity measures have provided insights to understanding the generalizability in neural networks from perspectives of PAC-Bayes, robustness, overparametrization, compression and so on.
1 code implementation • 21 Dec 2019 • Yanchao Sun, Furong Huang
We propose a new model-based method called Greedy Inference Model (GIM) that infers the unknown dynamics from known dynamics based on the internal spectral properties of the environment.
Model-based Reinforcement Learning reinforcement-learning +1