On Lp-norm Robustness of Ensemble Decision Stumps and Trees

Recent papers have demonstrated that ensemble stumps and trees could be vulnerable to small input perturbations, so robustness verification and defense for those models have become an important research problem. However, due to the structure of decision trees, where each node makes decision purely based on one feature value, all the previous works only consider the $\ell_\infty$ norm perturbation. To study robustness with respect to a general $\ell_p$ norm perturbation, one has to consider correlation between perturbations on different features, which has not been handled by previous algorithms. In this paper, we study the robustness verification and defense with respect to general $\ell_p$ norm perturbation for ensemble trees and stumps. For robustness verification, we prove that exact verification is NP-complete for $p\in(0, \infty)$ while polynomial time algorithms exist for $p=0$ or $\infty$. Approximation algorithms based on dynamic programming is then developed for verifying ensemble trees and stumps. For robustness training, we propose the first certified defense method for training ensemble stumps and trees with respect to $\ell_p$ norm perturbations. The effectiveness of proposed algorithms is verified empirically on real datasets.