Neural Attention Forests: Transformer-Based Forest Improvement

1 code implementation • 12 Apr 2023 • Andrei V. Konstantinov, Lev V. Utkin, Alexey A. Lukashin, Vladimir A. Muliukha

The main idea behind the proposed NAF model is to introduce the attention mechanism into the random forest by assigning attention weights calculated by neural networks of a specific form to data in leaves of decision trees and to the random forest itself in the framework of the Nadaraya-Watson kernel regression.

regression