Sparsemax

Introduced by Martins et al. in From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

Sparsemax is a type of activation/output function similar to the traditional softmax, but able to output sparse probabilities.

$$ \text{sparsemax}\left(z\right) = \arg_{p∈\Delta^{K−1}}\min||\mathbf{p} - \mathbf{z}||^{2} $$

Source: From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

Read Paper See Code

Paper	Code	Results	Date	Stars

Task	Papers	Share
Machine Translation	4	8.33%
Translation	4	8.33%
Language Modelling	3	6.25%
General Classification	3	6.25%
Question Answering	2	4.17%
Visual Question Answering	2	4.17%
Visual Question Answering (VQA)	2	4.17%
Morphological Inflection	2	4.17%
NER	1	2.08%

This feature is experimental; we are continuously improving our matching algorithm.

Component	Type	Add Remove
🤖 No Components Found	You can add them if they exist; e.g. Mask R-CNN uses RoIAlign