Transformers

Universal Transformer

Introduced by Dehghani et al. in Universal Transformers

The Universal Transformer is a generalization of the Transformer architecture. Universal Transformers combine the parallelizability and global receptive field of feed-forward sequence models like the Transformer with the recurrent inductive bias of RNNs. They also utilise a dynamic per-position halting mechanism.

Source: Universal Transformers

Papers


Paper Code Results Date Stars

Categories