Search Results for author:
Found 29 papers, 14 papers with code
Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations
In this work, we propose the Scene Representation Transformer (SRT), a method which processes posed or unposed RGB images of a new area, infers a "set-latent scene representation", and synthesises novel views, all in a single feed-forward pass.
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers
Vision Transformers (ViT) have been shown to attain highly competitive performance for a wide range of vision applications, such as image classification, object detection and semantic image segmentation.
MLP-Mixer: An all-MLP Architecture for Vision
Convolutional Neural Networks (CNNs) are the go-to model for computer vision.
Ranked #17 on
Image Classification
on OmniBenchmark
Differentiable Patch Selection for Image Recognition
Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited.
Ranked #1 on
Image Classification
on CIFAR-10
Towards End-to-End In-Image Neural Machine Translation
In this paper, we offer a preliminary investigation into the task of in-image machine translation: transforming an image containing text in one language into an image containing the same text in another language.
Object-Centric Learning with Slot Attention
Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features.
An Empirical Study of Generation Order for Machine Translation
In this work, we present an empirical study of generation order for machine translation.
Scaling Autoregressive Video Models
Due to the statistical complexity of video, the high degree of inherent stochasticity, and the sheer amount of data, generating natural video remains a challenging task.
Ranked #7 on
Video Generation
on BAIR Robot Pushing
KERMIT: Generative Insertion-Based Modeling for Sequences
During training, one can feed KERMIT paired data $(x, y)$ to learn the joint distribution $p(x, y)$, and optionally mix in unpaired data $x$ or $y$ to refine the marginals $p(x)$ or $p(y)$.
Ranked #39 on
Machine Translation
on WMT2014 English-German
Natural Questions: a Benchmark for Question Answering Research
The public release consists of 307, 373 training examples with single annotations, 7, 830 examples with 5-way annotations for development data, and a further 7, 842 examples 5-way annotated sequestered as test data.
Ranked #7 on
Question Answering
on Natural Questions (long)
Insertion Transformer: Flexible Sequence Generation via Insertion Operations
We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations.
Blockwise Parallel Decoding for Deep Autoregressive Models
Deep autoregressive sequence-to-sequence models have demonstrated impressive performance across a wide variety of tasks in recent years.
Music Transformer
This is impractical for long sequences such as musical compositions since their memory complexity for intermediate relative information is quadratic in the sequence length.
Ranked #3 on
Music Modeling
on JSB Chorales
Universal Transformers
Feed-forward and convolutional architectures have recently been shown to achieve superior results on some sequence modeling tasks such as machine translation, with the added advantage that they concurrently process all inputs in the sequence, leading to easy parallelization and faster training times.
Ranked #30 on
Language Modelling
on LAMBADA
Tensor2Tensor for Neural Machine Translation
Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.
Fast Decoding in Sequence Models using Discrete Latent Variables
Finally, we evaluate our model end-to-end on the task of neural machine translation, where it is an order of magnitude faster at decoding than comparable autoregressive models.
Self-Attention with Relative Position Representations
On the WMT 2014 English-to-German and English-to-French translation tasks, this approach yields improvements of 1. 3 BLEU and 0. 3 BLEU over absolute position representations, respectively.
Ranked #22 on
Machine Translation
on WMT2014 English-French
Image Transformer
Image generation has been successfully cast as an autoregressive sequence generation or transformation problem.
Ranked #3 on
Density Estimation
on CIFAR-10
Large Scale Multi-Domain Multi-Task Learning with MultiModel
We present a single model that yields good results on a number of problems spanning multiple domains.
Coarse-to-Fine Question Answering for Long Documents
We present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models.
One Model To Learn Them All
We present a single model that yields good results on a number of problems spanning multiple domains.
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration.
Ranked #2 on
Multimodal Machine Translation
on Multi30K
(BLUE (DE-EN) metric)
Neural Paraphrase Identification of Questions with Noisy Pretraining
We present a solution to the problem of paraphrase identification of questions.
Ranked #15 on
Paraphrase Identification
on Quora Question Pairs
(Accuracy metric)
Hierarchical Question Answering for Long Documents
We present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models.
A Decomposable Attention Model for Natural Language Inference
We propose a simple neural architecture for natural language inference.
Ranked #47 on
Natural Language Inference
on SNLI
