Search Results for author:
Found 35 papers, 14 papers with code
Machine Translation Aided Bilingual Data-to-Text Generation and Semantic Parsing
We present a system for bilingual Data-ToText Generation and Semantic Parsing.
Scaling Motion Forecasting Models with Ensemble Distillation
These experiments demonstrate distillation from ensembles as an effective method for improving accuracy of predictive models for robotic systems with limited compute budgets.
Let Your Graph Do the Talking: Encoding Structured Data for LLMs
How can we best encode structured data into sequential form for use in large language models (LLMs)?
MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Here, we represent continuous trajectories as sequences of discrete motion tokens and cast multi-agent motion prediction as a language modeling task over this domain.
Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization
Most of previous work on learning diacritization of the Arabic language relied on training models from scratch.
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks
In this paper, we present Wayformer, a family of attention based architectures for motion forecasting that are simple and homogeneous.
Ranked #6 on
Motion Forecasting
on Argoverse CVPR 2020
Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting
These models are intrinsically invariant to translation and rotation between scene elements, are best-performing on public leaderboards, but scale quadratically with the number of agents and scene elements.
VN-Transformer: Rotation-Equivariant Attention for Vector Neurons
Rotation equivariance is a desirable property in many practical applications such as motion forecasting and 3D perception, where it can offer benefits like sample efficiency, better generalization, and robustness to input perturbations.
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task.
nmT5 - Is parallel data still relevant for pre-training massively multilingual language models?
Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks.
nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?
Recently, mT5 - a massively multilingual version of T5 - leveraged a unified text-to-text format to attain state-of-the-art results on a wide variety of multilingual NLP tasks.
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Most widely-used pre-trained language models operate on sequences of tokens corresponding to word or subword units.
Ranked #1 on
Cross-Lingual Natural Language Inference
on XNLI
Cross-Lingual Natural Language Inference
Cross-Lingual NER
+3
The Power of Scale for Parameter-Efficient Prompt Tuning
More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned).
Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training
Prior work on Data-To-Text Generation, the task of converting knowledge graph (KG) triples into natural text, focused on domain-specific benchmark datasets.
mT5: A massively multilingual pre-trained text-to-text transformer
The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks.
Ranked #2 on
Reading Comprehension
on MuSeRC
Wiki-40B: Multilingual Language Model Dataset
We released the cleaned-up text of 40+ Wikipedia language editions, the corresponding trained monolingual language models, and several multilingual language models with different fixed vocabulary sizes.
LAReQA: Language-agnostic answer retrieval from a multilingual pool
We present LAReQA, a challenging new benchmark for language-agnostic answer retrieval from a multilingual candidate pool.
Bridging the Gap for Tokenizer-Free Language Models
Purely character-based language models (LMs) have been lagging in quality on large scale datasets, and current state-of-the-art LMs rely on word tokenization.
DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
Second, for each pair of graphs, we train a cross-graph attention network which uses the node representations of an anchor graph to reconstruct another graph.
Ranked #3 on
Graph Classification
on D&D
Character-Level Language Modeling with Deeper Self-Attention
LSTMs and other RNN variants have shown strong performance on character-level language modeling.
Ranked #8 on
Language Modelling
on Hutter Prize
A Tutorial on Network Embeddings
We further demonstrate the applications of network embeddings, and conclude the survey with future work in this area.
Social and Information Networks
Watch Your Step: Learning Node Embeddings via Graph Attention
Graph embedding methods represent nodes in a continuous vector space, preserving information from the graph (e. g. by sampling random walks).
Ranked #69 on
Node Classification
on Citeseer
CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks
Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology.
Learning Edge Representations via Low-Rank Asymmetric Projections
Individually, both of these contributions improve the learned representations, especially when there are memory constraints on the total size of the embeddings.
Efficient Natural Language Response Suggestion for Smart Reply
This paper presents a computationally efficient machine-learned method for natural language response suggestion.
Visualizing Linguistic Shift
Neural network based models are a very powerful tool for creating word embeddings, the objective of these models is to group similar words together.
A Growing Long-term Episodic & Semantic Memory
The long-term memory of most connectionist systems lies entirely in the weights of the system.
Conversational Contextual Cues: The Case of Personalization and History for Response Ranking
Unlike previous efforts, which focused on modeling messages and responses, we extend the modeling to long context and participant's history.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Statistically Significant Detection of Linguistic Change
We propose a new computational approach for tracking and detecting statistically significant linguistic shifts in the meaning and usage of words.
POLYGLOT-NER: Massive Multilingual Named Entity Recognition
We describe a system that builds Named Entity Recognition (NER) annotators for 40 major languages using Wikipedia and Freebase.
DeepWalk: Online Learning of Social Representations
We present DeepWalk, a novel approach for learning latent representations of vertices in a network.
Ranked #1 on
Link Property Prediction
on ogbl-ppa
Inducing Language Networks from Continuous Space Word Representations
Recent advancements in unsupervised feature learning have developed powerful latent representations of words.
Polyglot: Distributed Word Representations for Multilingual NLP
We quantitatively demonstrate the utility of our word embeddings by using them as the sole features for training a part of speech tagger for a subset of these languages.
The Expressive Power of Word Embeddings
We seek to better understand the difference in quality of the several publicly released embeddings.
