Search Results for author:
Found 35 papers, 13 papers with code
The DeepMind Chinese–English Document Translation System at WMT2020
This paper describes the DeepMind submission to the Chinese\rightarrowEnglish constrained data track of the WMT2020 Shared Task on News Translation.
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment.
Optimizing Memory Mapping Using Deep Reinforcement Learning
We also introduce a Reinforcement Learning agent, mallocMuZero, and show that it is capable of playing this game to discover new and improved memory mapping solutions that lead to faster execution times on real ML workloads on ML accelerators.
Improving alignment of dialogue agents via targeted human judgements
We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines.
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Large language models produce human-like text that drive a growing number of applications.
Competition-Level Code Generation with AlphaCode
Programming is a powerful and ubiquitous problem-solving tool.
Ranked #1 on
Code Generation
on CodeContests
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Language Modelling
on StackExchange
Ethical and social risks of harm from Language Models
We discuss the points of origin of different risks and point to potential mitigation approaches.
Challenges in Detoxifying Language Models
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks.
Self-supervised Adversarial Robustness for the Low-label, High-data Regime
Experiments on CIFAR-10 against $\ell_2$ and $\ell_\infty$ norm-bounded perturbations demonstrate that BYORL achieves near state-of-the-art robustness with as little as 500 labeled examples.
Towards Verified Robustness under Text Deletion Interventions
Neural networks are widely used in Natural Language Processing, yet despite their empirical successes, their behaviour is brittle: they are both over-sensitive to small input changes, and under-sensitive to deletions of large fractions of input text.
Achieving Robustness in the Wild via Adversarial Mixing with Disentangled Representations
Specifically, we leverage the disentangled latent representations computed by a StyleGAN model to generate perturbations of an image that are similar to real-world variations (like adding make-up, or changing the skin-tone of a person) and train models to be invariant to these perturbations.
Towards Robust Image Classification Using Sequential Attention Models
In this paper we propose to augment a modern neural-network architecture with an attention model inspired by human perception.
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text.
Learning Transferable Graph Exploration
We propose a `learning to explore' framework where we learn a policy from a distribution of environments.
An Alternative Surrogate Loss for PGD-based Adversarial Testing
Adversarial testing methods based on Projected Gradient Descent (PGD) are widely used for searching norm-bounded perturbations that cause the inputs of neural networks to be misclassified.
Scalable Neural Learning for Verifiable Consistency with Temporal Specifications
Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Neural networks are part of many contemporary NLP systems, yet their empirical successes come at the price of vulnerability to adversarial attacks.
Are Labels Required for Improving Adversarial Robustness?
Recent work has uncovered the interesting (and somewhat surprising) finding that training models to be invariant to adversarial perturbations requires substantially larger datasets than those required for standard classification.
Knowing When to Stop: Evaluation and Verification of Conformity to Output-size Specifications
This behavior can have severe consequences such as usage of increased computation and induce faults in downstream modules that expect outputs of a certain length.
Neural Phrase-to-Phrase Machine Translation
We also design an efficient dynamic programming algorithm to decode segments that allows the model to be trained faster than the existing neural phrase-based machine translation method by Huang et al. (2018).
Robust Text-to-SQL Generation with Execution-Guided Decoding
We consider the problem of neural semantic parsing, which translates natural language questions into executable SQL queries.
Discourse-Aware Neural Rewards for Coherent Text Generation
In this paper, we investigate the use of discourse-aware rewards with reinforcement learning to guide a model to generate long, coherent text.
Natural Language to Structured Query Generation via Meta-Learning
In conventional supervised training, a model is trained to fit all the training examples.
Ranked #7 on
Code Generation
on WikiSQL
M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search
In order to effectively train the agent from sparse rewards, we combine MCTS with the neural policy to generate trajectories yielding more positive rewards.
Ranked #44 on
Link Prediction
on WN18RR
(Hits@3 metric)
Modeling Large-Scale Structured Relationships with Shared Memory for Knowledge Base Completion
However, due to the size of knowledge bases, learning multi-step relations directly on top of observed triplets could be costly.
Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension
We develop a technique for transfer learning in machine comprehension (MC) using a novel two-stage synthesis network (SynNet).
Towards Neural Phrase-based Machine Translation
In this paper, we present Neural Phrase-based Machine Translation (NPMT).
Ranked #7 on
Machine Translation
on IWSLT2015 English-German
Sequence Modeling via Segmentations
The probability of a segmented sequence is calculated as the product of the probabilities of all its segments, where each segment is modeled using existing tools such as recurrent neural networks.
Link Prediction using Embedded Knowledge Graphs
Since large knowledge bases are typically incomplete, missing facts need to be inferred from observed facts in a task called knowledge base completion.
ReasoNet: Learning to Stop Reading in Machine Comprehension
Teaching a computer to read and answer general questions pertaining to a document is a challenging yet unsolved problem.
Ranked #7 on
Question Answering
on CNN / Daily Mail
Unsupervised Learning of Predictors from Unpaired Input-Output Samples
In particular, we show that with regularization via a generative model, learning with the proposed unsupervised objective function converges to an optimal solution.
Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation
In this paper, we explore joint optimization of masking functions and deep recurrent neural networks for monaural source separation tasks, including monaural speech separation, monaural singing voice separation, and speech denoising.
Deep learning for monaural speech separation
In this paper, we study deep learning for monaural speech separation.
Learning deep structured semantic models for web search using clickthrough data
The proposed deep structured semantic models are discriminatively trained by maximizing the conditional likelihood of the clicked documents given a query using the clickthrough data.
