Search Results for author:
Found 12 papers, 7 papers with code
XGen-7B Technical Report
Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context.
Don’t throw away that linear head: Few-shot protein fitness prediction with generative models
Predicting the fitness, i. e. functional value, of a protein sequence is an important and challenging task in biology, particularly due to the scarcity of assay-labeled data.
Deep Extrapolation for Attribute-Enhanced Generation
Attribute extrapolation in sample generation is challenging for deep neural networks operating beyond the training distribution.
Explaining and Improving Model Behavior with k Nearest Neighbor Representations
Interpretability techniques in NLP have mainly focused on understanding individual predictions using attention visualization or gradient-based saliency maps over tokens.
GeDi: Generative Discriminator Guided Sequence Generation
While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate.
Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width
We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width.
Dynamic Evaluation of Transformer Language Models
This research note combines two methods that have recently improved the state of the art in language modeling: Transformers and dynamic evaluation.
Ranked #1 on
Language Modelling
on Hutter Prize
Talking to myself: self-dialogues as data for conversational agents
Conversational agents are gaining popularity with the increasing ubiquity of smart devices.
Edina: Building an Open Domain Socialbot with Self-dialogues
We present Edina, the University of Edinburgh's social bot for the Amazon Alexa Prize competition.
Dynamic Evaluation of Neural Sequence Models
We present methodology for using dynamic evaluation to improve neural sequence models.
Ranked #10 on
Language Modelling
on Hutter Prize
Multiplicative LSTM for sequence modelling
We introduce multiplicative LSTM (mLSTM), a recurrent neural network architecture for sequence modelling that combines the long short-term memory (LSTM) and multiplicative recurrent neural network architectures.
Ranked #14 on
Language Modelling
on Hutter Prize
Optimizing and Contrasting Recurrent Neural Network Architectures
Recurrent Neural Networks (RNNs) have long been recognized for their potential to model complex time series.
