Search Results for author:
Found 32 papers, 20 papers with code
Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model
While various models and computational tools have been proposed for structure and property analysis of molecules, generating molecules that conform to all desired structures and properties remains a challenge.
CrossBind: Collaborative Cross-Modal Identification of Protein Nucleic-Acid-Binding Residues
Accurate identification of protein nucleic-acid-binding residues poses a significant challenge with important implications for various biological processes and drug design.
ContraNovo: A Contrastive Learning Approach to Enhance De Novo Peptide Sequencing
De novo peptide sequencing from mass spectrometry (MS) data is a critical task in proteomics research.
Accurate Prediction of Antibody Function and Structure Using Bio-Inspired Antibody Language Model
Remarkably, BALMFold outperforms those well-established methods like AlphaFold2, IgFold, ESMFold, and OmegaFold in the antibody benchmark, demonstrating significant potential to advance innovative engineering and streamline therapeutic antibody development by reducing the need for unnecessary trials.
Enhancing the Protein Tertiary Structure Prediction by Multiple Sequence Alignment Generation
The field of protein folding research has been greatly advanced by deep learning methods, with AlphaFold2 (AF2) demonstrating exceptional performance and atomic-level precision.
AF2-Mutation: Adversarial Sequence Mutations against AlphaFold2 on Protein Tertiary Structure Prediction
Deep learning-based approaches, such as AlphaFold2 (AF2), have significantly advanced protein tertiary structure prediction, achieving results comparable to real biological experimental methods.
E2Efold-3D: End-to-End Deep Learning Method for accurate de novo RNA 3D Structure Prediction
When E2Efold-3D is coupled with the experimental techniques, the RNA structure prediction field can be greatly advanced.
Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions
Non-coding RNA structure and function are essential to understanding various biological processes, such as cell signaling, gene expression, and post-transcriptional regulations.
Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
Surprisingly, we found that REtrieving from the traINing datA (REINA) only can lead to significant gains on multiple NLG and NLU tasks.
Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
In particular, we focus on the task of Commonsense Reasoning, demonstrating that the proposed external attention mechanism can augment existing transformer models and significantly improve the model's reasoning capabilities.
Ranked #1 on
Common Sense Reasoning
on CommonsenseQA
(using extra training data)
Leveraging Knowledge in Multilingual Commonsense Reasoning
Then we utilize a diverse of 4 English knowledge sources to provide more comprehensive coverage of knowledge in different formats.
RetGen: A Joint framework for Retrieval and Grounded Text Generation Modeling
We propose a framework that alleviates this data constraint by jointly training a grounded generator and document retriever on the language model signal.
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
Multimodal pre-training has propelled great advancement in vision-and-language research.
Cluster-Former: Clustering-based Sparse Transformer for Question Answering
Transformer has become ubiquitous in the deep learning field.
Cross-Thought for Sentence Encoder Pre-training
In this paper, we propose Cross-Thought, a novel approach to pre-training sequence encoder, which is instrumental in building reusable sequence embeddings for large-scale NLP tasks such as question answering.
Contrastive Distillation on Intermediate Representations for Language Model Compression
Existing language model compression methods mostly use a simple L2 loss to distill knowledge in the intermediate representations of a large BERT model to a smaller one.
Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
Transformer has become ubiquitous in the deep learning field.
Ranked #1 on
Open-Domain Question Answering
on SearchQA
FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding
During inference, the model makes predictions based on the text input in the target language and its translation in the source language.
Ranked #18 on
Zero-Shot Cross-Lingual Transfer
on XTREME
Accelerating Real-Time Question Answering via Question Generation
Although deep neural networks have achieved tremendous success for question answering (QA), they are still suffering from heavy computational and energy cost for real product deployment.
Unsupervised and Supervised Structure Learning for Protein Contact Prediction
Protein contacts provide key information for the understanding of protein structure and function, and therefore contact prediction from sequences is an important problem.
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
We present a large, tunable neural conversational response generation model, DIALOGPT (dialogue generative pre-trained transformer).
Hierarchical Graph Network for Multi-hop Question Answering
In this paper, we present Hierarchical Graph Network (HGN) for multi-hop question answering.
Ranked #32 on
Question Answering
on HotpotQA
DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
We present a large, tunable neural conversational response generation model, DialoGPT (dialogue generative pre-trained transformer).
FreeLB: Enhanced Adversarial Training for Natural Language Understanding
Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models.
Patient Knowledge Distillation for BERT Model Compression
Pre-trained language models such as BERT have proven to be highly effective for natural language processing (NLP) tasks.
Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems
This proposal introduces a Dialogue Challenge for building end-to-end task-completion dialogue systems, with the goal of encouraging the dialogue research community to collaborate and benchmark on standard datasets and unified experimental environment.
Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
Using our predicted contacts as restraints, we can (ab initio) fold 208 of the 398 membrane proteins with TMscore>0. 5.
Graphical Model Sketch
Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables.
Learning structured densities via infinite dimensional exponential families
Learning the structure of a probabilistic graphical models is a well studied problem in the machine learning community due to its importance in many applications.
AUC-maximized Deep Convolutional Neural Fields for Sequence Labeling
Our experimental results confirm that maximum-AUC greatly outperforms the other two training methods on 8-state secondary structure prediction and disorder prediction since their label distributions are highly imbalanced and also have similar performance as the other two training methods on the solvent accessibility prediction problem which has three equally-distributed labels.
Learning Nonparametric Forest Graphical Models with Prior Information
We present a framework for incorporating prior information into nonparametric estimation of graphical models.
Learning Scale-Free Networks by Dynamic Node-Specific Degree Prior
Learning the network structure underlying data is an important problem in machine learning.
