Search Results for author: Bing Xiang

Found 85 papers, 32 papers with code

H2KGAT: Hierarchical Hyperbolic Knowledge Graph Attention Network

no code implementations EMNLP 2020 Shen Wang, Xiaokai Wei, Cicero Nogueira dos santos, Zhiguo Wang, Ramesh Nallapati, Andrew Arnold, Bing Xiang, Philip S. Yu

Existing knowledge graph embedding approaches concentrate on modeling symmetry/asymmetry, inversion, and composition typed relations but overlook the hierarchical nature of relations.

Graph Attention Knowledge Graph Embedding +2

Code Representation Learning At Scale

no code implementations2 Feb 2024 Dejiao Zhang, Wasi Ahmad, Ming Tan, Hantian Ding, Ramesh Nallapati, Dan Roth, Xiaofei Ma, Bing Xiang

Recent studies have shown that code language models at scale demonstrate significant performance gains on downstream tasks, i. e., code generation.

Code Generation Contrastive Learning +3

Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

no code implementations10 Aug 2023 Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang

We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data.

Data-to-Text Generation

Lightweight reranking for language model generations

no code implementations11 Jul 2023 Siddhartha Jain, Xiaofei Ma, Anoop Deoras, Bing Xiang

We show strong improvements for selecting the best k generations for code generation tasks as well as robust improvements for the best generation for the tasks of autoformalization, summarization, and translation.

Code Generation Language Modelling

UNITE: A Unified Benchmark for Text-to-SQL Evaluation

1 code implementation25 May 2023 Wuwei Lan, Zhiguo Wang, Anuj Chauhan, Henghui Zhu, Alexander Li, Jiang Guo, Sheng Zhang, Chung-Wei Hang, Joseph Lilien, Yiqun Hu, Lin Pan, Mingwen Dong, Jun Wang, Jiarong Jiang, Stephen Ash, Vittorio Castelli, Patrick Ng, Bing Xiang

A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures.

Text-To-SQL

CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

no code implementations20 Dec 2022 Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, Bing Xiang

While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.

Code Completion

ReCode: Robustness Evaluation of Code Generation Models

2 code implementations20 Dec 2022 Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang

Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.

Code Generation

Importance of Synthesizing High-quality Data for Text-to-SQL Parsing

no code implementations17 Dec 2022 Yiyun Zhao, Jiarong Jiang, Yiqun Hu, Wuwei Lan, Henry Zhu, Anuj Chauhan, Alexander Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Marvin Dong, Joe Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, Bing Xiang

In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data.

SQL Parsing SQL-to-Text +2

Multi-lingual Evaluation of Code Generation Models

2 code implementations26 Oct 2022 Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.

Code Completion Code Translation +1

DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases

1 code implementation30 Sep 2022 Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, William Wang, Zhiguo Wang, Bing Xiang

Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs.

Entity Linking Question Answering +2

Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding

no code implementations28 Sep 2022 Jun Wang, Patrick Ng, Alexander Hanbo Li, Jiarong Jiang, Zhiguo Wang, Ramesh Nallapati, Bing Xiang, Sudipta Sengupta

When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.

NER Semantic Parsing +1

REKnow: Enhanced Knowledge for Joint Entity and Relation Extraction

no code implementations10 Jun 2022 Sheng Zhang, Patrick Ng, Zhiguo Wang, Bing Xiang

Our generative model is a unified framework to sequentially generate relational triplets under various relation extraction settings and explicitly utilizes relevant knowledge from Knowledge Graph (KG) to resolve ambiguities.

Joint Entity and Relation Extraction Relation

Learning Dialogue Representations from Consecutive Utterances

1 code implementation NAACL 2022 Zhihan Zhou, Dejiao Zhang, Wei Xiao, Nicholas Dingwall, Xiaofei Ma, Andrew O. Arnold, Bing Xiang

In this paper, we introduce Dialogue Sentence Embedding (DSE), a self-supervised contrastive learning method that learns effective dialogue representations suitable for a wide range of dialogue tasks.

Contrastive Learning Conversational Question Answering +14

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization

2 code implementations ACL 2022 Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth

Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.

Knowledge Distillation Model Compression +2

Attention-guided Generative Models for Extractive Question Answering

no code implementations12 Oct 2021 Peng Xu, Davis Liang, Zhiheng Huang, Bing Xiang

We propose a simple strategy to obtain an extractive answer span from the generative model by leveraging the decoder cross-attention patterns.

Extractive Question-Answering Hallucination +2

Multiplicative Position-aware Transformer Models for Language Understanding

no code implementations27 Sep 2021 Zhiheng Huang, Davis Liang, Peng Xu, Bing Xiang

Transformer models, which leverage architectural improvements like self-attention, perform remarkably well on Natural Language Processing (NLP) tasks.

Position

Pairwise Supervised Contrastive Learning of Sentence Representations

1 code implementation EMNLP 2021 Dejiao Zhang, Shang-Wen Li, Wei Xiao, Henghui Zhu, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang

Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese loss.

Contrastive Learning Natural Language Inference +4

Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering

1 code implementation ACL 2021 Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, Bing Xiang

However, a large amount of world's knowledge is stored in structured databases, and need to be accessed using query languages such as SQL.

Open-Domain Question Answering

Joint Text and Label Generation for Spoken Language Understanding

no code implementations11 May 2021 Yang Li, Ben Athiwaratkun, Cicero Nogueira dos santos, Bing Xiang

In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data.

intent-classification Intent Classification +2

Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering

no code implementations EACL 2021 Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

Experiments show that: (1) Our IR-based retrieval method is able to collect high-quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi-task learning, the unified model obtains further improvements with only 1/3 of the original parameters.

Entity Linking Information Retrieval +4

Structured Prediction as Translation between Augmented Natural Languages

1 code implementation ICLR 2021 Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cicero Nogueira dos santos, Bing Xiang, Stefano Soatto

We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extraction, nested named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, and dialogue state tracking.

coreference-resolution Dialogue State Tracking +11

Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training

3 code implementations18 Dec 2020 Peng Shi, Patrick Ng, Zhiguo Wang, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Cicero Nogueira dos santos, Bing Xiang

Most recently, there has been significant interest in learning contextual representations for various NLP tasks, by leveraging large scale text corpora to train large neural language models with self-supervised learning objectives, such as Masked Language Model (MLM).

Ranked #6 on Text-To-SQL on spider (Exact Match Accuracy (Dev) metric)

Language Modelling Self-Supervised Learning +2

Beyond [CLS] through Ranking by Generation

no code implementations EMNLP 2020 Cicero Nogueira dos santos, Xiaofei Ma, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

Generative models for Information Retrieval, where ranking of documents is viewed as the task of generating a query from a document's language model, were very successful in various IR tasks in the past.

Answer Selection Information Retrieval +4

Embedding-based Zero-shot Retrieval through Query Generation

1 code implementation22 Sep 2020 Davis Liang, Peng Xu, Siamak Shakeri, Cicero Nogueira dos Santos, Ramesh Nallapati, Zhiheng Huang, Bing Xiang

In some cases, our model trained on synthetic data can even outperform the same model trained on real data

Passage Retrieval Retrieval

Augmented Natural Language for Generative Sequence Labeling

no code implementations EMNLP 2020 Ben Athiwaratkun, Cicero Nogueira dos santos, Jason Krone, Bing Xiang

We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot ($75. 0\% \rightarrow 90. 9\%$) and 1-shot ($70. 4\% \rightarrow 81. 0\%$) state-of-the-art results.

intent-classification Intent Classification +4

Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering

1 code implementation ACL 2020 Alexander R. Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.

Language Modelling Question Answering +3

TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding

no code implementations16 Mar 2020 Zhiheng Huang, Peng Xu, Davis Liang, Ajay Mishra, Bing Xiang

Prior to the transformer era, bidirectional Long Short-Term Memory (BLSTM) has been the dominant modeling architecture for neural machine translation and question answering.

Machine Translation Natural Language Inference +5

Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer

1 code implementation25 Nov 2019 Henghui Zhu, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to.

Domain Adaptation with BERT-based Domain Classification and Data Selection

no code implementations WS 2019 Xiaofei Ma, Peng Xu, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

The performance of deep neural models can deteriorate substantially when there is a domain shift between training and test data.

Classification Domain Adaptation +2

Universal Text Representation from BERT: An Empirical Study

no code implementations17 Oct 2019 Xiaofei Ma, Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks.

Learning-To-Rank Natural Language Inference +4

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

no code implementations IJCNLP 2019 Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages.

Open-Domain Question Answering

Passage Ranking with Weak Supervision

no code implementations ICLR Workshop LLD 2019 Peng Xu, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

In this paper, we propose a \textit{weak supervision} framework for neural ranking tasks based on the data programming paradigm \citep{Ratner2016}, which enables us to leverage multiple weak supervision signals from different sources.

Passage Ranking

WeNet: Weighted Networks for Recurrent Network Architecture Search

no code implementations8 Apr 2019 Zhiheng Huang, Bing Xiang

In this paper, we propose a novel way of architecture search by means of weighted networks (WeNet), which consist of a number of networks, with each assigned a weight.

General Classification Image Classification +2

WEAKLY SEMI-SUPERVISED NEURAL TOPIC MODELS

no code implementations ICLR Workshop LLD 2019 Ian Gemp, Ramesh Nallapati, Ran Ding, Feng Nan, Bing Xiang

We extend NTMs to the weakly semi-supervised setting by using informative priors in the training objective.

Topic Models

Coherence-Aware Neural Topic Modeling

2 code implementations EMNLP 2018 Ran Ding, Ramesh Nallapati, Bing Xiang

Topic models are evaluated based on their ability to describe documents well (i. e. low perplexity) and to produce topics that carry coherent semantic meaning.

Topic Models Variational Inference

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks

no code implementations28 Sep 2017 Mingbo Ma, Kai Zhao, Liang Huang, Bing Xiang, Bo-Wen Zhou

In order to utilize the potential benefits from their correlations, we propose a jointly trained model for learning the two tasks simultaneously via Long Short-Term Memory (LSTM) networks.

Classification General Classification +10

Neural Models for Sequence Chunking

1 code implementation15 Jan 2017 Feifei Zhai, Saloni Potdar, Bing Xiang, Bo-Wen Zhou

Many natural language understanding (NLU) tasks, such as shallow parsing (i. e., text chunking) and semantic slot filling, require the assignment of representative labels to the meaningful chunks in a sentence.

Chunking Natural Language Understanding +3

GaDei: On Scale-up Training As A Service For Deep Learning

no code implementations18 Nov 2016 Wei Zhang, Minwei Feng, Yunhui Zheng, Yufei Ren, Yandong Wang, Ji Liu, Peng Liu, Bing Xiang, Li Zhang, Bo-Wen Zhou, Fei Wang

By evaluating the NLC workloads, we show that only the conservative hyper-parameter setup (e. g., small mini-batch size and small learning rate) can guarantee acceptable model accuracy for a wide range of customers.

End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension

no code implementations31 Oct 2016 Yang Yu, Wei zhang, Kazi Hasan, Mo Yu, Bing Xiang, Bo-Wen Zhou

This paper proposes dynamic chunk reader (DCR), an end-to-end neural reading comprehension (RC) model that is able to extract and rank a set of answer candidates from a given document to answer questions.

Question Answering Reading Comprehension

Simple Question Answering by Attentive Convolutional Neural Network

no code implementations COLING 2016 Wenpeng Yin, Mo Yu, Bing Xiang, Bo-Wen Zhou, Hinrich Schütze

In fact selection, we match the subject entity in a fact candidate with the entity mention in the question by a character-level convolutional neural network (char-CNN), and match the predicate in that fact with the question by a word-level CNN (word-CNN).

Entity Linking Fact Selection +1

Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond

4 code implementations CONLL 2016 Ramesh Nallapati, Bo-Wen Zhou, Cicero Nogueira dos santos, Caglar Gulcehre, Bing Xiang

In this work, we model abstractive text summarization using Attentional Encoder-Decoder Recurrent Neural Networks, and show that they achieve state-of-the-art performance on two different corpora.

Abstractive Text Summarization Sentence +2

Attentive Pooling Networks

3 code implementations11 Feb 2016 Cicero dos Santos, Ming Tan, Bing Xiang, Bo-Wen Zhou

In this work, we propose Attentive Pooling (AP), a two-way attention mechanism for discriminative model training.

Answer Selection Representation Learning

Leveraging Sentence-level Information with Encoder LSTM for Semantic Slot Filling

no code implementations EMNLP 2016 Gakuto Kurata, Bing Xiang, Bo-Wen Zhou, Mo Yu

Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling.

Natural Language Understanding Sentence +2

ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

8 code implementations TACL 2016 Wenpeng Yin, Hinrich Schütze, Bing Xiang, Bo-Wen Zhou

(ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart.

Answer Selection Natural Language Inference +2

Good, Better, Best: Choosing Word Embedding Context

no code implementations19 Nov 2015 James Cross, Bing Xiang, Bo-Wen Zhou

We propose two methods of learning vector representations of words and phrases that each combine sentence context with structural features extracted from dependency trees.

Sentence

LSTM-based Deep Learning Models for Non-factoid Answer Selection

2 code implementations12 Nov 2015 Ming Tan, Cicero dos Santos, Bing Xiang, Bo-Wen Zhou

One direction is to define a more composite representation for questions and answers by combining convolutional neural network with the basic framework.

Answer Selection

Distributed Deep Learning for Question Answering

no code implementations3 Nov 2015 Minwei Feng, Bing Xiang, Bo-Wen Zhou

This paper is an empirical study of the distributed deep learning for question answering subtasks: answer selection and question classification.

Answer Selection General Classification

Empirical Study on Deep Learning Models for Question Answering

no code implementations26 Oct 2015 Yang Yu, Wei zhang, Chung-Wei Hang, Bing Xiang, Bo-Wen Zhou

In this paper we explore deep learning models with memory component or attention mechanism for question answering task.

Machine Translation Question Answering +1

Applying Deep Learning to Answer Selection: A Study and An Open Task

2 code implementations7 Aug 2015 Minwei Feng, Bing Xiang, Michael R. Glass, Lidan Wang, Bo-Wen Zhou

We apply a general deep learning framework to address the non-factoid question answering task.

Answer Selection

Dependency-based Convolutional Neural Networks for Sentence Embedding

1 code implementation IJCNLP 2015 Mingbo Ma, Liang Huang, Bing Xiang, Bo-Wen Zhou

In sentence modeling and classification, convolutional neural network approaches have recently achieved state-of-the-art results, but all such efforts process word vectors sequentially and neglect long-distance dependencies.

Classification General Classification +3

Classifying Relations by Ranking with Convolutional Neural Networks

2 code implementations IJCNLP 2015 Cicero Nogueira dos Santos, Bing Xiang, Bo-Wen Zhou

Relation classification is an important semantic processing task for which state-ofthe-art systems still rely on costly handcrafted features.

Classification General Classification +4

Cannot find the paper you are looking for? You can Submit a new open access paper.