Search Results for author: Zuchao Li

Found 73 papers, 32 papers with code

Restricted or Not: A General Training Framework for Neural Machine Translation

no code implementations ACL 2022 Zuchao Li, Masao Utiyama, Eiichiro Sumita, Hai Zhao

Although this can satisfy the requirements overall, it usually requires a larger beam size and far longer decoding time than unrestricted translation, which limits the concurrent processing ability of the translation model in deployment, and thus its practicality.

Machine Translation Translation

Nested Named Entity Recognition as Corpus Aware Holistic Structure Parsing

no code implementations COLING 2022 Yifei Yang, Zuchao Li, Hai Zhao

Thus in order to address this mismatch, this work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all.

Domain Adaptation named-entity-recognition +4

BiBL: AMR Parsing and Generation with Bidirectional Bayesian Learning

1 code implementation COLING 2022 Ziming Cheng, Zuchao Li, Hai Zhao

Abstract Meaning Representation (AMR) offers a unified semantic representation for natural language sentences.

 Ranked #1 on AMR-to-Text Generation on LDC2017T10 (using extra training data)

AMR Parsing AMR-to-Text Generation +1

What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation

no code implementations Findings (ACL) 2022 Zuchao Li, Yiran Wang, Masao Utiyama, Eiichiro Sumita, Hai Zhao, Taro Watanabe

Inspired by this discovery, we then propose approaches to improving it, with respect to model structure and model training, to make the deep decoder practical in NMT.

Language Modelling Machine Translation +2

Multi-modal Auto-regressive Modeling via Visual Words

1 code implementation12 Mar 2024 Tianshuo Peng, Zuchao Li, Lefei Zhang, Hai Zhao, Ping Wang, Bo Du

Large Language Models (LLMs), benefiting from the auto-regressive modelling approach performed on massive unannotated texts corpora, demonstrates powerful perceptual and reasoning capabilities.

Visual Question Answering

Sparse is Enough in Fine-tuning Pre-trained Large Language Model

1 code implementation19 Dec 2023 Weixi Song, Zuchao Li, Lefei Zhang, Hai Zhao, Bo Du

With the prevalence of pre-training-fine-tuning paradigm, how to efficiently adapt the pre-trained model to the downstream tasks has been an intriguing issue.

Language Modelling Large Language Model

Multi-modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models

no code implementations14 Dec 2023 Liqi He, Zuchao Li, Xiantao Cai, Ping Wang

Overall, our approach offers a more robust and effective solution for multi-modal reasoning in language models, enhancing their ability to tackle complex real-world problems.

Machine Translation

N-Gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding

2 code implementations13 Dec 2023 Jinhao Tian, Zuchao Li, Jiajia Li, Ping Wang

Experiment on various datasets demonstrate the effectiveness of our method and achieved state-of-the-art performance on a series of music understanding downstream tasks.

A Novel Energy based Model Mechanism for Multi-modal Aspect-Based Sentiment Analysis

1 code implementation13 Dec 2023 Tianshuo Peng, Zuchao Li, Ping Wang, Lefei Zhang, Hai Zhao

However, previous methods still have certain limitations: (i) They ignore the difference in the focus of visual information between different analysis targets (aspect or sentiment).

Aspect-Based Sentiment Analysis Sentiment Analysis

Bootstrapping Interactive Image-Text Alignment for Remote Sensing Image Captioning

1 code implementation2 Dec 2023 Cong Yang, Zuchao Li, Lefei Zhang

To efficiently align the image-text, we propose a novel two-stage vision-language pre-training-based approach to bootstrap interactive image-text alignment for remote sensing image captioning, called BITA, which relies on the design of a lightweight interactive Fourier Transformer to better align remote sensing image-text features.

Causal Language Modeling Contrastive Learning +4

ArcMMLU: A Library and Information Science Benchmark for Large Language Models

1 code implementation30 Nov 2023 Shitou Zhang, Zuchao Li, Xingshen Liu, Liming Yang, Ping Wang

In response to this need, this paper introduces ArcMMLU, a specialized benchmark tailored for the Library & Information Science (LIS) domain in Chinese.

Enhancing Visually-Rich Document Understanding via Layout Structure Modeling

1 code implementation15 Aug 2023 Qiwei Li, Zuchao Li, Xiantao Cai, Bo Du, Hai Zhao

In this paper, we propose GraphLayoutLM, a novel document understanding model that leverages the modeling of layout structure graph to inject document layout knowledge into the model.

document understanding

ArcGPT: A Large Language Model Tailored for Real-world Archival Applications

1 code implementation27 Jul 2023 Shitou Zhang, Jingrui Hou, Siyuan Peng, Zuchao Li, Qibiao Hu, Ping Wang

Ultimately, ArcGPT aims to better serve the archival community, aiding archivists in their crucial role of preserving and harnessing our collective information and knowledge.

Language Modelling Large Language Model +1

Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers

1 code implementation2 Jul 2023 Yineng Chen, Zuchao Li, Lefei Zhang, Bo Du, Hai Zhao

SGD and Adam are two classical and effective optimizers on which researchers have proposed many variants, such as SGDM and RAdam.

Bidirectional Correlation-Driven Inter-Frame Interaction Transformer for Referring Video Object Segmentation

no code implementations2 Jul 2023 Meng Lan, Fu Rong, Zuchao Li, Wei Yu, Lefei Zhang

Moreover, a bidirectional vision-language interaction module is implemented before the multimodal Transformer to enhance the correlation between the visual and linguistic features, thus facilitating the language queries to decode more precise object information from visual features and ultimately improving the segmentation performance.

Object Referring Video Object Segmentation +4

BatGPT: A Bidirectional Autoregessive Talker from Generative Pre-trained Transformer

1 code implementation1 Jul 2023 Zuchao Li, Shitou Zhang, Hai Zhao, Yifei Yang, Dongjie Yang

BatGPT is a large-scale language model designed and trained jointly by Wuhan University and Shanghai Jiao Tong University.

Language Modelling Question Answering +1

FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction

1 code implementation19 Jun 2023 Tianshuo Peng, Zuchao Li, Lefei Zhang, Bo Du, Hai Zhao

To address these deficiencies, we propose the Fuzzy Span Universal Information Extraction (FSUIE) framework.

UIE

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models

2 code implementations26 May 2023 Yao Yao, Zuchao Li, Hai Zhao

Therefore, we propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph.

GSM8K Multimodal Reasoning +1

Centroid-centered Modeling for Efficient Vision Transformer Pre-training

no code implementations8 Mar 2023 Xin Yan, Zuchao Li, Lefei Zhang, Bo Du, DaCheng Tao

Our proposed approach, \textbf{CCViT}, leverages k-means clustering to obtain centroids for image modeling without supervised training of tokenizer model.

Semantic Segmentation

Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning

no code implementations23 Aug 2022 Letian Peng, Zuchao Li, Hai Zhao

In detail, it works on PLMs according to the Replaced Token Detection (RTD) pre-training objective in ELECTRA, in which the corruption detection objective reflects the confidence on contextual integrity that is more relevant to commonsense reasoning than existing probability.

Language Modelling Question Answering +1

Nested Named Entity Recognition as Holistic Structure Parsing

no code implementations17 Apr 2022 Yifei Yang, Zuchao Li, Hai Zhao

Thus in order to address this mismatch, this work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all.

Domain Adaptation named-entity-recognition +4

Semantics-Preserved Distortion for Personal Privacy Protection in Information Management

no code implementations4 Jan 2022 Jiajia Li, Letian Peng, Ping Wang, Zuchao Li, Xueyi Li, Hai Zhao

As the model training on information from users is likely to invade personal privacy, many methods have been proposed to block the learning and memorizing of the sensitive data in raw texts.

Attribute Constituency Parsing +6

Multilingual Pre-training with Universal Dependency Learning

no code implementations NeurIPS 2021 Kailai Sun, Zuchao Li, Hai Zhao

The pre-trained language model (PrLM) demonstrates domination in downstream natural language processing tasks, in which multilingual PrLM takes advantage of language universality to alleviate the issue of limited resources for low-resource languages.

Dependency Parsing Natural Language Understanding +1

Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model

1 code implementation EMNLP 2021 Hongjiang Jing, Zuchao Li, Hai Zhao, Shu Jiang

Therefore, we propose a joint ABSA model, which not only enjoys the benefits of encoder sharing but also focuses on the difference to improve the effectiveness of the model.

Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA) +1

Contextualized Semantic Distance between Highly Overlapped Texts

1 code implementation4 Oct 2021 Letian Peng, Zuchao Li, Hai Zhao

Overlapping frequently occurs in paired texts in natural language processing tasks like text editing and semantic similarity evaluation.

Domain Adaptation Language Modelling +8

Sparse Fuzzy Attention for Structured Sentiment Analysis

no code implementations14 Sep 2021 Letian Peng, Zuchao Li, Hai Zhao

Attention scorers have achieved success in parsing tasks like semantic and syntactic dependency parsing.

Dependency Parsing Sentiment Analysis

Cross-lingual Transferring of Pre-trained Contextualized Language Models

no code implementations27 Jul 2021 Zuchao Li, Kevin Parnow, Hai Zhao, Zhuosheng Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita

Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.

Language Modelling Machine Translation +1

Grammatical Error Correction as GAN-like Sequence Labeling

no code implementations Findings (ACL) 2021 Kevin Parnow, Zuchao Li, Hai Zhao

In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is an iterative process, as sentences are passed to the model for multiple rounds of correction, which exposes the model to sentences with progressively fewer errors at each round.

Grammatical Error Correction

Head-driven Phrase Structure Parsing in O($n^3$) Time Complexity

no code implementations20 May 2021 Zuchao Li, Junru Zhou, Hai Zhao, Kevin Parnow

Constituent and dependency parsing, the two classic forms of syntactic parsing, have been found to benefit from joint training and decoding under a uniform formalism, Head-driven Phrase Structure Grammar (HPSG).

Dependency Parsing

Neural Unsupervised Semantic Role Labeling

no code implementations19 Apr 2021 Kashif Munir, Hai Zhao, Zuchao Li

To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules.

Clustering Semantic Role Labeling +1

Text Compression-aided Transformer Encoding

no code implementations11 Feb 2021 Zuchao Li, Zhuosheng Zhang, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita

In this paper, we propose explicit and implicit text compression approaches to enhance the Transformer encoding and evaluate models using this approach on several typical downstream tasks that rely on the encoding heavily.

Text Compression

Switching-Aligned-Words Data Augmentation for Neural Machine Translation

no code implementations1 Jan 2021 Fengshun Xiao, Zuchao Li, Hai Zhao

In neural machine translation (NMT), data augmentation methods such as back-translation make it possible to use extra monolingual data to help improve translation performance, while it needs extra training data and the in-domain monolingual data is not always available.

Data Augmentation Machine Translation +3

Cross-lingual Transfer Learning for Pre-trained Contextualized Language Models

no code implementations1 Jan 2021 Zuchao Li, Kevin Barry Parnow, Hai Zhao, Zhuosheng Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita

Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.

Cross-Lingual Transfer Language Modelling +3

Adaptive Convolution for Semantic Role Labeling

no code implementations27 Dec 2020 Kashif Munir, Hai Zhao, Zuchao Li

Semantic role labeling (SRL) aims at elaborating the meaning of a sentence by forming a predicate-argument structure.

Semantic Role Labeling Sentence

Cross-lingual Universal Dependency Parsing Only from One Monolingual Treebank

no code implementations24 Dec 2020 Kailai Sun, Zuchao Li, Hai Zhao

As it is unlikely to obtain a treebank for every human language, in this work, we propose an effective cross-lingual UD parsing framework for transferring parser from only one source monolingual treebank to any other target languages without treebank available.

Cross-Lingual Transfer Dependency Parsing +3

Document-level Neural Machine Translation with Document Embeddings

no code implementations16 Sep 2020 Shu Jiang, Hai Zhao, Zuchao Li, Bao-liang Lu

Standard neural machine translation (NMT) is on the assumption of document-level context independent.

Machine Translation NMT +1

Syntax Role for Neural Semantic Role Labeling

no code implementations CL (ACL) 2021 Zuchao Li, Hai Zhao, Shexia He, Jiaxun Cai

Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence.

Semantic Role Labeling Sentence

Data-dependent Gaussian Prior Objective for Language Generation

no code implementations ICLR 2020 Zuchao Li, Rui Wang, Kehai Chen, Masso Utiyama, Eiichiro Sumita, Zhuosheng Zhang, Hai Zhao

However, MLE focuses on once-to-all matching between the predicted sequence and gold-standard, consequently treating all incorrect predictions as being equally incorrect.

Image Captioning L2 Regularization +4

Neural Machine Translation with Universal Visual Representation

1 code implementation ICLR 2020 Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao

Though visual information has been introduced for enhancing neural machine translation (NMT), its effectiveness strongly relies on the availability of large amounts of bilingual parallel sentence pairs with manual image annotations.

Machine Translation NMT +2

Reference Language based Unsupervised Neural Machine Translation

1 code implementation Findings of the Association for Computational Linguistics 2020 Zuchao Li, Hai Zhao, Rui Wang, Masao Utiyama, Eiichiro Sumita

Further enriching the idea of pivot translation by extending the use of parallel corpora beyond the source-target paradigm, we propose a new reference language-based framework for UNMT, RUNMT, in which the reference language only shares a parallel corpus with the source, but this corpus still indicates a signal clear enough to help the reconstruction training of UNMT through a proposed reference agreement mechanism.

Machine Translation Translation

Explicit Sentence Compression for Neural Machine Translation

1 code implementation27 Dec 2019 Zuchao Li, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Zhuosheng Zhang, Hai Zhao

In this paper, we propose an explicit sentence compression method to enhance the source sentence representation for NMT.

Machine Translation NMT +3

Global Greedy Dependency Parsing

1 code implementation20 Nov 2019 Zuchao Li, Hai Zhao, Kevin Parnow

Most syntactic dependency parsing models may fall into one of two categories: transition- and graph-based models.

Dependency Parsing Re-Ranking +1

Dependency and Span, Cross-Style Semantic Role Labeling on PropBank and NomBank

no code implementations7 Nov 2019 Zuchao Li, Hai Zhao, Junru Zhou, Kevin Parnow, Shexia He

In this paper, we define a new cross-style semantic role label convention and propose a new cross-style joint optimization model designed around the most basic linguistic meaning of a semantic role, providing a solution to make the results of the two styles more comparable and allowing both formalisms of SRL to benefit from their natural connections in both linguistics and computation.

Semantic Role Labeling

SJTU-NICT at MRP 2019: Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing

no code implementations CONLL 2019 Zuchao Li, Hai Zhao, Zhuosheng Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita

This paper describes our SJTU-NICT{'}s system for participating in the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL).

Multi-Task Learning

Document-level Neural Machine Translation with Associated Memory Network

no code implementations31 Oct 2019 Shu Jiang, Rui Wang, Zuchao Li, Masao Utiyama, Kehai Chen, Eiichiro Sumita, Hai Zhao, Bao-liang Lu

Most existing document-level NMT approaches are satisfied with a smattering sense of global document-level information, while this work focuses on exploiting detailed document-level context in terms of a memory network.

Machine Translation NMT +2

Subword ELMo

no code implementations18 Sep 2019 Jiangtong Li, Hai Zhao, Zuchao Li, Wei Bi, Xiaojiang Liu

Embedding from Language Models (ELMo) has shown to be effective for improving many natural language processing (NLP) tasks, and ELMo takes character information to compose word representation to train language models. However, the character is an insufficient and unnatural linguistic unit for word representation. Thus we introduce Embedding from Subword-aware Language Models (ESuLMo) which learns word representation from subwords using unsupervised segmentation over words. We show that ESuLMo can enhance four benchmark NLP tasks more effectively than ELMo, including syntactic dependency parsing, semantic role labeling, implicit discourse relation recognition and textual entailment, which brings a meaningful improvement over ELMo.

Dependency Parsing Natural Language Inference +1

Semantics-aware BERT for Language Understanding

1 code implementation5 Sep 2019 Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, Xiang Zhou

The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks.

Language Modelling Machine Reading Comprehension +5

Modeling Named Entity Embedding Distribution into Hypersphere

no code implementations3 Sep 2019 Zhuosheng Zhang, Bingjie Tang, Zuchao Li, Hai Zhao

This work models named entity distribution from a way of visualizing topological structure of embedding space, so that we make an assumption that most, if not all, named entities (NEs) for a language tend to aggregate together to be accommodated by a specific hypersphere in embedding space.

named-entity-recognition Named Entity Recognition +1

Syntax-aware Multilingual Semantic Role Labeling

1 code implementation IJCNLP 2019 Shexia He, Zuchao Li, Hai Zhao

Recently, semantic role labeling (SRL) has earned a series of success with even higher performance improvements, which can be mainly attributed to syntactic integration and enhanced word representation.

Semantic Role Labeling

Parsing All: Syntax and Semantics, Dependencies and Spans

1 code implementation Findings of the Association for Computational Linguistics 2020 Junru Zhou, Zuchao Li, Hai Zhao

Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former.

Semantic Parsing

Controllable Dual Skew Divergence Loss for Neural Machine Translation

no code implementations22 Aug 2019 Zuchao Li, Hai Zhao, Yingting Wu, Fengshun Xiao, Shu Jiang

Our experiments indicate that switching to the DSD loss after the convergence of ML training helps models escape local optima and stimulates stable performance improvements.

Machine Translation NMT +1

Joint Learning of POS and Dependencies for Multilingual Universal Dependency Parsing

1 code implementation CONLL 2018 Zuchao Li, Shexia He, Zhuosheng Zhang, Hai Zhao

This paper describes the system of team LeisureX in the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.

Lemmatization Part-Of-Speech Tagging +3

Explicit Contextual Semantics for Text Comprehension

no code implementations8 Sep 2018 Zhuosheng Zhang, Yuwei Wu, Zuchao Li, Hai Zhao

Who did what to whom is a major focus in natural language understanding, which is right the aim of semantic role labeling (SRL) task.

Machine Reading Comprehension Natural Language Understanding +1

Attentive Semantic Role Labeling with Boundary Indicator

no code implementations8 Sep 2018 Zhuosheng Zhang, Shexia He, Zuchao Li, Hai Zhao

The goal of semantic role labeling (SRL) is to discover the predicate-argument structure of a sentence, which plays a critical role in deep processing of natural language.

Semantic Role Labeling Sentence

A Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?

1 code implementation11 Aug 2018 Jiaxun Cai, Shexia He, Zuchao Li, Hai Zhao

Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling.

Semantic Role Labeling Sentence

A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware?

no code implementations COLING 2018 Jiaxun Cai, Shexia He, Zuchao Li, Hai Zhao

Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling.

Machine Translation Question Answering +3

Seq2seq Dependency Parsing

3 code implementations COLING 2018 Zuchao Li, Jiaxun Cai, Shexia He, Hai Zhao

This paper presents a sequence to sequence (seq2seq) dependency parser by directly predicting the relative position of head for each given word, which therefore results in a truly end-to-end seq2seq dependency parser for the first time.

Dependency Parsing Feature Engineering +1

Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association

no code implementations ACL 2018 Yafang Huang, Zuchao Li, Zhuosheng Zhang, Hai Zhao

Chinese pinyin input method engine (IME) lets user conveniently input Chinese into a computer by typing pinyin through the common keyboard.

Information Retrieval Machine Translation +3

Cannot find the paper you are looking for? You can Submit a new open access paper.