Search Results for author: Weiguo Gao

Found 10 papers, 3 papers with code

MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding Length Extrapolation

no code implementations26 Mar 2024 Weiguo Gao

Drawing on the ALiBi approach, this study proposes a novel relative positional encoding method, called MEP, which employs a weighted average to combine distinct kernel functions(such as the exponential kernel and the Gaussian kernel) to generate a bias that is applied to post-softmax attention scores.

PAI at SemEval-2023 Task 2: A Universal System for Named Entity Recognition with External Entity Information

1 code implementation10 May 2023 Long Ma, Kai Lu, Tianbo Che, Hailong Huang, Weiguo Gao, Xuan Li

The MultiCoNER II task aims to detect complex, ambiguous, and fine-grained named entities in low-context situations and noisy scenarios like the presence of spelling mistakes and typos for multiple languages.

named-entity-recognition Named Entity Recognition +3

A More Stable Accelerated Gradient Method Inspired by Continuous-Time Perspective

no code implementations9 Dec 2021 Yasong Feng, Weiguo Gao

Experiments of matrix completion and handwriting digit recognition demonstrate that the stability of our new method is better.

Matrix Completion

SOFT: Softmax-free Transformer with Linear Complexity

2 code implementations NeurIPS 2021 Jiachen Lu, Jinghan Yao, Junge Zhang, Xiatian Zhu, Hang Xu, Weiguo Gao, Chunjing Xu, Tao Xiang, Li Zhang

Crucially, with a linear complexity, much longer token sequences are permitted in SOFT, resulting in superior trade-off between accuracy and complexity.

Computational Efficiency

A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification

no code implementations24 Sep 2021 Guang Liu, Hailong Huang, Yuzhao Mao, Weiguo Gao, Xuan Li, Jianping Shen

Previous studies mostly use a fine-tuned Language Model (LM) to strengthen the constraints but ignore the fact that the potential of diversity could improve the effectiveness of generated data.

Data Augmentation Language Modelling

Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup

1 code implementation EMNLP 2021 Guang Liu, Yuzhao Mao, Hailong Huang, Weiguo Gao, Xuan Li

To address these issues, we propose the Adversarial Mixing Policy (AMP), organized in a min-max-rand formulation, to relax the Locally Linear Constraints in Mixup.

text-classification Text Classification

A new accelerated gradient method inspired by continuous-time perspective

no code implementations1 Jan 2021 Yasong Feng, Weiguo Gao

To give more insight about the acceleration phenomenon, an ordinary differential equation was obtained from Nesterov's accelerated method by taking step sizes approaching zero, and the relationship between Nesterov's method and the differential equation is still of research interest.

Matrix Completion

Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion

no code implementations3 Mar 2020 Jingyuan Yang, Guang Liu, Yuzhao Mao, Zhiwei Zhao, Weiguo Gao, Xuan Li, Haiqin Yang, Jianping Shen

Task 1 of the DSTC8-track1 challenge aims to develop an end-to-end multi-domain dialogue system to accomplish complex users' goals under tourist information desk settings.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.