Search Results for author: Guotong Xie

Found 38 papers, 9 papers with code

GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning

no code implementations EMNLP 2021 Wei Zhu, Xiaoling Wang, Yuan Ni, Guotong Xie

From this observation, we use mutual learning to improve BERT’s early exiting performances, that is, we ask each exit of a multi-exit BERT to distill knowledge from each other.

Knowledge Distillation

Pre-training Entity Relation Encoder with Intra-span and Inter-span Information

no code implementations EMNLP 2020 Yijun Wang, Changzhi Sun, Yuanbin Wu, Junchi Yan, Peng Gao, Guotong Xie

In particular, a span encoder is trained to recover a random shuffling of tokens in a span, and a span pair encoder is trained to predict positive pairs that are from the same sentences and negative pairs that are from different sentences using contrastive loss.

Relation Relation Extraction +1

Text2MDT: Extracting Medical Decision Trees from Medical Texts

1 code implementation4 Jan 2024 Wei Zhu, Wenfeng Li, Xing Tian, Pengfei Wang, Xiaoling Wang, Jin Chen, Yuanbin Wu, Yuan Ni, Guotong Xie

In this work, we propose a novel task, Text2MDT, to explore the automatic extraction of MDTs from medical texts such as medical guidelines and textbooks.

UltraFeedback: Boosting Language Models with High-quality Feedback

1 code implementation2 Oct 2023 Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, Maosong Sun

However, the scarcity of diverse, naturalistic datasets of human preferences on LLM outputs at scale poses a great challenge to RLHF as well as feedback learning research within the open-source community.

Language Modelling

Interactive Molecular Discovery with Natural Language

1 code implementation21 Jun 2023 Zheni Zeng, Bangchen Yin, Shipeng Wang, Jiarui Liu, Cheng Yang, Haishen Yao, Xingzhi Sun, Maosong Sun, Guotong Xie, Zhiyuan Liu

Natural language is expected to be a key medium for various human-machine interactions in the era of large language models.

Property Prediction

Exploring the Impact of Model Scaling on Parameter-Efficient Tuning

1 code implementation4 Jun 2023 Yusheng Su, Chi-Min Chan, Jiali Cheng, Yujia Qin, Yankai Lin, Shengding Hu, Zonghan Yang, Ning Ding, Xingzhi Sun, Guotong Xie, Zhiyuan Liu, Maosong Sun

Our investigations reveal that model scaling (1) mitigates the effects of the positions of tunable parameters on performance, and (2) enables tuning methods to achieve performance comparable to full-parameter fine-tuning by optimizing fewer tunable parameters.

Unified Demonstration Retriever for In-Context Learning

1 code implementation7 May 2023 Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, Xipeng Qiu

To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.

In-Context Learning Language Modelling +1

Filter Pruning via Filters Similarity in Consecutive Layers

no code implementations26 Apr 2023 Xiaorui Wang, Jun Wang, Xin Tang, Peng Gao, Rui Fang, Guotong Xie

Filter pruning is widely adopted to compress and accelerate the Convolutional Neural Networks (CNNs), but most previous works ignore the relationship between filters and channels in different layers.

Automatic Fine-grained Glomerular Lesion Recognition in Kidney Pathology

no code implementations11 Mar 2022 Yang Nan, Fengyi Li, Peng Tang, Guyue Zhang, Caihong Zeng, Guotong Xie, Zhihong Liu, Guang Yang

Recognition of glomeruli lesions is the key for diagnosis and treatment planning in kidney pathology; however, the coexisting glomerular structures such as mesangial regions exacerbate the difficulties of this task.

Fine-Grained Image Classification whole slide images

A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation

1 code implementation Findings (ACL) 2022 Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, Xipeng Qiu

Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.

CandidateDrug4Cancer: An Open Molecular Graph Learning Benchmark on Drug Discovery for Cancer

no code implementations2 Mar 2022 Xianbin Ye, Ziliang Li, Fei Ma, Zongbi Yi, Pengyong Li, Jun Wang, Peng Gao, Yixuan Qiao, Guotong Xie

Anti-cancer drug discoveries have been serendipitous, we sought to present the Open Molecular Graph Learning Benchmark, named CandidateDrug4Cancer, a challenging and realistic benchmark dataset to facilitate scalable, robust, and reproducible graph machine learning research for anti-cancer drug discovery.

Drug Discovery Graph Learning

Superpixel-Based Building Damage Detection from Post-earthquake Very High Resolution Imagery Using Deep Neural Networks

no code implementations9 Dec 2021 Jun Wang, Zhoujing Li, Yixuan Qiao, Qiming Qin, Peng Gao, Guotong Xie

This paper presents a novel superpixel based approach combining DNN and a modified segmentation method, to detect damaged buildings from VHR imagery.

Denoising Semantic Similarity +1

Discovering Better Model Architectures for Medical Query Understanding

no code implementations NAACL 2021 Wei Zhu, Yuan Ni, Xiaoling Wang, Guotong Xie

In developing an online question-answering system for the medical domains, natural language inference (NLI) models play a central role in question matching and intention detection.

Natural Language Inference Neural Architecture Search +2

Lesion Segmentation and RECIST Diameter Prediction via Click-driven Attention and Dual-path Connection

no code implementations5 May 2021 YouBao Tang, Ke Yan, Jinzheng Cai, Lingyun Huang, Guotong Xie, Jing Xiao, JingJing Lu, Gigin Lin, Le Lu

PDNet learns comprehensive and representative deep image features for our tasks and produces more accurate results on both lesion segmentation and RECIST diameter prediction.

Lesion Segmentation Segmentation

An effective self-supervised framework for learning expressive molecular global representations to drug discovery

1 code implementation Briefings in Bioinformatics 2021 Pengyong Li, Jun Wang, Yixuan Qiao, Hao Chen, Yihuan Yu, Xiaojun Yao, Peng Gao, Guotong Xie, Sen Song

In MPG, we proposed a powerful GNN for modelling molecular graph named MolGNet, and designed an effective self-supervised strategy for pre-training the model at both the node and graph-level.

Drug Discovery

Weakly-Supervised Universal Lesion Segmentation with Regional Level Set Loss

no code implementations3 May 2021 YouBao Tang, Jinzheng Cai, Ke Yan, Lingyun Huang, Guotong Xie, Jing Xiao, JingJing Lu, Gigin Lin, Le Lu

Accurately segmenting a variety of clinically significant lesions from whole body computed tomography (CT) scans is a critical task on precision oncology imaging, denoted as universal lesion segmentation (ULS).

Computed Tomography (CT) Lesion Segmentation +2

Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph

no code implementations29 Apr 2021 Xiao-Yun Zhou, Bolin Lai, Weijian Li, Yirui Wang, Kang Zheng, Fakai Wang, ChiHung Lin, Le Lu, Lingyun Huang, Mei Han, Guotong Xie, Jing Xiao, Kuo Chang-Fu, Adam Harrison, Shun Miao

It first trains a DAG model on the labeled data and then fine-tunes the pre-trained model on the unlabeled data with a teacher-student SSL mechanism.

Learning from Subjective Ratings Using Auto-Decoded Deep Latent Embeddings

no code implementations12 Apr 2021 Bowen Li, Xinping Ren, Ke Yan, Le Lu, Lingyun Huang, Guotong Xie, Jing Xiao, Dar-In Tai, Adam P. Harrison

Importantly, ADDLE does not expect multiple raters per image in training, meaning it can readily learn from data mined from hospital archives.

Semi-Supervised Learning for Bone Mineral Density Estimation in Hip X-ray Images

no code implementations24 Mar 2021 Kang Zheng, Yirui Wang, XiaoYun Zhou, Fakai Wang, Le Lu, ChiHung Lin, Lingyun Huang, Guotong Xie, Jing Xiao, Chang-Fu Kuo, Shun Miao

Specifically, we propose a new semi-supervised self-training algorithm to train the BMD regression model using images coupled with DEXA measured BMDs and unlabeled images with pseudo BMDs.

Density Estimation regression

Learn molecular representations from large-scale unlabeled molecules for drug discovery

no code implementations21 Dec 2020 Pengyong Li, Jun Wang, Yixuan Qiao, Hao Chen, Yihuan Yu, Xiaojun Yao, Peng Gao, Guotong Xie, Sen Song

Here, we proposed a novel Molecular Pre-training Graph-based deep learning framework, named MPG, that leans molecular representations from large-scale unlabeled molecules.

Drug Discovery

Semi-supervised Active Learning for Instance Segmentation via Scoring Predictions

no code implementations9 Dec 2020 Jun Wang, Shaoguo Wen, Kaixing Chen, Jianghua Yu, Xin Zhou, Peng Gao, Changsheng Li, Guotong Xie

Active learning generally involves querying the most representative samples for human labeling, which has been widely studied in many fields such as image classification and object detection.

Active Learning Image Classification +5

AutoTrans: Automating Transformer Design via Reinforced Architecture Search

3 code implementations4 Sep 2020 Wei Zhu, Xiaoling Wang, Xipeng Qiu, Yuan Ni, Guotong Xie

Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, especially the need for a principled way of warm-up which has shown importance for stable training of a transformer, as well as whether the task at hand prefer to scale the attention product or not.

Natural Language Understanding Navigate

Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks

no code implementations WS 2019 Xiepeng Li, Zhexi Zhang, Wei Zhu, Zheng Li, Yuan Ni, Peng Gao, Junchi Yan, Guotong Xie

We have experimented both (a) improving the fine-tuning of pre-trained language models on a task with a small dataset size, by leveraging datasets of similar tasks; and (b) incorporating the distributional representations of a KG onto the representations of pre-trained language models, via simply concatenation or multi-head attention.

Common Sense Reasoning Machine Reading Comprehension +2

PANLP at MEDIQA 2019: Pre-trained Language Models, Transfer Learning and Knowledge Distillation

no code implementations WS 2019 Wei Zhu, Xiaofeng Zhou, Keqiang Wang, Xun Luo, Xiepeng Li, Yuan Ni, Guotong Xie

Transfer learning from the NLI task to the RQE task is also experimented, which proves to be useful in improving the results of fine-tuning MT-DNN large.

Knowledge Distillation Re-Ranking +1

Inpatient2Vec: Medical Representation Learning for Inpatients

no code implementations18 Apr 2019 Ying Wang, Xiao Xu, Tao Jin, Xiang Li, Guotong Xie, Jian-Min Wang

In addition, for unordered medical activity set, existing medical RL methods utilize a simple pooling strategy, which would result in indistinguishable contributions among the activities for learning.

Representation Learning Semantic Similarity +1

Cannot find the paper you are looking for? You can Submit a new open access paper.