Search Results for author: Yi Lu

Found 35 papers, 7 papers with code

LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration

1 code implementation18 Feb 2024 Jun Zhao, Can Zu, Hao Xu, Yi Lu, wei he, Yiwen Ding, Tao Gui, Qi Zhang, Xuanjing Huang

Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks.

Multi-hop Question Answering Question Answering +1

LongHeads: Multi-Head Attention is Secretly a Long Context Processor

1 code implementation16 Feb 2024 Yi Lu, Xin Zhou, wei he, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Instead of allowing each head to attend to the full sentence, which struggles with generalizing to longer sequences due to out-of-distribution (OOD) issues, we allow each head to process in-distribution length by selecting and attending to important context chunks.

Sentence

LKCA: Large Kernel Convolutional Attention

1 code implementation11 Jan 2024 Chenghao Li, Boheng Zeng, Yi Lu, Pengbo Shi, Qingzi Chen, Jirui Liu, Lingyun Zhu

We revisit the relationship between attention mechanisms and large kernel ConvNets in visual transformers and propose a new spatial attention named Large Kernel Convolutional Attention (LKCA).

Making Harmful Behaviors Unlearnable for Large Language Models

no code implementations2 Nov 2023 Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

Specifically, we introduce ``security vectors'', a few new parameters that can be separated from the LLM, to ensure LLM's responses are consistent with the harmful behavior.

Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection

no code implementations1 Feb 2023 Chenglong Wang, Yi Lu, Yongyu Mu, Yimin Hu, Tong Xiao, Jingbo Zhu

Knowledge distillation addresses the problem of transferring knowledge from a teacher model to a student model.

Knowledge Distillation

Joint RIS Calibration and Multi-User Positioning

no code implementations8 Dec 2022 Yi Lu, Hui Chen, Jukka Talvitie, Henk Wymeersch, Mikko Valkama

Reconfigurable intelligent surfaces (RISs) are expected to be a key component enabling the mobile network evolution towards a flexible and intelligent 6G wireless platform.

Nonparametric Decoding for Generative Retrieval

1 code implementation5 Oct 2022 Hyunji Lee, Jaeyoung Kim, Hoyeon Chang, Hanseok Oh, Sohee Yang, Vlad Karpukhin, Yi Lu, Minjoon Seo

The generative retrieval model depends solely on the information encoded in its model parameters without external memory, its information capacity is limited and fixed.

Language Modelling Retrieval +1

The Short-term Impact of Congestion Taxes on Ridesourcing Demand and Traffic Congestion: Evidence from Chicago

no code implementations5 Jul 2022 Yuan Liang, Bingjie Yu, Xiaojian Zhang, Yi Lu, Linchuan Yang

To this end, this study applies difference-in-differences (i. e., a regression-based causal inference approach) to empirically evaluate the effects of the congestion tax policy on ridesourcing demand and traffic congestion in Chicago.

Causal Inference regression

DePS: An improved deep learning model for de novo peptide sequencing

no code implementations16 Mar 2022 Cheng Ge, Yi Lu, Jia Qu, Liangxu Xie, Feng Wang, Hong Zhang, Ren Kong, Shan Chang

De novo peptide sequencing from mass spectrometry data is an important method for protein identification.

de novo peptide sequencing

C+1 Loss: Learn to Classify C Classes of Interest and the Background Class Differentially

no code implementations29 Sep 2021 Changhuai Chen, Xile Shen, Mengyu Ye, Yi Lu, Jun Che, ShiLiang Pu

We figure out that the background class should be treated differently from the classes of interest during training.

Classification Human Parsing +3

Joint Positioning and Tracking via NR Sidelink in 5G-Empowered Industrial IoT: Releasing the Potential of V2X Technology

no code implementations15 Jan 2021 Yi Lu, Mike Koivisto, Jukka Talvitie, Elizaveta Rastorgueva-Foi, Toni Levanen, Elena Simona Lohan, Mikko Valkama

The fifth generation (5G) mobile networks with enhanced connectivity and positioning capabilities play an increasingly important role in the development of automated vehicle-to-everything (V2X) and other advanced industrial Internet of Things (IoT) systems.

On the Transferability of Minimal Prediction Preserving Inputs in Question Answering

no code implementations NAACL 2021 Shayne Longpre, Yi Lu, Christopher DuBois

In the context of question answering, we investigate competing hypotheses for the existence of MPPIs, including poor posterior calibration of neural models, lack of pretraining, and "dataset bias" (where a model learns to attend to spurious, non-generalizable cues in the training data).

Adversarial Robustness Question Answering

Prob2Vec: Mathematical Semantic Embedding for Problem Retrieval in Adaptive Tutoring

no code implementations ICLR 2019 Du Su, Ali Yekkehkhany, Yi Lu, Wenmiao Lu

We propose a hierarchical problem embedding algorithm, called Prob2Vec, that consists of abstraction and embedding steps.

Retrieval Sentence +2

Artificial Intelligence Distinguishes COVID-19 from Community Acquired Pneumonia on Chest CT

1 code implementation Radiology 2020 Lin Li, Lixin Qin, Zeguo Xu, Youbing Yin, Xin Wang, Bin Kong, Junjie Bai, Yi Lu, Zhenghan Fang, Qi Song, Kunlin Cao, Daliang Liu, Guisheng Wang, Qizhong Xu, Xisheng Fang, Shiqin Zhang, Juan Xia, Jun Xia

Materials and Methods In this retrospective and multi-center study, a deep learning model, COVID-19 detection neural network (COVNet), was developed to extract visual features from volumetric chest CT exams for the detection of COVID-19.

COVID-19 Image Segmentation Specificity

Graph-FCN for image semantic segmentation

no code implementations2 Jan 2020 Yi Lu, Yaran Chen, Dongbin Zhao, Jianxin Chen

Then we apply graph convolutional network to solve this graph node classification problem.

General Classification Node Classification +2

An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering

no code implementations WS 2019 Shayne Longpre, Yi Lu, Zhucheng Tu, Chris DuBois

To produce a domain-agnostic question answering model for the Machine Reading Question Answering (MRQA) 2019 Shared Task, we investigate the relative benefits of large pre-trained language models, various data sampling strategies, as well as query and context paraphrases generated by back-translation.

Data Augmentation Question Answering +2

DeepCenterline: a Multi-task Fully Convolutional Network for Centerline Extraction

no code implementations25 Mar 2019 Zhihui Guo, Junjie Bai, Yi Lu, Xin Wang, Kunlin Cao, Qi Song, Milan Sonka, Youbing Yin

The proposed method generates well-positioned centerlines, exhibiting lower number of missing branches and is more robust in the presence of minor imperfections of the object segmentation mask.

Object Semantic Segmentation

Attention-driven Tree-structured Convolutional LSTM for High Dimensional Data Understanding

no code implementations29 Jan 2019 Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Kunlin Cao, Qi Song, Shaoting Zhang, Siwei Lyu, Youbing Yin

In order to address these limitations, we present tree-structured ConvLSTM models for tree-structured image analysis tasks which can be trained end-to-end.

Vocal Bursts Intensity Prediction

Residual Attention based Network for Hand Bone Age Assessment

no code implementations21 Dec 2018 Eric Wu, Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Shaoting Zhang, Kunlin Cao, Qi Song, Siwei Lyu, Youbing Yin

The hierarchical attention components of the residual attention subnet force our network to focus on the key components of the X-ray images and generate the final predictions as well as the associated visual supports, which is similar to the assessment procedure of clinicians.

Hand Segmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.