Search Results for author: Zi Yang

Found 11 papers, 3 papers with code

COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization

no code implementations11 Mar 2024 Aozhong zhang, Zi Yang, Naigang Wang, Yingyong Qin, Jack Xin, Xin Li, Penghang Yin

Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error.

Quantization

Attendre: Wait To Attend By Retrieval With Evicted Queries in Memory-Based Transformers for Long Context Processing

no code implementations10 Jan 2024 Zi Yang, Nan Hua

As LLMs have become capable of processing more complex types of inputs, researchers have recently studied how to efficiently and affordably process possibly arbitrarily long sequences.

Reading Comprehension Retrieval +1

Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understanding

no code implementations1 Jun 2023 Zi Yang, Samridhi Choudhary, Siegfried Kunzmann, Zheng Zhang

To improve the convergence, a layer-by-layer distillation is applied to distill a quantized and tensor-compressed student model from a pre-trained transformer.

Natural Language Understanding Quantization

Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior

1 code implementation Findings of the Association for Computational Linguistics 2020 Zi Lin, Jeremiah Zhe Liu, Zi Yang, Nan Hua, Dan Roth

Traditional (unstructured) pruning methods for a Transformer model focus on regularizing the individual weights by penalizing them toward zero.

Towards a Human-like Open-Domain Chatbot

2 code implementations27 Jan 2020 Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, Quoc V. Le

We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations.

Chatbot Specificity

Best arm identification in multi-armed bandits with delayed feedback

no code implementations29 Mar 2018 Aditya Grover, Todor Markov, Peter Attia, Norman Jin, Nicholas Perkins, Bryan Cheong, Michael Chen, Zi Yang, Stephen Harris, William Chueh, Stefano Ermon

We propose a generalization of the best arm identification problem in stochastic multi-armed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback.

Hyperparameter Optimization Multi-Armed Bandits

Tackling Biomedical Text Summarization: OAQA at BioASQ 5B

no code implementations WS 2017 Khyathi u, Aakanksha Naik, Ch, Aditya rasekar, Zi Yang, Niloy Gupta, Eric Nyberg

In this paper, we describe our participation in phase B of task 5b of the fifth edition of the annual BioASQ challenge, which includes answering factoid, list, yes-no and summary questions from biomedical data.

Answer Generation Clustering +4

Structural Embedding of Syntactic Trees for Machine Comprehension

no code implementations EMNLP 2017 Rui Liu, Junjie Hu, Wei Wei, Zi Yang, Eric Nyberg

Deep neural networks for machine comprehension typically utilizes only word or character embeddings without explicitly taking advantage of structured linguistic information such as constituency trees and dependency trees.

Question Answering Reading Comprehension

Cannot find the paper you are looking for? You can Submit a new open access paper.