no code implementations • EMNLP (sustainlp) 2020 • Moshe Wasserblat, Oren Pereg, Peter Izsak
We also show that the distillation of large pre-trained models is more effective in real-life scenarios where limited amounts of labeled training are available.
no code implementations • WASSA (ACL) 2022 • Ayal Klein, Oren Pereg, Daniel Korat, Vasudev Lal, Moshe Wasserblat, Ido Dagan
In this paper, we investigate and establish empirically a prior conjecture, which suggests that the linguistic relations connecting opinion terms to their aspects transfer well across domains and therefore can be leveraged for cross-domain aspect term extraction.
no code implementations • 7 May 2024 • Jonathan Mamou, Oren Pereg, Daniel Korat, Moshe Berchansky, Nadav Timor, Moshe Wasserblat, Roy Schwartz
Speculative decoding is a promising method for reducing the inference latency of large language models.
no code implementations • 16 Apr 2024 • Moshe Berchansky, Daniel Fleischer, Moshe Wasserblat, Peter Izsak
This approach focuses the reasoning process on generating an attribution-centric output.
1 code implementation • 20 Oct 2023 • Moshe Berchansky, Peter Izsak, Avi Caciularu, Ido Dagan, Moshe Wasserblat
Fusion-in-Decoder (FiD) is an effective retrieval-augmented language model applied across a variety of open-domain tasks, such as question answering, fact checking, etc.
1 code implementation • 28 Jun 2023 • Haihao Shen, Hengyu Meng, Bo Dong, Zhe Wang, Ofir Zafrir, Yi Ding, Yu Luo, Hanwen Chang, Qun Gao, Ziheng Wang, Guy Boudoukh, Moshe Wasserblat
We apply our sparse accelerator on widely-used Transformer-based language models including Bert-Mini, DistilBERT, Bert-Base, and BERT-Large.
2 code implementations • 31 Oct 2022 • Shira Guskin, Moshe Wasserblat, Chang Wang, Haihao Shen
Our quantized length-adaptive MiniLM model (QuaLA-MiniLM) is trained only once, dynamically fits any inference scenario, and achieves an accuracy-efficiency trade-off superior to any other efficient approaches per any computational budget on the SQuAD1. 1 dataset (up to x8. 8 speedup with <1% accuracy loss).
1 code implementation • 27 Oct 2022 • Haihao Shen, Ofir Zafrir, Bo Dong, Hengyu Meng, Xinyu Ye, Zhe Wang, Yi Ding, Hanwen Chang, Guy Boudoukh, Moshe Wasserblat
In this work, we propose a new pipeline for creating and running Fast Transformer models on CPUs, utilizing hardware-aware pruning, knowledge distillation, quantization, and our own Transformer inference runtime engine with optimized kernels for sparse and quantized operators.
1 code implementation • 18 Oct 2022 • Phillip Howard, Arden Ma, Vasudev Lal, Ana Paula Simoes, Daniel Korat, Oren Pereg, Moshe Wasserblat, Gadi Singer
The extraction of aspect terms is a critical step in fine-grained sentiment analysis of text.
1 code implementation • 22 Sep 2022 • Lewis Tunstall, Nils Reimers, Unso Eun Seo Jo, Luke Bates, Daniel Korat, Moshe Wasserblat, Oren Pereg
This simple framework requires no prompts or verbalizers, and achieves high accuracy with orders of magnitude less parameters than existing techniques.
no code implementations • 13 Apr 2022 • Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Roy Schwartz
In order to reduce this computational load in inference time, we present TangoBERT, a cascaded model architecture in which instances are first processed by an efficient but less accurate first tier model, and only part of those instances are additionally processed by a less efficient but more accurate second tier model.
no code implementations • 18 Nov 2021 • Shira Guskin, Moshe Wasserblat, Ke Ding, Gyuwan Kim
Additionally, a separate model must be trained for each inference scenario with its distinct computational budget.
2 code implementations • 10 Nov 2021 • Ofir Zafrir, Ariel Larey, Guy Boudoukh, Haihao Shen, Moshe Wasserblat
We show how the compressed sparse pre-trained models we trained transfer their knowledge to five different downstream natural language tasks with minimal accuracy loss.
Ranked #2 on Natural Language Inference on MultiNLI Dev
no code implementations • EACL 2021 • Vasudev Lal, Arden Ma, Estelle Aflalo, Phillip Howard, Ana Simoes, Daniel Korat, Oren Pereg, Gadi Singer, Moshe Wasserblat
With the increasingly widespread use of Transformer-based models for NLU/NLP tasks, there is growing interest in understanding the inner workings of these models, why they are so effective at a wide range of tasks, and how they can be further tuned and improved.
Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA)
no code implementations • COLING 2020 • Oren Pereg, Daniel Korat, Moshe Wasserblat
A fundamental task of fine-grained sentiment analysis is aspect and opinion terms extraction.
no code implementations • 14 Oct 2019 • Peter Izsak, Shira Guskin, Moshe Wasserblat
In this work-in-progress we combined the effectiveness of transfer learning provided by pre-trained masked language models with a semi-supervised approach to train a fast and compact model using labeled and unlabeled examples.
5 code implementations • 14 Oct 2019 • Ofir Zafrir, Guy Boudoukh, Peter Izsak, Moshe Wasserblat
Recently, pre-trained Transformer based language models such as BERT and GPT, have shown great improvement in many Natural Language Processing (NLP) tasks.
Ranked #13 on Semantic Textual Similarity on STS Benchmark
1 code implementation • IJCNLP 2019 • Oren Pereg, Daniel Korat, Moshe Wasserblat, Jonathan Mamou, Ido Dagan
We present ABSApp, a portable system for weakly-supervised aspect-based sentiment extraction.
no code implementations • WS 2019 • Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Ido Dagan
In this paper, we present a novel algorithm that combines multi-context term embeddings using a neural classifier and we test this approach on the use case of corpus-based term set expansion.
no code implementations • EMNLP 2018 • Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, Daniel Korat
We present SetExpander, a corpus-based system for expanding a seed set of terms into amore complete set of terms that belong to the same semantic class.
no code implementations • COLING 2018 • Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Ido Dagan, Yoav Goldberg, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, Daniel Korat
We present SetExpander, a corpus-based system for expanding a seed set of terms into a more complete set of terms that belong to the same semantic class.
no code implementations • 26 Jul 2018 • Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Ido Dagan, Yoav Goldberg, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, Daniel Korat
We present SetExpander, a corpus-based system for expanding a seed set of terms into a more complete set of terms that belong to the same semantic class.