LlamBERT: Large-scale low-cost data annotation in NLP
Large Language Models (LLMs), such as GPT-4 and Llama 2, show remarkable proficiency in a wide range of natural language processing (NLP) tasks. Despite their effectiveness, the high costs associated with their use pose a challenge. We present LlamBERT, a hybrid approach that leverages LLMs to annotate a small subset of large, unlabeled databases and uses the results for fine-tuning transformer encoders like BERT and RoBERTa. This strategy is evaluated on two diverse datasets: the IMDb review dataset and the UMLS Meta-Thesaurus. Our results indicate that the LlamBERT approach slightly compromises on accuracy while offering much greater cost-effectiveness.
PDF Abstract
Datasets
IMDb Movie Reviews
Results from the Paper
Ranked #1 on
Sentiment Analysis
on IMDb
(using extra training data)
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Uses Extra Training Data |
Benchmark |
|---|---|---|---|---|---|---|---|
| Sentiment Analysis | IMDb | Llama-2-70b-chat (0-shot) | Accuracy | 95.39 | # 16 | ||
| Sentiment Analysis | IMDb | RoBERTa-large with LlamBERT | Accuracy | 96.68 | # 1 | ||
| Sentiment Analysis | IMDb | RoBERTa-large | Accuracy | 96.54 | # 2 |
