2 code implementations • 4 Jan 2024 • Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, Wei Lu
We present TinyLlama, a compact 1. 1B language model pretrained on around 1 trillion tokens for approximately 3 epochs.
1 code implementation • 2 Jun 2023 • Tianduo Wang, Wei Lu
Mathematical reasoning is regarded as a necessary ability for Language Models (LMs).
Ranked #2 on Math Word Problem Solving on MAWPS
1 code implementation • 29 Oct 2022 • Tianduo Wang, Wei Lu
Fine-tuning a pre-trained language model via the contrastive learning framework with a large amount of unlabeled sentences or labeled sentence pairs is a common way to obtain high-quality sentence representations.