Search Results for author: Mengsay Loem

Found 7 papers, 0 papers with code

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities

no code implementations27 Apr 2024 Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, Naoaki Okazaki

The results showed that the efficiency gained through vocabulary expansion had no negative impact on performance, except for the summarization task, and that the combined use of parallel corpora enhanced translation ability.

Question Answering

Building a Large Japanese Web Corpus for Large Language Models

no code implementations27 Apr 2024 Naoaki Okazaki, Kakeru Hattori, Hirai Shota, Hiroki Iida, Masanari Ohi, Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Rio Yokota, Sakae Mizuki

Open Japanese large language models (LLMs) have been trained on the Japanese portions of corpora such as CC-100, mC4, and OSCAR.

SAIE Framework: Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks

no code implementations14 Nov 2023 Mengsay Loem, Masahiro Kaneko, Naoaki Okazaki

Large Language Models (LLMs) can justify or critique their predictions through discussions with other models or humans, thereby enriching their intrinsic understanding of instances.

GSM8K Math

Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention

no code implementations27 Jul 2022 Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Impressive performance of Transformer has been attributed to self-attention, where dependencies between entire input in a sequence are considered at every position.

Position

ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization

no code implementations NAACL (ACL) 2022 Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Through experiments, we show that ExtraPhrase improves the performance of abstractive summarization tasks by more than 0. 50 points in ROUGE scores compared to the setting without data augmentation.

Abstractive Text Summarization Data Augmentation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.