CommitBench: A Benchmark for Commit Message Generation

1 code implementation • 8 Mar 2024 • Maximilian Schall, Tamara Czinczoll, Gerard de Melo

Writing commit messages is a tedious daily task for many software developers, and often remains neglected.

NextLevelBERT: Investigating Masked Language Modeling with Higher-Level Representations for Long Documents

1 code implementation • 27 Feb 2024 • Tamara Czinczoll, Christoph Hönes, Maximilian Schall, Gerard de Melo

While (large) language models have significantly improved over the last years, they still struggle to sensibly process long sequences found, e. g., in books, due to the quadratic scaling of the underlying attention mechanism.

Document Classification Language Modelling +4

Efficient Parallelization Layouts for Large-Scale Distributed Model Training

1 code implementation • 9 Nov 2023 • Johannes Hagemann, Samuel Weinbach, Konstantin Dobler, Maximilian Schall, Gerard de Melo

In this work, we conduct a comprehensive ablation study of possible training configurations for large language models.