Search Results for author:
Found 2 papers, 1 papers with code
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
Autoregressive transformers are spectacular models for short sequences but scale poorly to long sequences such as high-resolution images, podcasts, code, or books.
SemDeDup: Data-efficient learning at web-scale through semantic deduplication
Analyzing a subset of LAION, we show that SemDeDup can remove 50% of the data with minimal performance loss, effectively halving training time.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
