no code implementations • 16 Feb 2024 • Dylan Zhang, Justin Wang, Francois Charton
We investigate the trade-off between the number of instructions the model is trained on and the number of training samples provided for each instruction and observe that the diversity of the instruction set determines generalization.
no code implementations • 10 Feb 2024 • Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, Julia Kempe
We discover a wide range of decay phenomena, analyzing loss of scaling, shifted scaling with number of generations, the ''un-learning" of skills, and grokking when mixing human and synthesized data.
no code implementations • 7 Mar 2023 • Cathy Li, Jana Sotáková, Emily Wenger, Mohamed Malhou, Evrard Garcelon, Francois Charton, Kristin Lauter
However, this attack assumes access to millions of eavesdropped LWE samples and fails at higher Hamming weights or dimensions.
1 code implementation • 30 Jun 2022 • Marc Szafraniec, Baptiste Roziere, Hugh Leather, Francois Charton, Patrick Labatut, Gabriel Synnaeve
Here we propose to augment code translation with IRs, specifically LLVM IR, with results on the C++, Java, Rust, and Go languages.
1 code implementation • ICLR 2022 • Baptiste Roziere, Jie M. Zhang, Francois Charton, Mark Harman, Gabriel Synnaeve, Guillaume Lample
With little to no parallel data available for programming languages, unsupervised methods are well-suited to source code translation.
no code implementations • 29 Sep 2021 • Francois Charton
Most applications of transformers to mathematics, from integration to theorem proving, focus on symbolic computation.
no code implementations • 25 Sep 2019 • Jean-Remi King, Francois Charton, Maxime Oquab, David Lopez-Paz
Identifying causes from observations can be particularly challenging when i) potential factors are difficult to manipulate individually and ii) observations are complex and multi-dimensional.