Search Results for author:
Found 8 papers, 2 papers with code
The Falcon Series of Open Language Models
We report detailed evaluations, as well as a deep dive into the methods and custom tooling employed to pretrain Falcon.
Ranked #17 on
Sentence Completion
on HellaSwag
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only
Large language models are commonly trained on a mixture of filtered web data and curated high-quality corpora, such as social media conversations, books, or technical papers.
Scaling Laws Beyond Backpropagation
Alternatives to backpropagation have long been studied to better understand how biological brains may learn.
PAGnol: An Extra-Large French Generative Model
We fit a scaling law for compute for the French language, and compare it with its English counterpart.
ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients
Robustness to adversarial attacks is typically obtained through expensive adversarial training with Projected Gradient Descent.
Photonic Differential Privacy with Direct Feedback Alignment
Optical Processing Units (OPUs) -- low-power photonic chips dedicated to large scale random projections -- have been used in previous work to train deep neural networks using Direct Feedback Alignment (DFA), an effective alternative to backpropagation.
Photonic co-processors in HPC: using LightOn OPUs for Randomized Numerical Linear Algebra
Randomized Numerical Linear Algebra (RandNLA) is a powerful class of methods, widely used in High Performance Computing (HPC).
Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
In the white-box setting, our defense works by obfuscating the parameters of the random projection.
