no code implementations • 30 Apr 2024 • Samuel Lavoie, Polina Kirichenko, Mark Ibrahim, Mahmoud Assran, Andrew Gordon Wildon, Aaron Courville, Nicolas Ballas
Contrastive Language Pretraining (CLIP) on the other hand, works by mapping an image and its caption to a single vector -- limiting how well CLIP-like models can represent the diverse ways to describe an image.
no code implementations • 25 Apr 2024 • Mazda Moayeri, Michael Rabbat, Mark Ibrahim, Diane Bouchacourt
We propose a method to encode and account for diversity within a class using inferred attributes, still in the zero-shot setting without retraining.
no code implementations • 16 Apr 2024 • Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, Mark Ibrahim
A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability.
no code implementations • 11 Feb 2024 • Caner Hazirbas, Alicia Sun, Yonathan Efroni, Mark Ibrahim
Despite the remarkable performance of foundation vision-language models, the shared representation space for text and vision can also encode harmful label associations detrimental to fairness.
1 code implementation • 27 Nov 2023 • Youssef Benchekroun, Megi Dervishi, Mark Ibrahim, Jean-Baptiste Gaya, Xavier Martinet, Grégoire Mialon, Thomas Scialom, Emmanuel Dupoux, Dieuwke Hupkes, Pascal Vincent
We propose WorldSense, a benchmark designed to assess the extent to which LLMs are consistently able to sustain tacit world models, by testing how they draw simple inferences from descriptions of simple arrangements of entities.
no code implementations • 15 Nov 2023 • Cian Eastwood, Julius von Kügelgen, Linus Ericsson, Diane Bouchacourt, Pascal Vincent, Bernhard Schölkopf, Mark Ibrahim
Self-supervised representation learning often uses data augmentations to induce some invariance to "style" attributes of the data.
2 code implementations • NeurIPS 2023 • Micah Goldblum, Hossein Souri, Renkun Ni, Manli Shu, Viraj Prabhu, Gowthami Somepalli, Prithvijit Chattopadhyay, Mark Ibrahim, Adrien Bardes, Judy Hoffman, Rama Chellappa, Andrew Gordon Wilson, Tom Goldstein
Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more.
no code implementations • 28 Sep 2023 • Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim, Nicolas Ballas, Pascal Vincent, David Lopez-Paz
Successful out-of-distribution generalization requires environment annotations.
no code implementations • 26 Sep 2023 • Jiachen Sun, Mark Ibrahim, Melissa Hall, Ivan Evtimov, Z. Morley Mao, Cristian Canton Ferrer, Caner Hazirbas
Inspired by the success of textual prompting, several studies have investigated the efficacy of visual prompt tuning.
1 code implementation • NeurIPS 2023 • Florian Bordes, Shashank Shekhar, Mark Ibrahim, Diane Bouchacourt, Pascal Vincent, Ari S. Morcos
Synthetic image datasets offer unmatched advantages for designing and evaluating deep neural networks: they make it possible to (i) render as many data samples as needed, (ii) precisely control each scene and yield granular ground truth labels (and captions), (iii) precisely control distribution shifts between training and testing to isolate variables of interest for sound experimentation.
no code implementations • 24 Jul 2023 • Megan Richards, Polina Kirichenko, Diane Bouchacourt, Mark Ibrahim
Second, we study model generalization across geographies by measuring the disparities in performance across regions, a more fine-grained measure of real world generalization.
no code implementations • 24 Apr 2023 • Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, Avi Schwarzschild, Andrew Gordon Wilson, Jonas Geiping, Quentin Garrido, Pierre Fernandez, Amir Bar, Hamed Pirsiavash, Yann Lecun, Micah Goldblum
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning.
1 code implementation • 11 Apr 2023 • Laura Gustafson, Megan Richards, Melissa Hall, Caner Hazirbas, Diane Bouchacourt, Mark Ibrahim
As an example, we show that mitigating a model's vulnerability to texture can improve performance on the lower income level.
1 code implementation • CVPR 2023 • Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, Mark Ibrahim
Key to advancing the reliability of vision systems is understanding whether existing methods can overcome multiple shortcuts or struggle in a Whac-A-Mole game, i. e., where mitigating one shortcut amplifies reliance on others.
Ranked #1 on Out-of-Distribution Generalization on ImageNet-W
no code implementations • 3 Nov 2022 • Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, Mark Ibrahim
Equipped with ImageNet-X, we investigate 2, 200 current recognition models and study the types of mistakes as a function of model's (1) architecture, e. g. transformer vs. convolutional, (2) learning paradigm, e. g. supervised vs. self-supervised, and (3) training procedures, e. g., data augmentation.
no code implementations • 24 Oct 2022 • Mark Ibrahim, Quentin Garrido, Ari Morcos, Diane Bouchacourt
We study not only how robust recent state-of-the-art models are, but also the extent to which models can generalize variation in factors when they're present during training.
no code implementations • 24 Oct 2022 • Mark Ibrahim, Diane Bouchacourt, Ari Morcos
Our approach applies the formalism of Lie groups to capture continuous transformations to improve models' robustness to distributional shifts.
1 code implementation • 13 Oct 2022 • Karsten Roth, Mark Ibrahim, Zeynep Akata, Pascal Vincent, Diane Bouchacourt
We show that the use of HFS consistently facilitates disentanglement and recovery of ground-truth factors across a variety of correlation settings and benchmarks, even under severe training correlations and correlation shifts, with in parts over $+60\%$ in relative improvement over existing disentanglement methods.
1 code implementation • NeurIPS 2021 • Brian Knott, Shobha Venkataraman, Awni Hannun, Shubho Sengupta, Mark Ibrahim, Laurens van der Maaten
To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks.
1 code implementation • NeurIPS 2021 • Diane Bouchacourt, Mark Ibrahim, Ari S. Morcos
While prior work has focused on synthetic data, we attempt here to characterize the factors of variation in a real dataset, ImageNet, and study the invariance of both standard residual networks and the recently proposed vision transformer with respect to changes in these factors.
1 code implementation • NeurIPS 2021 • Diane Bouchacourt, Mark Ibrahim, Ari S. Morcos
While prior work has focused on synthetic data, we attempt here to characterize the factors of variation in a real dataset, ImageNet, and study the invariance of both standard residual networks and the recently proposed vision transformer with respect to changes in these factors.
1 code implementation • 10 Feb 2021 • Diane Bouchacourt, Mark Ibrahim, Stéphane Deny
A core challenge in Machine Learning is to learn to disentangle natural factors of variation in data (e. g. object shape vs. pose).
no code implementations • 1 Jan 2021 • Diane Bouchacourt, Mark Ibrahim, Stephane Deny
A core challenge in Machine Learning is to disentangle natural factors of variation in data (e. g. object shape vs pose).
1 code implementation • 6 Feb 2019 • Mark Ibrahim, Melissa Louie, Ceena Modarres, John Paisley
A barrier to the wider adoption of neural networks is their lack of interpretability.
no code implementations • 23 Dec 2018 • Ghazal Fazelnia, Mark Ibrahim, Ceena Modarres, Kevin Wu, John Paisley
Models for sequential data such as the recurrent neural network (RNN) often implicitly model a sequence as having a fixed time interval between observations and do not account for group-level effects when multiple sequences are observed.
no code implementations • 15 Nov 2018 • Ceena Modarres, Mark Ibrahim, Melissa Louie, John Paisley
Deep learning adoption in the financial services industry has been limited due to a lack of model interpretability.