Search Results for author: Sarah Schwettmann

Found 8 papers, 4 papers with code

Automatic Discovery of Visual Circuits

1 code implementation22 Apr 2024 Achyuta Rajaram, Neil Chowdhury, Antonio Torralba, Jacob Andreas, Sarah Schwettmann

To date, most discoveries of network subcomponents that implement human-interpretable computations in deep vision models have involved close study of single units and large amounts of human labor.

A Multimodal Automated Interpretability Agent

no code implementations22 Apr 2024 Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, Antonio Torralba

Interpretability experiments proposed by MAIA compose these tools to describe and explain system behavior.

Language Modelling

FIND: A Function Description Benchmark for Evaluating Interpretability Methods

1 code implementation NeurIPS 2023 Sarah Schwettmann, Tamar Rott Shaham, Joanna Materzynska, Neil Chowdhury, Shuang Li, Jacob Andreas, David Bau, Antonio Torralba

FIND contains functions that resemble components of trained neural networks, and accompanying descriptions of the kind we seek to generate.

Multimodal Neurons in Pretrained Text-Only Transformers

no code implementations3 Aug 2023 Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, Antonio Torralba

Language models demonstrate remarkable capacity to generalize representations learned in one modality to downstream tasks in other modalities.

Image Captioning

Natural Language Descriptions of Deep Visual Features

2 code implementations26 Jan 2022 Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, Jacob Andreas

Given a neuron, MILAN generates a description by searching for a natural language string that maximizes pointwise mutual information with the image regions in which the neuron is active.

Attribute

Toward a Visual Concept Vocabulary for GAN Latent Space

1 code implementation ICCV 2021 Sarah Schwettmann, Evan Hernandez, David Bau, Samuel Klein, Jacob Andreas, Antonio Torralba

A large body of recent work has identified transformations in the latent spaces of generative adversarial networks (GANs) that consistently and interpretably transform generated images.

Disentanglement

Natural Language Descriptions of Deep Features

no code implementations ICLR 2022 Evan Hernandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, Jacob Andreas

Given a neuron, MILAN generates a description by searching for a natural language string that maximizes pointwise mutual information with the image regions in which the neuron is active.

Attribute

Latent Compass: Creation by Navigation

no code implementations20 Dec 2020 Sarah Schwettmann, Hendrik Strobelt, Mauro Martino

Our approach puts creators in the discovery loop during real-time tool use, in order to identify directions that are perceptually meaningful to them, and generate interpretable image translations along those directions.

Image Manipulation

Cannot find the paper you are looking for? You can Submit a new open access paper.