Search Results for author: Seong Joon Oh

Found 47 papers, 28 papers with code

Do Deep Neural Network Solutions Form a Star Domain?

1 code implementation12 Mar 2024 Ankit Sonthalia, Alexander Rubinstein, Ehsan Abbasnejad, Seong Joon Oh

This means that two independent solutions can be connected by a linear path with low loss, given one of them is appropriately permuted.

Calibrating Large Language Models Using Their Generations Only

1 code implementation9 Mar 2024 Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, Seong Joon Oh

As large language models (LLMs) are increasingly deployed in user-facing applications, building trust and maintaining safety by accurately quantifying a model's confidence in its prediction becomes even more important.

Question Answering Text Generation

Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks

1 code implementation29 Feb 2024 Bálint Mucsányi, Michael Kirchhof, Seong Joon Oh

Uncertainty quantification, once a singular task, has evolved into a spectrum of tasks, including abstained prediction, out-of-distribution detection, and aleatoric uncertainty quantification.

Benchmarking Disentanglement +2

Pretrained Visual Uncertainties

1 code implementation26 Feb 2024 Michael Kirchhof, Mark Collier, Seong Joon Oh, Enkelejda Kasneci

Similar to standard pretraining this enables the zero-shot transfer of uncertainties learned on a large pretraining dataset to specialized downstream datasets.

Retrieval

TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification

1 code implementation20 Feb 2024 Martin Gubri, Dennis Ulmer, Hwaran Lee, Sangdoo Yun, Seong Joon Oh

Large Language Model (LLM) services and models often come with legal rules on who can use them and how they must use them.

Language Modelling Large Language Model

Mitigating Biases with Diverse Ensembles and Diffusion Models

no code implementations23 Nov 2023 Luca Scimeca, Alexander Rubinstein, Damien Teney, Seong Joon Oh, Armand Mihai Nicolicioiu, Yoshua Bengio

Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut learning, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones.

Exploring Practitioner Perspectives On Training Data Attribution Explanations

no code implementations31 Oct 2023 Elisa Nguyen, Evgenii Kortukov, Jean Y. Song, Seong Joon Oh

Explainable AI (XAI) aims to provide insight into opaque model reasoning to humans and as such is an interdisciplinary field by nature.

A Bayesian Approach To Analysing Training Data Attribution In Deep Learning

1 code implementation NeurIPS 2023 Elisa Nguyen, Minjoon Seo, Seong Joon Oh

We recommend that future researchers and practitioners trust TDA estimates only in such cases.

Playing repeated games with Large Language Models

no code implementations26 May 2023 Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, Eric Schulz

In a large set of two players-two strategies games, we find that LLMs are particularly good at games where valuing their own self-interest pays off, like the iterated Prisoner's Dilemma family.

Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts

3 code implementations30 Mar 2023 Dongyoon Han, Junsuk Choe, Seonghyeok Chun, John Joon Young Chung, Minsuk Chang, Sangdoo Yun, Jean Y. Song, Seong Joon Oh

We refer to the new paradigm of training models with annotation byproducts as learning using annotation byproducts (LUAB).

Time Series

Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs

1 code implementation6 Feb 2023 Michael Kirchhof, Enkelejda Kasneci, Seong Joon Oh

We prove that these distributions recover the correct posteriors of the data-generating process, including its level of aleatoric uncertainty, up to a rotation of the latent space.

Contrastive Learning Image Retrieval +1

SelecMix: Debiased Learning by Contradicting-pair Sampling

1 code implementation4 Nov 2022 Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, Byoung-Tak Zhang

Experiments on standard benchmarks demonstrate the effectiveness of the method, in particular when label noise complicates the identification of bias-conflicting examples.

Scratching Visual Transformer's Back with Uniform Attention

no code implementations ICCV 2023 Nam Hyeon-Woo, Kim Yu-Ji, Byeongho Heo, Doonyoon Han, Seong Joon Oh, Tae-Hyun Oh

We observe that the inclusion of CB reduces the degree of density in the original attention maps and increases both the capacity and generalizability of the ViT models.

Dataset Condensation via Efficient Synthetic-Data Parameterization

2 code implementations30 May 2022 Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, JoonHyun Jeong, Jung-Woo Ha, Hyun Oh Song

The great success of machine learning with massive amounts of data comes at a price of huge computation costs and storage for training and tuning.

Dataset Condensation

Weakly Supervised Semantic Segmentation using Out-of-Distribution Data

1 code implementation CVPR 2022 Jungbeom Lee, Seong Joon Oh, Sangdoo Yun, Junsuk Choe, Eunji Kim, Sungroh Yoon

However, training on class labels only, classifiers suffer from the spurious correlation between foreground and background cues (e. g. train and rail), fundamentally bounding the performance of WSSS.

Weakly supervised Semantic Segmentation Weakly-Supervised Semantic Segmentation

ALP: Data Augmentation using Lexicalized PCFGs for Few-Shot Text Classification

no code implementations16 Dec 2021 Hazel Kim, Daecheol Woo, Seong Joon Oh, Jeong-Won Cha, Yo-Sub Han

Taken together, our contributions on the data augmentation strategies yield a strong training recipe for few-shot text classification tasks.

Data Augmentation Few-Shot Text Classification +3

Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective

no code implementations ICLR 2022 Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Michael Poli, Sangdoo Yun

This phenomenon, also known as shortcut learning, is emerging as a key limitation of the current generation of machine learning models.

Keep CALM and Improve Visual Feature Attribution

1 code implementation ICCV 2021 Jae Myung Kim, Junsuk Choe, Zeynep Akata, Seong Joon Oh

The class activation mapping, or CAM, has been the cornerstone of feature attribution methods for multiple vision tasks.

Weakly-Supervised Object Localization

Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions

no code implementations NeurIPS 2021 Michael Poli, Stefano Massaroli, Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, Animesh Garg

Effective control and prediction of dynamical systems often require appropriate handling of continuous-time and discrete, event-triggered processes.

Rethinking Spatial Dimensions of Vision Transformers

10 code implementations ICCV 2021 Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, Seong Joon Oh

We empirically show that such a spatial dimension reduction is beneficial to a transformer architecture as well, and propose a novel Pooling-based Vision Transformer (PiT) upon the original ViT model.

Dimensionality Reduction Image Classification +2

Probabilistic Embeddings for Cross-Modal Retrieval

3 code implementations CVPR 2021 Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio de Rezende, Yannis Kalantidis, Diane Larlus

Instead, we propose to use Probabilistic Cross-Modal Embedding (PCME), where samples from the different modalities are represented as probabilistic distributions in the common embedding space.

Cross-Modal Retrieval Retrieval

VideoMix: Rethinking Data Augmentation for Video Classification

2 code implementations7 Dec 2020 Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Jinhyung Kim

Recent data augmentation strategies have been reported to address the overfitting problems in static image classifiers.

Action Localization Action Recognition +5

Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets

2 code implementations8 Jul 2020 Junsuk Choe, Seong Joon Oh, Sanghyuk Chun, Seungho Lee, Zeynep Akata, Hyunjung Shim

In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.

Few-Shot Learning Model Selection +1

AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

4 code implementations ICLR 2021 Byeongho Heo, Sanghyuk Chun, Seong Joon Oh, Dongyoon Han, Sangdoo Yun, Gyuwan Kim, Youngjung Uh, Jung-Woo Ha

Because of the scale invariance, this modification only alters the effective step sizes without changing the effective update directions, thus enjoying the original convergence properties of GD optimizers.

Audio Classification Image Classification +3

An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods

no code implementations9 Mar 2020 Sanghyuk Chun, Seong Joon Oh, Sangdoo Yun, Dongyoon Han, Junsuk Choe, Youngjoon Yoo

Despite apparent human-level performances of deep neural networks (DNN), they behave fundamentally differently from humans.

Bayesian Inference

Reliable Fidelity and Diversity Metrics for Generative Models

2 code implementations ICML 2020 Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, Jaejun Yoo

In this paper, we show that even the latest version of the precision and recall metrics are not reliable yet.

Image Generation

Evaluating Weakly Supervised Object Localization Methods Right

2 code implementations CVPR 2020 Junsuk Choe, Seong Joon Oh, Seungho Lee, Sanghyuk Chun, Zeynep Akata, Hyunjung Shim

In this paper, we argue that WSOL task is ill-posed with only image-level labels, and propose a new evaluation protocol where full supervision is limited to only a small held-out set not overlapping with the test set.

Few-Shot Learning Model Selection +2

Learning De-biased Representations with Biased Representations

3 code implementations ICML 2020 Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, Seong Joon Oh

This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias.

Modeling Uncertainty with Hedged Instance Embeddings

no code implementations ICLR 2019 Seong Joon Oh, Kevin P. Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, Andrew C. Gallagher

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering.

Clustering Metric Learning +1

Modeling Uncertainty with Hedged Instance Embedding

1 code implementation30 Sep 2018 Seong Joon Oh, Kevin Murphy, Jiyan Pan, Joseph Roth, Florian Schroff, Andrew Gallagher

Instance embeddings are an efficient and versatile image representation that facilitates applications like recognition, verification, retrieval, and clustering.

Clustering Metric Learning +1

Sequential Attacks on Agents for Long-Term Adversarial Goals

no code implementations31 May 2018 Edgar Tretschk, Seong Joon Oh, Mario Fritz

As a result of our attack, the victim agent is misguided to optimise for the adversarial reward over time.

Adversarial Attack Reinforcement Learning (RL) +1

Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning

no code implementations15 May 2018 Tribhuvanesh Orekondy, Seong Joon Oh, Yang Zhang, Bernt Schiele, Mario Fritz

At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data.

Data Augmentation Federated Learning +1

Natural and Effective Obfuscation by Head Inpainting

no code implementations CVPR 2018 Qianru Sun, Liqian Ma, Seong Joon Oh, Luc van Gool, Bernt Schiele, Mario Fritz

As more and more personal photos are shared online, being able to obfuscate identities in such photos is becoming a necessity for privacy protection.

Towards Reverse-Engineering Black-Box Neural Networks

3 code implementations ICLR 2018 Seong Joon Oh, Max Augustin, Bernt Schiele, Mario Fritz

On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model.

Person Recognition in Personal Photo Collections

no code implementations9 Oct 2017 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Person recognition in social media photos sets new challenges for computer vision, including non-cooperative subjects (e. g. backward viewpoints, unusual poses) and great changes in appearance.

Face Recognition Person Recognition

Generating Descriptions with Grounded and Co-Referenced People

no code implementations CVPR 2017 Anna Rohrbach, Marcus Rohrbach, Siyu Tang, Seong Joon Oh, Bernt Schiele

At training time, we first learn how to localize characters by relating their visual appearance to mentions in the descriptions via a semi-supervised approach.

Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective

no code implementations ICCV 2017 Seong Joon Oh, Mario Fritz, Bernt Schiele

We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.

Person Recognition

Exploiting saliency for object segmentation from image level labels

no code implementations CVPR 2017 Seong Joon Oh, Rodrigo Benenson, Anna Khoreva, Zeynep Akata, Mario Fritz, Bernt Schiele

We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.

Object Semantic Segmentation

Faceless Person Recognition; Privacy Implications in Social Media

no code implementations28 Jul 2016 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy.

Person Recognition

Person Recognition in Personal Photo Collections

no code implementations ICCV 2015 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Recognising persons in everyday photos presents major challenges (occluded faces, different clothing, locations, etc.)

Informativeness Person Recognition

Cannot find the paper you are looking for? You can Submit a new open access paper.