Search Results for author: Duen Horng Chau

Found 71 papers, 43 papers with code

UniTable: Towards a Unified Framework for Table Structure Recognition via Self-Supervised Pretraining

1 code implementation7 Mar 2024 Shengyun Peng, Seongmin Lee, XiaoJing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

Tables convey factual and quantitative data with implicit conventions created by humans that are often challenging for machines to parse.

Language Modelling

Self-Supervised Pre-Training for Table Structure Recognition Transformer

1 code implementation23 Feb 2024 Shengyun Peng, Seongmin Lee, XiaoJing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model.

Representation Learning

Mobile Fitting Room: On-device Virtual Try-on via Diffusion Models

no code implementations2 Feb 2024 Justin Blalock, David Munechika, Harsha Karanth, Alec Helbling, Pratham Mehta, Seongmin Lee, Duen Horng Chau

The growing digital landscape of fashion e-commerce calls for interactive and user-friendly interfaces for virtually trying on clothes.

Image Generation Model Compression +1

Wordflow: Social Prompt Engineering for Large Language Models

1 code implementation25 Jan 2024 Zijie J. Wang, Aishwarya Chakravarthy, David Munechika, Duen Horng Chau

To investigate social prompt engineering, we introduce Wordflow, an open-source and social text editor that enables everyday users to easily create, run, share, and discover LLM prompts.

Prompt Engineering

High-Performance Transformers for Table Structure Recognition Need Early Convolutions

2 code implementations9 Nov 2023 Shengyun Peng, Seongmin Lee, XiaoJing Wang, Rajarajeswari Balasubramaniyan, Duen Horng Chau

This allows it to "see" an appropriate portion of the table and "store" the complex table structure within sufficient context length for the subsequent transformer.

Representation Learning Self-Supervised Learning +1

REVAMP: Automated Simulations of Adversarial Attacks on Arbitrary Objects in Realistic Scenes

1 code implementation18 Oct 2023 Matthew Hull, Zijie J. Wang, Duen Horng Chau

Generating these adversarial objects in the digital space has been extensively studied, however successfully transferring these attacks from the digital realm to the physical realm has proven challenging when controlling for real-world environmental factors.

Object

ObjectComposer: Consistent Generation of Multiple Objects Without Fine-tuning

no code implementations10 Oct 2023 Alec Helbling, Evan Montoya, Duen Horng Chau

We build upon the recent BLIP-Diffusion model, which can generate images of single objects specified by reference images.

Robust Principles: Architectural Design Principles for Adversarially Robust CNNs

1 code implementation30 Aug 2023 Shengyun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, Duen Horng Chau

Our research aims to unify existing works' diverging opinions on how architectural components affect the adversarial robustness of CNNs.

Adversarial Robustness

LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked

no code implementations14 Aug 2023 Mansi Phute, Alec Helbling, Matthew Hull, Shengyun Peng, Sebastian Szyller, Cory Cornelius, Duen Horng Chau

We test LLM Self Defense on GPT 3. 5 and Llama 2, two of the current most prominent LLMs against various types of attacks, such as forcefully inducing affirmative responses to prompts and prompt engineering attacks.

Language Modelling Large Language Model +2

ManimML: Communicating Machine Learning Architectures with Animation

1 code implementation29 Jun 2023 Alec Helbling, Duen Horng Chau

A user can take a preexisting neural network architecture and easily write a specification for an animation in ManimML, which will then automatically compose animations for different components of the system into a final animation of the entire neural network.

WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings

2 code implementations15 Jun 2023 Zijie J. Wang, Fred Hohman, Duen Horng Chau

Machine learning models often learn latent embedding representations that capture the domain semantics of their training data.

Navigate

SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks

2 code implementations4 May 2023 Zijie J. Wang, David Munechika, Seongmin Lee, Duen Horng Chau

Through this work, we identify unique design opportunities and considerations for future notebook VA tools, such as using and manipulating multimodal data in notebooks as well as balancing the degree of visualization-notebook integration.

Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion

1 code implementation4 May 2023 Seongmin Lee, Benjamin Hoover, Hendrik Strobelt, Zijie J. Wang, Shengyun Peng, Austin Wright, Kevin Li, Haekyu Park, Haoyang Yang, Duen Horng Chau

Diffusion Explainer tightly integrates a visual overview of Stable Diffusion's complex components with detailed explanations of their underlying operations, enabling users to fluidly transition between multiple levels of abstraction through animations and interactive elements.

Image Generation

WebSHAP: Towards Explaining Any Machine Learning Models Anywhere

1 code implementation16 Mar 2023 Zijie J. Wang, Duen Horng Chau

As machine learning (ML) is increasingly integrated into our everyday Web experience, there is a call for transparent and explainable web-based ML.

GAM Coach: Towards Interactive and User-centered Algorithmic Recourse

1 code implementation27 Feb 2023 Zijie J. Wang, Jennifer Wortman Vaughan, Rich Caruana, Duen Horng Chau

Machine learning (ML) recourse techniques are increasingly used in high-stakes domains, providing end users with actions to alter ML predictions, but they assume ML developers understand what input variables can be changed.

Additive models counterfactual

Lessons from the Development of an Anomaly Detection Interface on the Mars Perseverance Rover using the ISHMAP Framework

no code implementations14 Feb 2023 Austin P. Wright, Peter Nemere, Adrian Galvin, Duen Horng Chau, Scott Davidoff

While anomaly detection stands among the most important and valuable problems across many scientific domains, anomaly detection research often focuses on AI methods that can lack the nuance and interpretability so critical to conducting scientific inquiry.

Anomaly Detection

Energy Transformer

4 code implementations NeurIPS 2023 Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobelt, Duen Horng Chau, Mohammed J. Zaki, Dmitry Krotov

Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory.

Graph Anomaly Detection Graph Classification

RobArch: Designing Robust Architectures against Adversarial Attacks

1 code implementation8 Jan 2023 Shengyun Peng, Weilin Xu, Cory Cornelius, Kevin Li, Rahul Duggal, Duen Horng Chau, Jason Martin

Adversarial Training is the most effective approach for improving the robustness of Deep Neural Networks (DNNs).

DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models

2 code implementations26 Oct 2022 Zijie J. Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, Duen Horng Chau

With recent advancements in diffusion models, users can generate high-quality images by writing text prompts in natural language.

Misinformation

NeuroMapper: In-browser Visualizer for Neural Network Training

1 code implementation22 Oct 2022 Zhiyan Zhou, Kevin Li, Haekyu Park, Megan Dass, Austin Wright, Nilaksh Das, Duen Horng Chau

We present our ongoing work NeuroMapper, an in-browser visualization tool that helps machine learning (ML) developers interpret the evolution of a model during training, providing a new way to monitor the training process and visually discover reasons for suboptimal training.

Dimensionality Reduction

IMB-NAS: Neural Architecture Search for Imbalanced Datasets

no code implementations30 Sep 2022 Rahul Duggal, Shengyun Peng, Hao Zhou, Duen Horng Chau

In this paper, we propose a new and complementary direction for improving performance on long tailed datasets - optimizing the backbone architecture through neural architecture search (NAS).

Neural Architecture Search Representation Learning

Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values

2 code implementations30 Jun 2022 Zijie J. Wang, Alex Kale, Harsha Nori, Peter Stella, Mark E. Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, Rich Caruana

Machine learning (ML) interpretability techniques can reveal undesirable patterns in data that models exploit to make predictions--potentially causing harms once deployed.

Additive models BIG-bench Machine Learning +1

Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases

no code implementations25 Jun 2022 David Munechika, Zijie J. Wang, Jack Reidy, Josh Rubin, Krishna Gade, Krishnaram Kenthapadi, Duen Horng Chau

Recent research has developed algorithms for effectively identifying intersectional bias in the form of interpretable, underperforming subsets (or slices) of the data.

Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning

no code implementations5 Apr 2022 Nilaksh Das, Duen Horng Chau

In this work, we investigate the impact of performing such multi-task learning on the adversarial robustness of ASR models in the speech domain.

Adversarial Attack Adversarial Robustness +4

SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning

1 code implementation2 Apr 2022 Nilaksh Das, Sheng-Yun Peng, Duen Horng Chau

Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics.

Adversarial Robustness Autonomous Driving +3

Concept Evolution in Deep Learning Training: A Unified Interpretation Framework and Discoveries

no code implementations30 Mar 2022 Haekyu Park, Seongmin Lee, Benjamin Hoover, Austin P. Wright, Omar Shaikh, Rahul Duggal, Nilaksh Das, Kevin Li, Judy Hoffman, Duen Horng Chau

We present ConceptEvo, a unified interpretation framework for deep neural networks (DNNs) that reveals the inception and evolution of learned concepts during training.

Decision Making

GAM Changer: Editing Generalized Additive Models with Interactive Visualization

1 code implementation6 Dec 2021 Zijie J. Wang, Alex Kale, Harsha Nori, Peter Stella, Mark Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, Rich Caruana

Recent strides in interpretable machine learning (ML) research reveal that models exploit undesirable patterns in the data to make predictions, which potentially causes harms in deployment.

Additive models Interpretable Machine Learning

A Search Engine for Discovery of Scientific Challenges and Directions

1 code implementation NeurIPS Workshop AI4Scien 2021 Dan Lahav, Jon Saad Falcon, Bailey Kuehl, Sophie Johnson, Sravanthi Parasa, Noam Shomron, Duen Horng Chau, Diyi Yang, Eric Horvitz, Daniel S. Weld, Tom Hope

To address this problem, we present a novel task of extraction and search of scientific challenges and directions, to facilitate rapid knowledge discovery.

NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks

1 code implementation29 Aug 2021 Haekyu Park, Nilaksh Das, Rahul Duggal, Austin P. Wright, Omar Shaikh, Fred Hohman, Duen Horng Chau

Through a large-scale human evaluation, we demonstrate that our technique discovers neuron groups that represent coherent, human-meaningful concepts.

Semantic Similarity Semantic Textual Similarity

Quantifying the Impact of Human Capital, Job History, and Language Factors on Job Seniority with a Large-scale Analysis of Resumes

no code implementations15 Jun 2021 Austin P Wright, Caleb Ziems, Haekyu Park, Jon Saad-Falcon, Duen Horng Chau, Diyi Yang, Maria Tomprou

As job markets worldwide have become more competitive and applicant selection criteria have become more opaque, and different (and sometimes contradictory) information and advice is available for job seekers wishing to progress in their careers, it has never been more difficult to determine which factors in a r\'esum\'e most effectively help career progression.

EnergyVis: Interactively Tracking and Exploring Energy Consumption for ML Models

2 code implementations30 Mar 2021 Omar Shaikh, Jon Saad-Falcon, Austin P Wright, Nilaksh Das, Scott Freitas, Omar Isaac Asensio, Duen Horng Chau

The advent of larger machine learning (ML) models have improved state-of-the-art (SOTA) performance in various modeling tasks, ranging from computer vision to natural language.

Dodrio: Exploring Transformer Models with Interactive Visualization

1 code implementation ACL 2021 Zijie J. Wang, Robert Turko, Duen Horng Chau

Why do large pre-trained transformer-based models perform so well across a wide variety of NLP tasks?

RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization

no code implementations8 Feb 2021 Austin P Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Duen Horng Chau, Diyi Yang

With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments.

MalNet: A Large-Scale Image Database of Malicious Software

1 code implementation31 Jan 2021 Scott Freitas, Rahul Duggal, Duen Horng Chau

Computer vision is playing an increasingly important role in automated malware detection with the rise of the image-based binary representation.

Feature Engineering imbalanced classification +1

SkeletonVis: Interactive Visualization for Understanding Adversarial Attacks on Human Action Recognition Models

no code implementations26 Jan 2021 Haekyu Park, Zijie J. Wang, Nilaksh Das, Anindya S. Paul, Pruthvi Perumalla, Zhiyan Zhou, Duen Horng Chau

Skeleton-based human action recognition technologies are increasingly used in video based applications, such as home robotics, healthcare on aging population, and surveillance.

Action Recognition Temporal Action Localization

A Large-Scale Database for Graph Representation Learning

2 code implementations16 Nov 2020 Scott Freitas, Yuxiao Dong, Joshua Neil, Duen Horng Chau

With the rapid emergence of graph representation learning, the construction of new large-scale datasets is necessary to distinguish model capabilities and accurately assess the strengths and weaknesses of each technique.

Graph Representation Learning imbalanced classification

A Comparative Analysis of Industry Human-AI Interaction Guidelines

no code implementations22 Oct 2020 Austin P. Wright, Zijie J. Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Mennatallah El-Assady, Alex Endert, Daniel Keim, Duen Horng Chau

We have then used this framework to compare each of the surveyed companies to find differences in areas of emphasis.

Human-Computer Interaction

Examining the Ordering of Rhetorical Strategies in Persuasive Requests

1 code implementation Findings of the Association for Computational Linguistics 2020 Omar Shaikh, Jiaao Chen, Jon Saad-Falcon, Duen Horng Chau, Diyi Yang

We find that specific (orderings of) strategies interact uniquely with a request's content to impact success rate, and thus the persuasiveness of a request.

Persuasiveness

Mapping Researchers with PeopleMap

1 code implementation31 Aug 2020 Jon Saad-Falcon, Omar Shaikh, Zijie J. Wang, Austin P. Wright, Sasha Richardson, Duen Horng Chau

Discovering research expertise at universities can be a difficult task.

Evaluating Graph Vulnerability and Robustness using TIGER

1 code implementation10 Jun 2020 Scott Freitas, Diyi Yang, Srijan Kumar, Hanghang Tong, Duen Horng Chau

By democratizing the tools required to study network robustness, our goal is to assist researchers and practitioners in analyzing their own networks; and facilitate the development of new research in the field.

CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization

5 code implementations30 Apr 2020 Zijie J. Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, Duen Horng Chau

Deep learning's great success motivates many practitioners and students to learn about this exciting technology.

UnMask: Adversarial Detection and Defense Through Robust Feature Alignment

2 code implementations21 Feb 2020 Scott Freitas, Shang-Tse Chen, Zijie J. Wang, Duen Horng Chau

UnMask detects such attacks and defends the model by rectifying the misclassification, re-classifying the image based on its robust features.

Medical Diagnosis Self-Driving Cars

REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild

1 code implementation29 Jan 2020 Rahul Duggal, Scott Freitas, Cao Xiao, Duen Horng Chau, Jimeng Sun

By deploying these models to an Android application on a smartphone, we quantitatively observe that REST allows models to achieve up to 17x energy reduction and 9x faster inference.

EEG Electroencephalogram (EEG) +2

Massif: Interactive Interpretation of Adversarial Attacks on Deep Learning

no code implementations21 Jan 2020 Nilaksh Das, Haekyu Park, Zijie J. Wang, Fred Hohman, Robert Firstman, Emily Rogers, Duen Horng Chau

Deep neural networks (DNNs) are increasingly powering high-stakes applications such as autonomous cars and healthcare; however, DNNs are often treated as "black boxes" in such applications.

Adversarial Attack

RECAST: Interactive Auditing of Automatic Toxicity Detection Models

no code implementations7 Jan 2020 Austin P. Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Diyi Yang, Duen Horng Chau

As toxic language becomes nearly pervasive online, there has been increasing interest in leveraging the advancements in natural language processing (NLP), from very large transformer models to automatically detecting and removing toxic comments.

Adversarial Robustness Fairness

CNN 101: Interactive Visual Learning for Convolutional Neural Networks

no code implementations7 Jan 2020 Zijie J. Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, Duen Horng Chau

The success of deep learning solving previously-thought hard problems has inspired many non-experts to learn and understand this exciting technology.

ElectroLens: Understanding Atomistic Simulations Through Spatially-resolved Visualization of High-dimensional Features

no code implementations20 Aug 2019 Xiangyun Lei, Fred Hohman, Duen Horng Chau, Andrew J. Medford

In recent years, machine learning (ML) has gained significant popularity in the field of chemical informatics and electronic structure theory.

BIG-bench Machine Learning

NeuralDivergence: Exploring and Understanding Neural Networks by Comparing Activation Distributions

no code implementations2 Jun 2019 Haekyu Park, Fred Hohman, Duen Horng Chau

As deep neural networks are increasingly used in solving high-stake problems, there is a pressing need to understand their internal decision mechanisms.

Talk Proposal: Towards the Realistic Evaluation of Evasion Attacks using CARLA

3 code implementations18 Apr 2019 Cory Cornelius, Shang-Tse Chen, Jason Martin, Duen Horng Chau

In this talk we describe our content-preserving attack on object detectors, ShapeShifter, and demonstrate how to evaluate this threat in realistic scenarios.

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

1 code implementation10 Apr 2019 Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng Chau

We present FairVis, a mixed-initiative visual analytics system that integrates a novel subgroup discovery technique for users to audit the fairness of machine learning models.

BIG-bench Machine Learning Fairness +1

GOGGLES: Automatic Image Labeling with Affinity Coding

1 code implementation11 Mar 2019 Nilaksh Das, Sanya Chaba, Renzhi Wu, Sakshi Gandhi, Duen Horng Chau, Xu Chu

We build the GOGGLES system that implements affinity coding for labeling image datasets by designing a novel set of reusable affinity functions for images, and propose a novel hierarchical generative model for class inference using a small development set.

Few-Shot Learning

The Efficacy of SHIELD under Different Threat Models

no code implementations1 Feb 2019 Cory Cornelius, Nilaksh Das, Shang-Tse Chen, Li Chen, Michael E. Kounavis, Duen Horng Chau

To evaluate the robustness of the defense against an adaptive attacker, we consider the targeted-attack success rate of the Projected Gradient Descent (PGD) attack, which is a strong gradient-based adversarial attack proposed in adversarial machine learning research.

Adversarial Attack Image Classification

GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation

1 code implementation5 Sep 2018 Minsuk Kahng, Nikhil Thorat, Duen Horng Chau, Fernanda Viégas, Martin Wattenberg

Recent success in deep learning has generated immense interest among practitioners and students, inspiring many to learn about this new technology.

Interactive Classification for Deep Learning Interpretation

1 code implementation14 Jun 2018 Ángel Alexander Cabrera, Fred Hohman, Jason Lin, Duen Horng Chau

We present an interactive system enabling users to manipulate images to explore the robustness and sensitivity of deep learning image classifiers.

Classification General Classification

ADAGIO: Interactive Experimentation with Adversarial Attack and Defense for Audio

no code implementations30 May 2018 Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Li Chen, Michael E. Kounavis, Duen Horng Chau

Adversarial machine learning research has recently demonstrated the feasibility to confuse automatic speech recognition (ASR) models by introducing acoustically imperceptible perturbations to audio samples.

Adversarial Attack Automatic Speech Recognition +2

ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

3 code implementations16 Apr 2018 Shang-Tse Chen, Cory Cornelius, Jason Martin, Duen Horng Chau

Given the ability to directly manipulate image pixels in the digital input space, an adversary can easily generate imperceptible perturbations to fool a Deep Neural Network (DNN) image classifier, as demonstrated in prior work.

Adversarial Attack Autonomous Vehicles +5

Shield: Fast, Practical Defense and Vaccination for Deep Learning using JPEG Compression

3 code implementations19 Feb 2018 Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E. Kounavis, Duen Horng Chau

The rapidly growing body of research in adversarial machine learning has demonstrated that deep neural networks (DNNs) are highly vulnerable to adversarially generated images.

Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers

no code implementations21 Jan 2018 Fred Hohman, Minsuk Kahng, Robert Pienta, Duen Horng Chau

We present a survey of the role of visual analytics in deep learning research, which highlights its short yet impactful history and thoroughly summarizes the state-of-the-art using a human-centered interrogative framework, focusing on the Five W's and How (Why, Who, What, How, When, and Where).

Decision Making

Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression

no code implementations8 May 2017 Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, Duen Horng Chau

Deep neural networks (DNNs) have achieved great success in solving a variety of machine learning (ML) problems, especially in the domain of image recognition.

ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

no code implementations6 Apr 2017 Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, Duen Horng Chau

While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge.

M3: Scaling Up Machine Learning via Memory Mapping

no code implementations11 Apr 2016 Dezhi Fang, Duen Horng Chau

To process data that do not fit in RAM, conventional wisdom would suggest using distributed approaches.

BIG-bench Machine Learning Graph Mining +1

Communication Efficient Distributed Agnostic Boosting

no code implementations21 Jun 2015 Shang-Tse Chen, Maria-Florina Balcan, Duen Horng Chau

We consider the problem of learning from distributed data in the agnostic setting, i. e., in the presence of arbitrary forms of noise.

Cannot find the paper you are looking for? You can Submit a new open access paper.