Search Results for author: Tomas Pfister

Found 72 papers, 28 papers with code

TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents

no code implementations3 Dec 2023 James Enouen, Hootan Nakhost, Sayna Ebrahimi, Sercan O Arik, Yan Liu, Tomas Pfister

Given their nature as black-boxes using complex reasoning processes on their inputs, it is inevitable that the demand for scalable and faithful explanations for LLMs' generated content will continue to grow.

Question Answering Text Generation

Effective Large Language Model Adaptation for Improved Grounding and Citation Generation

no code implementations16 Nov 2023 Xi Ye, Ruoxi Sun, Sercan Ö. Arik, Tomas Pfister

Our framework tunes LLMs to selfground the claims in their responses and provide accurate citations to retrieved documents.

Language Modelling Large Language Model +2

COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning

1 code implementation1 Nov 2023 Chuizheng Meng, Yihe Dong, Sercan Ö. Arik, Yan Liu, Tomas Pfister

Estimation of temporal counterfactual outcomes from observed history is crucial for decision-making in many domains such as healthcare and e-commerce, particularly when randomized controlled trials (RCTs) suffer from high cost or impracticality.

counterfactual Decision Making +2

Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs

no code implementations18 Oct 2023 Jiefeng Chen, Jinsung Yoon, Sayna Ebrahimi, Sercan O Arik, Tomas Pfister, Somesh Jha

Large language models (LLMs) have recently shown great advances in a variety of tasks, including natural language understanding and generation.

Decision Making Natural Language Understanding +1

Search-Adaptor: Embedding Customization for Information Retrieval

no code implementations12 Oct 2023 Jinsung Yoon, Sercan O Arik, Yanfei Chen, Tomas Pfister

Embeddings extracted by pre-trained Large Language Models (LLMs) have significant potential to improve information retrieval and search.

Information Retrieval Retrieval

PAITS: Pretraining and Augmentation for Irregularly-Sampled Time Series

1 code implementation25 Aug 2023 Nicasia Beebe-Wang, Sayna Ebrahimi, Jinsung Yoon, Sercan O. Arik, Tomas Pfister

In this paper, we present PAITS (Pretraining and Augmentation for Irregularly-sampled Time Series), a framework for identifying suitable pretraining strategies for sparse and irregularly sampled time series datasets.

Time Series

LANISTR: Multimodal Learning from Structured and Unstructured Data

no code implementations26 May 2023 Sayna Ebrahimi, Sercan O. Arik, Yihe Dong, Tomas Pfister

Multimodal large-scale pretraining has shown impressive performance for unstructured data including language, image, audio, and video.

Time Series

SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL

no code implementations26 May 2023 Ruoxi Sun, Sercan O. Arik, Hootan Nakhost, Hanjun Dai, Rajarishi Sinha, Pengcheng Yin, Tomas Pfister

One impressive emergent capability of large language models (LLMs) is generation of code, including Structured Query Language (SQL) for databases.

In-Context Learning Language Modelling +2

Universal Self-Adaptive Prompting

no code implementations24 May 2023 Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Martin Eisenschlos, Sercan O. Arik, Tomas Pfister

A hallmark of modern large language models (LLMs) is their impressive general zero-shot and few-shot abilities, often elicited through in-context learning (ICL) via prompting.

In-Context Learning Natural Language Understanding +2

Better Zero-Shot Reasoning with Self-Adaptive Prompting

no code implementations23 May 2023 Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan O. Arik, Tomas Pfister

Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans.

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes

1 code implementation3 May 2023 Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas Pfister

Third, we reduce both the model size and the amount of data required to outperform LLMs; our finetuned 770M T5 model outperforms the few-shot prompted 540B PaLM model using only 80% of available data on a benchmark, whereas standard finetuning the same T5 model struggles to match even by using 100% of the dataset.

ASPEST: Bridging the Gap Between Active Learning and Selective Prediction

1 code implementation7 Apr 2023 Jiefeng Chen, Jinsung Yoon, Sayna Ebrahimi, Sercan Arik, Somesh Jha, Tomas Pfister

In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain while increasing accuracy and coverage.

Active Learning

TSMixer: An All-MLP Architecture for Time Series Forecasting

2 code implementations10 Mar 2023 Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O. Arik, Tomas Pfister

Extending them, in this paper, we investigate the capabilities of linear models for time-series forecasting and present Time-Series Mixer (TSMixer), a novel architecture designed by stacking multi-layer perceptrons (MLPs).

Time Series Time Series Forecasting

Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval

1 code implementation CVPR 2023 Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas Pfister

Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image.

Attribute Retrieval +2

Neural Spline Search for Quantile Probabilistic Modeling

no code implementations12 Jan 2023 Ruoxi Sun, Chun-Liang Li, Sercan O. Arik, Michael W. Dusenberry, Chen-Yu Lee, Tomas Pfister

Accurate estimation of output quantiles is crucial in many use cases, where it is desired to model the range of possibility.

Attribute regression +2

QueryForm: A Simple Zero-shot Form Entity Query Framework

no code implementations14 Nov 2022 Zifeng Wang, Zizhao Zhang, Jacob Devlin, Chen-Yu Lee, Guolong Su, Hao Zhang, Jennifer Dy, Vincent Perot, Tomas Pfister

Zero-shot transfer learning for document understanding is a crucial yet under-investigated scenario to help reduce the high cost involved in annotating document entities.

document understanding Transfer Learning

Test-Time Adaptation for Visual Document Understanding

no code implementations15 Jun 2022 Sayna Ebrahimi, Sercan O. Arik, Tomas Pfister

For visual document understanding (VDU), self-supervised pretraining has been shown to successfully generate transferable representations, yet, effective adaptation of such representations to distribution shifts at test-time remains to be an unexplored area.

document understanding Language Modelling +5

Invariant Structure Learning for Better Generalization and Causal Explainability

no code implementations13 Jun 2022 Yunhao Ge, Sercan Ö. Arik, Jinsung Yoon, Ao Xu, Laurent Itti, Tomas Pfister

ISL splits the data into different environments, and learns a structure that is invariant to the target across different environments by imposing a consistency constraint.

Self-Supervised Learning

Interpretable Mixture of Experts

no code implementations5 Jun 2022 Aya Abdelsalam Ismail, Sercan Ö. Arik, Jinsung Yoon, Ankur Taly, Soheil Feizi, Tomas Pfister

In addition to constituting a standalone inherently-interpretable architecture, IME has the premise of being integrated with existing DNNs to offer interpretability to a subset of samples while maintaining the accuracy of the DNNs.

Decision Making Time Series

Prefix Conditioning Unifies Language and Label Supervision

no code implementations CVPR 2023 Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas Pfister

In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.

Classification Contrastive Learning +2

Data-Efficient and Interpretable Tabular Anomaly Detection

no code implementations3 Mar 2022 Chun-Hao Chang, Jinsung Yoon, Sercan Arik, Madeleine Udell, Tomas Pfister

In addition, the proposed framework, DIAD, can incorporate a small amount of labeled data to further boost anomaly detection performances in semi-supervised settings.

Additive models Anomaly Detection

Decoupling Local and Global Representations of Time Series

1 code implementation4 Feb 2022 Sana Tonekaboni, Chun-Liang Li, Sercan Arik, Anna Goldenberg, Tomas Pfister

Learning representations that capture the factors contributing to this variability enables a better understanding of the data via its underlying generative process and improves performance on downstream machine learning tasks.

counterfactual Time Series +1

Self-Adaptive Forecasting for Improved Deep Learning on Non-Stationary Time-Series

no code implementations4 Feb 2022 Sercan O. Arik, Nathanael C. Yoder, Tomas Pfister

Real-world time-series datasets often violate the assumptions of standard supervised learning for forecasting -- their distributions evolve over time, rendering the conventional training and model selection procedures suboptimal.

Model Selection Self-Supervised Learning +2

Towards Group Robustness in the presence of Partial Group Labels

no code implementations10 Jan 2022 Vishnu Suresh Lokhande, Kihyuk Sohn, Jinsung Yoon, Madeleine Udell, Chen-Yu Lee, Tomas Pfister

Such a requirement is impractical in situations where the data labeling efforts for minority or rare groups are significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.

Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types

2 code implementations21 Dec 2021 Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li, Chen-Yu Lee, Tomas Pfister

We define a distance function between images, each of which is represented as a bag of embeddings, by the Euclidean distance between weighted averaged embeddings.

Anomaly Detection Clustering +2

Learning to Prompt for Continual Learning

4 code implementations CVPR 2022 Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge.

Class Incremental Learning Image Classification

Invariant Learning with Partial Group Labels

no code implementations29 Sep 2021 Vishnu Suresh Lokhande, Kihyuk Sohn, Jinsung Yoon, Madeleine Udell, Chen-Yu Lee, Tomas Pfister

Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.

Unifying Distribution Alignment as a Loss for Imbalanced Semi-supervised Learning

no code implementations29 Sep 2021 Justin Lazarow, Kihyuk Sohn, Chun-Liang Li, Zizhao Zhang, Chen-Yu Lee, Tomas Pfister

While remarkable progress in imbalanced supervised learning has been made recently, less attention has been given to the setting of imbalanced semi-supervised learning (SSL) where not only is a few labeled data provided, but the underlying data distribution can be severely imbalanced.

Pseudo Label

Learning Fast Sample Re-weighting Without Reward Data

1 code implementation ICCV 2021 Zizhao Zhang, Tomas Pfister

Training sample re-weighting is an effective approach for tackling data biases such as imbalanced and corrupted labels.

Meta-Learning

Controlling Neural Networks with Rule Representations

1 code implementation NeurIPS 2021 Sungyong Seo, Sercan O. Arik, Jinsung Yoon, Xiang Zhang, Kihyuk Sohn, Tomas Pfister

The key aspect of DeepCTRL is that it does not require retraining to adapt the rule strength -- at inference, the user can adjust it based on the desired operation point on accuracy vs. rule verification ratio.

Decision Making

Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding

6 code implementations26 May 2021 Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, Sercan O. Arik, Tomas Pfister

Hierarchical structures are popular in recent vision transformers, however, they require sophisticated designs and massive datasets to work well.

Image Classification Image Generation

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization

2 code implementations CVPR 2021 Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, Tomas Pfister

We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data.

Data Augmentation Defect Detection +4

Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts

no code implementations11 Jan 2021 Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Zhichao Lu, Yun Fu, Tomas Pfister

To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.

Action Recognition Pseudo Label +1

Exploring Sub-Pseudo Labels for Learning from Weakly-Labeled Web Videos

no code implementations1 Jan 2021 Kunpeng Li, Zizhao Zhang, Guanhang Wu, Xuehan Xiong, Chen-Yu Lee, Yun Fu, Tomas Pfister

To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos.

Action Recognition Pseudo Label +1

Differentiable Top-k with Optimal Transport

no code implementations NeurIPS 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

Finding the k largest or smallest elements from a collection of scores, i. e., top-k operation, is an important model component widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

Differentiable Top-$k$ with Optimal Transport

no code implementations NeurIPS Workshop LMCA 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

The top-$k$ operation, i. e., finding the $k$ largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

A Simple Semi-Supervised Learning Framework for Object Detection

7 code implementations10 May 2020 Kihyuk Sohn, Zizhao Zhang, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister

Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data.

Ranked #13 on Semi-Supervised Object Detection on COCO 100% labeled data (using extra training data)

Data Augmentation Image Classification +4

Differentiable Top-k Operator with Optimal Transport

no code implementations16 Feb 2020 Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, Tomas Pfister

The top-k operation, i. e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.

Information Retrieval Retrieval

Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting

33 code implementations19 Dec 2019 Bryan Lim, Sercan O. Arik, Nicolas Loeff, Tomas Pfister

Multi-horizon forecasting problems often contain a complex mix of inputs -- including static (i. e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed historically -- without any prior information on how they interact with the target.

Interpretable Machine Learning Time Series +1

Distance-Based Learning from Errors for Confidence Calibration

no code implementations ICLR 2020 Chen Xing, Sercan Arik, Zizhao Zhang, Tomas Pfister

To circumvent this by inferring the distance for every test sample, we propose to train a confidence model jointly with the classification model.

Classification General Classification

On Completeness-aware Concept-Based Explanations in Deep Neural Networks

2 code implementations NeurIPS 2020 Chih-Kuan Yeh, Been Kim, Sercan O. Arik, Chun-Liang Li, Tomas Pfister, Pradeep Ravikumar

Next, we propose a concept discovery method that aims to infer a complete set of concepts that are additionally encouraged to be interpretable, which addresses the limitations of existing methods on concept explanations.

Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Cost

no code implementations ECCV 2020 Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan O. Arik, Larry S. Davis, Tomas Pfister

Active learning (AL) combines data labeling and model training to minimize the labeling cost by prioritizing the selection of high value data that can best improve model performance.

Active Learning Image Classification +1

Generative Modeling for Small-Data Object Detection

1 code implementation ICCV 2019 Lanlan Liu, Michael Muelly, Jia Deng, Tomas Pfister, Li-Jia Li

This paper explores object detection in the small data regime, where only a limited number of annotated bounding boxes are available due to data rarity and annotation expense.

Object object-detection +4

Distilling Effective Supervision from Severe Label Noise

2 code implementations CVPR 2020 Zizhao Zhang, Han Zhang, Sercan O. Arik, Honglak Lee, Tomas Pfister

For instance, on CIFAR100 with a $40\%$ uniform noise ratio and only 10 trusted labeled data per class, our method achieves $80. 2{\pm}0. 3\%$ classification accuracy, where the error rate is only $1. 4\%$ higher than a neural network trained without label noise.

Image Classification

LIMIS: Locally Interpretable Modeling using Instance-wise Subsampling

1 code implementation26 Sep 2019 Jinsung Yoon, Sercan O. Arik, Tomas Pfister

Understanding black-box machine learning models is crucial for their widespread adoption.

Reinforcement Learning (RL)

On Concept-Based Explanations in Deep Neural Networks

no code implementations25 Sep 2019 Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Pradeep Ravikumar, Tomas Pfister

Next, we propose a concept discovery method that considers two additional constraints to encourage the interpretability of the discovered concepts.

Data Valuation using Reinforcement Learning

1 code implementation ICML 2020 Jinsung Yoon, Sercan O. Arik, Tomas Pfister

To adaptively learn data values jointly with the target task predictor model, we propose a meta learning framework which we name Data Valuation using Reinforcement Learning (DVRL).

Data Valuation Domain Adaptation +4

Consistency-Based Semi-Supervised Active Learning: Towards Minimizing Labeling Budget

no code implementations25 Sep 2019 Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan O. Arik, Larry S. Davis, Tomas Pfister

Active learning (AL) aims to integrate data labeling and model training in a unified way, and to minimize the labeling budget by prioritizing the selection of high value data that can best improve model performance.

Active Learning Representation Learning

A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels

no code implementations20 Sep 2019 Yucen Luo, Jun Zhu, Tomas Pfister

Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance.

Learning with noisy labels

Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning

no code implementations ECCV 2020 Linchao Zhu, Sercan O. Arik, Yi Yang, Tomas Pfister

We propose a novel adaptive transfer learning framework, learning to transfer learn (L2TL), to improve performance on a target dataset by careful extraction of the related information from a source dataset.

reinforcement-learning Reinforcement Learning (RL) +1

TabNet: Attentive Interpretable Tabular Learning

19 code implementations20 Aug 2019 Sercan O. Arik, Tomas Pfister

We propose a novel high-performance and interpretable canonical deep tabular data learning architecture, TabNet.

Decision Making Poker Hand Classification +2

Inserting Videos into Videos

no code implementations CVPR 2019 Donghoon Lee, Tomas Pfister, Ming-Hsuan Yang

To synthesize a realistic video, the network renders each frame based on the current input and previous frames.

Object Object Tracking +1

Harmonic Unpaired Image-to-image Translation

no code implementations ICLR 2019 Rui Zhang, Tomas Pfister, Jia Li

The recent direction of unpaired image-to-image translation is on one hand very exciting as it alleviates the big burden in obtaining label-intensive pixel-to-pixel supervision, but it is on the other hand not fully satisfactory due to the presence of artifacts and degenerated transformations.

Image-to-Image Translation Translation

ProtoAttend: Attention-Based Prototypical Learning

4 code implementations17 Feb 2019 Sercan O. Arik, Tomas Pfister

We propose a novel inherently interpretable machine learning method that bases decisions on few relevant examples that we call prototypes.

Decision Making General Classification +1

Learning from Simulated and Unsupervised Images through Adversarial Training

9 code implementations CVPR 2017 Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, Russ Webb

With recent progress in graphics, it has become more tractable to train models on synthetic images, potentially avoiding the need for expensive annotations.

Ranked #3 on Image-to-Image Translation on Cityscapes Labels-to-Photo (Per-class Accuracy metric)

Domain Adaptation Gaze Estimation +2

Personalizing Human Video Pose Estimation

no code implementations CVPR 2016 James Charles, Tomas Pfister, Derek Magee, David Hogg, Andrew Zisserman

The outcome is a substantial improvement in the pose estimates for the target video using the personalized ConvNet compared to the original generic ConvNet.

Optical Flow Estimation Pose Estimation

Cannot find the paper you are looking for? You can Submit a new open access paper.