Search Results for author:
Found 67 papers, 23 papers with code
STENCIL: Submodular Mutual Information Based Weak Supervision for Cold-Start Active Learning
As supervised fine-tuning of pre-trained models within NLP applications increases in popularity, larger corpora of annotated data are required, especially with increasing parameter counts in large language models.
Theoretical Analysis of Submodular Information Measures for Targeted Data Subset Selection
With increasing volume of data being used across machine learning tasks, the capability to target specific subsets of data becomes more important.
Gradient Coreset for Federated Learning
We conduct experiments using four real-world datasets and show that GCFL is (1) more compute and energy efficient than FL, (2) robust to various kinds of noise in both the feature space and labels, (3) preserves the privacy of the validation dataset, and (4) introduces a small communication overhead but achieves significant gains in performance, particularly in cases when the clients' data is noisy.
SCoRe: Submodular Combinatorial Representation Learning for Real-World Class-Imbalanced Settings
In this paper, we introduce the SCoRe (Submodular Combinatorial Representation Learning) framework and propose a family of Submodular Combinatorial Loss functions to overcome these pitfalls in contrastive learning.
Beyond Active Learning: Leveraging the Full Potential of Human Interaction via Auto-Labeling, Human Correction, and Human Verification
Active Learning (AL) is a human-in-the-loop framework to interactively and adaptively label data instances, thereby enabling significant gains in model performance compared to random sampling.
STREAMLINE: Streaming Active Learning for Realistic Multi-Distributional Settings
However, learning unbiased models depends on building a dataset that is representative of a diverse range of realistic scenarios for a given task.
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models
A salient characteristic of pre-trained language models (PTLMs) is a remarkable improvement in their generalization capability and emergence of new capabilities with increasing model capacity and pre-training dataset size.
MILO: Model-Agnostic Subset Selection Framework for Efficient Model Training and Tuning
Training deep networks and tuning hyperparameters on large datasets is computationally intensive.
Partitioned Gradient Matching-based Data Subset Selection for Compute-Efficient Robust ASR Training
Training state-of-the-art ASR systems such as RNN-T often has a high associated financial and environmental cost.
CLINICAL: Targeted Active Learning for Imbalanced Medical Image Classification
It is often the case that a suboptimal performance is obtained on some classes due to the natural class imbalance issue that comes with medical data.
DIAGNOSE: Avoiding Out-of-distribution Data using Submodular Information Measures
Avoiding out-of-distribution (OOD) data is critical for training supervised machine learning models in the medical imaging domain.
Active Data Discovery: Mining Unknown Data using Submodular Information Measures
Most approaches assume access to a seed set of instances which contain these rare data instances.
AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient Hyper-parameter Tuning
Our central insight is that using an informative subset of the dataset for model training runs involved in hyper-parameter optimization, allows us to find the optimal hyper-parameter configuration significantly faster.
BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets
This issue is further pronounced in SSL methods, as they would use this biased model to obtain psuedo-labels (on the unlabeled data) during training.
Submodlib: A Submodular Optimization Library
A recent work has also leveraged submodular functions to propose submodular information measures which have been found to be very useful in solving the problems of guided subset selection and guided summarization.
PLATINUM: Semi-Supervised Model Agnostic Meta-Learning using Submodular Mutual Information
Meta learning has proven to be able to learn a parametrized model for FSC by training on various other classification tasks.
Learning to Select Exogenous Events for Marked Temporal Point Process
Marked temporal point processes (MTPPs) have emerged as a powerful modelingtool for a wide variety of applications which are characterized using discreteevents localized in continuous time.
TALISMAN: Targeted Active Learning for Object Detection with Rare Classes and Slices using Submodular Mutual Information
We propose TALISMAN, a novel framework for Targeted Active Learning or object detectIon with rare slices using Submodular MutuAl iNformation.
GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning
To address this, replay-based CL approaches maintain and repeatedly retrain on a small buffer of data selected across encountered tasks.
DITTO: Data-efficient and Fair Targeted Subset Selection for ASR Accent Adaptation
To address this problem, we propose DITTO (Data-efficient and faIr Targeted subseT selectiOn) that uses Submodular Mutual Information (SMI) functions as acquisition functions to find the most informative set of utterances matching a target accent within a fixed budget.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming
These LFs, in turn, have been used to generate a large amount of additional noisy labeled data, in a paradigm that is now commonly referred to as data programming.
SPEAR : Semi-supervised Data Programming in Python
SPEAR facilitates weak supervision in the form of heuristics (or rules) and association of noisy labels to the training dataset.
SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios
Active learning has proven to be useful for minimizing labeling costs by selecting the most informative samples.
Training Data Subset Selection for Regression with Controlled Generalization Error
First, we represent this problem with simplified constraints using the dual of the original training problem and show that the objective of this new representation is a monotone and alpha-submodular function, for a wide variety of modeling choices.
Effective Evaluation of Deep Active Learning on Image Classification Tasks
Issues in the current literature include sometimes contradictory observations on the performance of different AL algorithms, unintended exclusion of important generalization approaches such as data augmentation and SGD for optimization, a lack of study of evaluation facets like the labeling efficiency of AL, and little or no clarity on the scenarios in which AL outperforms random sampling (RS).
RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning
In this work, we propose RETRIEVE, a coreset selection framework for efficient and robust semi-supervised learning.
Fair Machine Learning under Limited Demographically Labeled Data
In such a setting, we design fair training algorithms which exhibit both good utility, and low bias.
Rule Augmented Unsupervised Constituency Parsing
We introduce a novel formulation that takes advantage of the syntactic grammar rules and is independent of the base system.
Submodular Mutual Information for Targeted Data Subset Selection
With the rapid growth of data, it is becoming increasingly difficult to train or improve deep learning models with the right subset of data.
PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection
Examples of such problems include: i)targeted learning, where the goal is to find subsets with rare classes or rare attributes on which the model is underperforming, and ii)guided summarization, where data (e. g., image collection, text, document or video) is summarized for quicker human consumption with specific additional user intent.
GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training
We show rigorous theoretical and convergence guarantees of the proposed algorithm and, through our extensive experiments on real-world datasets, show the effectiveness of our proposed framework.
How Good is a Video Summary? A New Benchmarking Dataset and Evaluation Framework Towards Realistic Video Summarization
For long videos, human reference summaries necessary for supervised video summarization techniques are difficult to obtain.
GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning
Finally, we propose Glister-Active, an extension to batch active learning, and we empirically demonstrate the performance of Glister on a wide range of tasks including, (a) data selection to reduce training time, (b) robust learning under label noise and imbalance settings, and (c) batch-active learning with several deep and shallow models.
A Nested Bi-level Optimization Framework for Robust Few Shot Learning
Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal.
Deep Submodular Networks for Extractive Data Summarization
Unfortunately, these models only learn the relative importance of the different submodular functions (such as diversity, representation or importance), but cannot learn more complex feature representations, which are often required for state-of-the-art performance.
Improving Accuracy of Federated Learning in Non-IID Settings
Particularly, in settings where local data distributions vastly differ among agents, FL performs rather poorly with respect to the centralized training.
A Unified Framework for Generic, Query-Focused, Privacy Preserving and Update Summarization using Submodular Information Measures
We study submodular information measures as a rich framework for generic, query-focused, privacy sensitive, and update summarization tasks.
How Out-of-Distribution Data Hurts Semi-Supervised Learning
This work addresses the following question: How do out-of-distribution (OOD) data adversely affect semi-supervised learning algorithms?
Semi-Supervised Data Programming with Subset Selection
The first contribution of this work is an introduction of a framework, \model which is a semi-supervised data programming paradigm that learns a \emph{joint model} that effectively uses the rules/labelling functions along with semi-supervised loss functions on the feature space.
Realistic Video Summarization through VISIOCITY: A New Benchmark and Evaluation Framework
Thirdly, we demonstrate that in the presence of multiple ground truth summaries (due to the highly subjective nature of the task), learning from a single combined ground truth summary using a single loss function is not a good idea.
Concave Aspects of Submodular Functions
In this paper, we try to provide a more complete picture of the relationship between submodularity with concavity.
Submodular Combinatorial Information Measures with Applications in Machine Learning
In this paper, we study combinatorial information measures that generalize independence, (conditional) entropy, (conditional) mutual information, and total correlation defined over sets of (not necessarily random) variables.
Robust Submodular Minimization with Applications to Cooperative Modeling
While several existing papers have studied robust submodular maximization, ours is the first work to study the minimization version under a broad range of combinatorial constraints including cardinality, knapsack, matroid as well as graph-based constraints such as cuts, paths, matchings, and trees.
A Unified Framework of Constrained Robust Submodular Optimization with Applications
Furthermore, we also study robust submodular minimization and maximization under multiple submodular upper and lower bound constraints.
Near Optimal Algorithms for Hard Submodular Programs with Discounted Cooperative Costs
In this paper, we investigate a class of submodular problems which in general are very hard.
A Memoization Framework for Scaling Submodular Optimization to Large Scale Problems
We are motivated by large scale submodular optimization problems, where standard algorithms that treat the submodular functions in the \emph{value oracle model} do not scale.
Demystifying Multi-Faceted Video Summarization: Tradeoff Between Diversity,Representation, Coverage and Importance
This paper addresses automatic summarization of videos in a unified manner.
Learning From Less Data: A Unified Data Subset Selection and Active Learning Framework for Computer Vision
Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry.
A Framework towards Domain Specific Video Summarization
We propose a novel framework for domain specific video summarization.
Vis-DSS: An Open-Source toolkit for Visual Data Selection and Summarization
With increasing amounts of visual data being created in the form of videos and images, visual data selection and summarization are becoming ever increasing problems.
A Unified Batch Online Learning Framework for Click Prediction
Through extensive experiments, we demonstrate the utility of of our OL framework; how the two OL schemes relate to each other and how they trade-off between the new and historical data.
Jensen: An Easily-Extensible C++ Toolkit for Production-Level Machine Learning and Convex Optimization
This paper introduces Jensen, an easily extensible and scalable toolkit for production-level machine learning and convex optimization.
Learning From Less Data: Diversified Subset Selection and Active Learning in Image Classification Tasks
Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry and pose the challenges of not having adequate computing resources and of high costs involved in human labeling efforts.
Deployment of Customized Deep Learning based Video Analytics On Surveillance Cameras
To our knowledge, this is the first work which provides a comprehensive evaluation of different deep learning models on various real-world customer deployment scenarios of surveillance video analytics.
Modeling and Simultaneously Removing Bias via Adversarial Neural Networks
In real world systems, the predictions of deployed Machine Learned models affect the training data available to build subsequent models.
A Unified Multi-Faceted Video Summarization System
Most importantly, we also show that we can summarize hours of video data in a few seconds, and our system allows the user to generate summaries of various lengths and types interactively on the fly.
Submodular Hamming Metrics
We show that there is a largely unexplored class of functions (positive polymatroids) that can define proper discrete metrics over pairs of binary vectors and that are fairly tractable to optimize over.
Mixed Robust/Average Submodular Partitioning: Fast Algorithms, Guarantees, and Applications to Parallel Machine Learning and Multi-Label Image Segmentation
While the robust versions have been studied in the theory community, existing work has focused on tight approximation guarantees, and the resultant algorithms are not, in general, scalable to very large real-world applications.
Polyhedral aspects of Submodularity, Convexity and Concavity
This manuscript provides a more complete picture on the relationship between submodularity with convexity and concavity, by extending many of the results connecting submodularity with convexity to the concave aspects of submodularity.
Discrete Mathematics Data Structures and Algorithms
The Lovasz-Bregman Divergence and connections to rank aggregation, clustering, and web ranking
We show how a number of recently used web ranking models are forms of Lovasz-Bregman rank aggregation and also observe that a natural form of Mallow's model using the LB divergence has been used as conditional ranking models for the "Learning to Rank" problem.
Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications
We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a dierence between submodular functions.
Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints
We are motivated by a number of real-world applications in machine learning including sensor placement and data subset selection, which require maximizing a certain submodular function (like coverage or diversity) while simultaneously minimizing another (like cooperative cost).
Curvature and Optimal Algorithms for Learning and Minimizing Submodular Functions
We either use a black-box transformation of the function (for approximation and learning), or a transformation of algorithms to use an appropriate surrogate function (for minimization).
The Lovasz-Bregman Divergence and connections to rank aggregation, clustering, and web ranking
We show how a number of recently used web ranking models are forms of Lovasz-Bregman rank aggregation and also observe that a natural form of Mallow's model using the LB divergence has been used as conditional ranking models for the 'Learning to Rank' problem.
Fast Semidifferential-based Submodular Function Optimization
We present a practical and powerful new framework for both unconstrained and constrained submodular function optimization based on discrete semidifferentials (sub- and super-differentials).
Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications
We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a difference between submodular functions.
