Search Results for author:
Found 25 papers, 17 papers with code
If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptions
EX2 uses reinforcement learning to align a large language model with VLM preferences and generates descriptions that incorporate the important features for the VLM.
Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation
Overall, we show that learning with synthetic instruction tuning datasets is an effective way to adapt language models to new domains.
LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons
We show that conditioning on bilingual lexicons is the key component of LexC-Gen. LexC-Gen is also practical -- it only needs a single GPU to generate data at scale.
Leveraging Large Language Models for Structure Learning in Prompted Weak Supervision
We propose a Structure Refining Module, a simple yet effective first approach based on the similarities of the prompts by taking advantage of the intrinsic structure in the embedding space.
Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification
However, current zero-shot methods select a subset of attributes regardless of commonalities between the target classes, potentially providing no useful information that would have helped to distinguish between them.
Low-Resource Languages Jailbreak GPT-4
AI safety training and red-teaming of large language models (LLMs) are measures to mitigate the generation of unsafe content.
Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning
We find that (1) unexplored prompt tuning strategies that iteratively refine pseudolabels consistently improve CLIP accuracy, by 19. 5 points in semi-supervised learning, by 28. 4 points in transductive zero-shot learning, and by 15. 2 points in unsupervised learning, and (2) unlike conventional semi-supervised pseudolabeling, which exacerbates model biases toward classes with higher-quality pseudolabels, prompt tuning leads to a more equitable distribution of per-class accuracy.
An Adaptive Method for Weak Supervision with Drifting Data
In particular, at each step, our algorithm guarantees an estimation of the current accuracies of the weak supervision sources over a window of past observations that minimizes a trade-off between the error due to the variance of the estimation and the error due to the drift.
Does CLIP Bind Concepts? Probing Compositionality in Large Image Models
In this work, we focus on the ability of a large pretrained vision and language model (CLIP) to encode compositional concepts and to bind variables in a structure-sensitive way (e. g., differentiating ''cube behind sphere'' from ''sphere behind cube'').
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Tight Lower Bounds on Worst-Case Guarantees for Zero-Shot Learning with Attributes
We develop the first non-trivial lower bound on the worst-case error of the best map from attributes to classes for this setting, even with perfect attribute detectors.
Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
We then leverage these properties to propose a novel evaluation framework which can quantitatively measure disparities in the quality of explanations output by state-of-the-art methods.
Language Models in the Loop: Incorporating Prompting into Weak Supervision
Our experimental evaluation shows that prompting large language models within a weak supervision framework can provide significant gains in accuracy.
Learning to Compose Soft Prompts for Compositional Zero-Shot Learning
We perform additional experiments to show that CSP improves generalization to higher-order attribute-attribute-object compositions (e. g., old white cat) and combinations of pretrained attributes and fine-tuned objects.
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
PromptSource is a system for creating, sharing, and using natural language prompts.
TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data
Machine learning practitioners often have access to a spectrum of data: labeled data for the target task (which is often limited), unlabeled data, and auxiliary data, the many available labeled datasets for other tasks.
Multitask Prompted Training Enables Zero-Shot Task Generalization
Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020).
What will it take to generate fairness-preserving explanations?
In situations where explanations of black-box models may be useful, the fairness of the black-box is also often a relevant concern.
Learning from Multiple Noisy Partial Labelers
We evaluate our framework on three text classification and six object classification tasks.
Extended Few-Shot Learning: Exploiting Existing Resources for Novel Tasks
In many practical few-shot learning problems, even though labeled examples are scarce, there are abundant auxiliary datasets that potentially contain useful information.
Zero-Shot Learning with Common Sense Knowledge Graphs
Zero-shot learning relies on semantic class representations such as hand-engineered attributes or learned embeddings to predict classes without any labeled examples.
Ranked #1 on
Generalized Zero-Shot Learning
on OntoNotes
Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale
Labeling training data is one of the most costly bottlenecks in developing machine learning-based applications.
Snorkel: Rapid Training Data Creation with Weak Supervision
In a user study, subject matter experts build models 2. 8x faster and increase predictive performance an average 45. 5% versus seven hours of hand labeling.
Learning the Structure of Generative Models without Labeled Data
Curating labeled training data has become the primary bottleneck in machine learning.
Hinge-Loss Markov Random Fields and Probabilistic Soft Logic
In this paper, we introduce two new formalisms for modeling structured data, and show that they can both capture rich structure and scale to big data.
