Search Results for author:
Found 9 papers, 4 papers with code
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
Public LLMs such as the Llama 2-Chat have driven huge activity in LLM research.
Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models
The paper develops a methodology for (a) designing and administering such an evaluation, and (b) automatic grading (plus spot-checking by humans) of the results using GPT-4 as well as the open LLaMA-2 70B model.
Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
By classifying infinite-width neural networks and identifying the *optimal* limit, Tensor Programs IV and V demonstrated a universal way, called $\mu$P, for *widthwise hyperparameter transfer*, i. e., predicting optimal hyperparameters of wide neural networks from narrow ones.
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound
Saliency methods compute heat maps that highlight portions of an input that were most {\em important} for the label assigned to it by a deep net.
A Kernel-Based View of Language Model Fine-Tuning
It has become standard to solve NLP tasks by fine-tuning pre-trained language models (LMs), especially in low-data settings.
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic and Sound
Saliency methods seek to provide human-interpretable explanations for the output of machine learning model on a given input.
Enhanced Convolutional Neural Tangent Kernels
An exact algorithm to compute CNTK (Arora et al., 2019) yielded the finding that classification accuracy of CNTK on CIFAR-10 is within 6-7% of that of that of the corresponding CNN architecture (best figure being around 78%) which is interesting performance for a fixed kernel.
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks
On VOC07 testbed for few-shot image classification tasks on ImageNet with transfer learning (Goyal et al., 2019), replacing the linear SVM currently used with a Convolutional NTK SVM consistently improves performance.
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee
Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data.
