Search Results for author: Ian Arawjo

Found 4 papers, 1 papers with code

Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences

no code implementations18 Apr 2024 Shreya Shankar, J. D. Zamfirescu-Pereira, Björn Hartmann, Aditya G. Parameswaran, Ian Arawjo

In particular, we identify a phenomenon we dub \emph{criteria drift}: users need criteria to grade outputs, but grading outputs helps users define criteria.

Antagonistic AI

no code implementations12 Feb 2024 Alice Cai, Ian Arawjo, Elena L. Glassman

The vast majority of discourse around AI development assumes that subservient, "moral" models aligned with "human values" are universally beneficial -- in short, that good AI is sycophantic AI.

ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing

1 code implementation17 Sep 2023 Ian Arawjo, Chelse Swoopes, Priyan Vaithilingam, Martin Wattenberg, Elena Glassman

Evaluating outputs of large language models (LLMs) is challenging, requiring making -- and making sense of -- many responses.

Model Selection Prompt Engineering +1

Cannot find the paper you are looking for? You can Submit a new open access paper.