Search Results for author:
Found 10 papers, 4 papers with code
PATCH -- Psychometrics-AssisTed benCHmarking of Large Language Models: A Case Study of Mathematics Proficiency
Third, we release 4 datasets to support measuring and comparing LLM proficiency in grade school mathematics and science against human populations.
USE: Dynamic User Modeling with Stateful Sequence Models
Furthermore, we introduce a novel training objective named future W-behavior prediction to transcend the limitations of next-token prediction by forecasting a broader horizon of upcoming user behaviors.
Designing and Evaluating General-Purpose User Representations Based on Behavioral Logs from a Measurement Process Perspective: A Case Study with Snapchat
Using this new framework, we design a Transformer-based user model that can produce high-quality general-purpose user representations for instant messaging platforms like Snapchat.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Epicurus at SemEval-2023 Task 4: Improving Prediction of Human Values behind Arguments by Leveraging Their Definitions
We describe our experiments for SemEval-2023 Task 4 on the identification of human values behind arguments (ValueEval).
Modelling Stance Detection as Textual Entailment Recognition and Leveraging Measurement Knowledge from Social Sciences
Stance detection (SD) can be considered a special case of textual entailment recognition (TER), a generic natural language task.
Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions
Our results thus highlight the necessity to examine the construct validity of text embeddings before deploying them in social science research.
Assessing the Reliability of Word Embedding Gender Bias Measures
In this paper, we assess three types of reliability of word embedding gender bias measures, namely test-retest reliability, inter-rater consistency and internal consistency.
Open Source Software for Efficient and Transparent Reviews
To help researchers conduct a systematic review or meta-analysis as efficiently and transparently as possible, we designed a tool (ASReview) to accelerate the step of screening titles and abstracts.
