Search Results for author: Brielen Madureira

Found 13 papers, 10 papers with code

Flamingos and Hedgehogs in the Croquet-Ground: Teaching Evaluation of NLP Systems for Undergraduate Students

no code implementations NAACL (TeachingNLP) 2021 Brielen Madureira

This report describes the course Evaluation of NLP Systems, taught for Computational Linguistics undergraduate students during the winter semester 20/21 at the University of Potsdam, Germany.

When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality

1 code implementation20 Feb 2024 Brielen Madureira, Patrick Kahardipraja, David Schlangen

Incremental models that process sentences one token at a time will sometimes encounter points where more than one interpretation is possible.

Dependency Parsing

Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests

1 code implementation30 Jan 2024 Brielen Madureira, David Schlangen

Clarification requests are a mechanism to help solve communication problems, e. g. due to ambiguity or underspecification, in instruction-following interactions.

Instruction Following

"Are you telling me to put glasses on the dog?'' Content-Grounded Annotation of Instruction Clarification Requests in the CoDraw Dataset

no code implementations4 Jun 2023 Brielen Madureira, David Schlangen

Instruction Clarification Requests are a mechanism to solve communication problems, which is very functional in instruction-following interactions.

Instruction Following

Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents

1 code implementation22 May 2023 Kranti Chalamalasetti, Jana Götze, Sherzod Hakimov, Brielen Madureira, Philipp Sadler, David Schlangen

Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents"-agents that operate in rich linguistic and non-linguistic contexts-through testing them in carefully constructed interactive settings.

Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset

1 code implementation28 Feb 2023 Brielen Madureira, David Schlangen

In visual instruction-following dialogue games, players can engage in repair mechanisms in face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in the world.

visual instruction following

Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge

1 code implementation ACL 2022 Brielen Madureira, David Schlangen

Our conclusion is that the ability to make the distinction between shared and privately known statements along the dialogue is moderately present in the analysed models, but not always incrementally consistent, which may partially be due to the limited need for grounding interactions in the original task.

Cannot find the paper you are looking for? You can Submit a new open access paper.