Search Results for author: Philipp Sadler

Found 10 papers, 4 papers with code

clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents

no code implementations31 May 2024 Anne Beyer, Kranti Chalamalasetti, Sherzod Hakimov, Brielen Madureira, Philipp Sadler, David Schlangen

In this paper, we take one of the proposed frameworks for setting up such game-play environments, and further test its usefulness as an evaluation instrument, along a number of dimensions: We show that it can easily keep up with new developments while avoiding data contamination, we show that the tests implemented within it are not yet saturated (human performance is substantially higher than that of even the best models), and we show that it lends itself to investigating additional questions, such as the impact of the prompting language on performance.

Sharing the Cost of Success: A Game for Evaluating and Learning Collaborative Multi-Agent Instruction Giving and Following Policies

1 code implementation26 Mar 2024 Philipp Sadler, Sherzod Hakimov, David Schlangen

In collaborative goal-oriented settings, the participants are not only interested in achieving a successful outcome, but do also implicitly negotiate the effort they put into the interaction (by adapting to each other).

Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game

no code implementations7 Feb 2024 Philipp Sadler, Sherzod Hakimov, David Schlangen

Albrecht and Stone (2018) state that modeling of changing behaviors remains an open problem "due to the essentially unconstrained nature of what other agents may do".

Pento-DIARef: A Diagnostic Dataset for Learning the Incremental Algorithm for Referring Expression Generation from Examples

1 code implementation24 May 2023 Philipp Sadler, David Schlangen

NLP tasks are typically defined extensionally through datasets containing example instantiations (e. g., pairs of image i and text t), but motivated intensionally through capabilities invoked in verbal descriptions of the task (e. g., "t is a description of i, for which the content of i needs to be recognised and understood").

Referring Expression Referring expression generation +1

Clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents

1 code implementation22 May 2023 Kranti Chalamalasetti, Jana Götze, Sherzod Hakimov, Brielen Madureira, Philipp Sadler, David Schlangen

Recent work has proposed a methodology for the systematic evaluation of "Situated Language Understanding Agents"-agents that operate in rich linguistic and non-linguistic contexts-through testing them in carefully constructed interactive settings.

Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers

1 code implementation22 May 2023 Philipp Sadler, Sherzod Hakimov, David Schlangen

The ability to pick up on language signals in an ongoing interaction is crucial for future machine learning models to collaborate and interact with humans naturally.

Referring Expression

Spatial Attention as an Interface for Image Captioning Models

no code implementations29 Sep 2020 Philipp Sadler

The internal workings of modern deep learning models stay often unclear to an external observer, although spatial attention mechanisms are involved.

Image Captioning Question Answering +1

Can Neural Image Captioning be Controlled via Forced Attention?

no code implementations WS 2019 Philipp Sadler, Tatjana Scheffler, David Schlangen

Learned dynamic weighting of the conditioning signal (attention) has been shown to improve neural language generation in a variety of settings.

Image Captioning Text Generation

Detecting cities in aerial night-time images by learning structural invariants using single reference augmentation

no code implementations19 Oct 2018 Philipp Sadler

First a principal component analysis in conjunction with a Fourier transform is trained on a single reference augmentation training dataset using the city images.

Data Augmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.