Search Results for author:
Found 34 papers, 10 papers with code
Annotation of Communicative Functions of Short Feedback Tokens in Switchboard
There has been a lot of work on predicting the timing of feedback in conversational systems.
How “open” are the conversations with open-domain chatbots? A proposal for Speech Event based evaluation
To clarify the boundaries of “openness”, we conduct two studies: First, we classify the types of “speech events” encountered in a chatbot evaluation data set (i. e., Meena by Google) and find that these conversations mainly cover the “small talk” category and exclude the other speech event categories encountered in real life human-human communication.
Projection of Turn Completion in Incremental Spoken Dialogue Systems
The ability to take turns in a fluent way (i. e., without long response delays or frequent interruptions) is a fundamental aspect of any spoken dialog system.
Multilingual Turn-taking Prediction Using Voice Activity Projection
The results show that a monolingual VAP model trained on one language does not make good predictions when applied to other languages.
An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems
To address this issue, we propose a framework for indirectly but objectively evaluating systems based on users' behaviors.
Real-time and Continuous Turn-taking Prediction Using Voice Activity Projection
A demonstration of a real-time and continuous turn-taking prediction system is presented.
Resolving References in Visually-Grounded Dialogue via Text Generation
Vision-language models (VLMs) have shown to be effective at image retrieval based on simple text queries, but text-image retrieval based on conversational input remains a challenge.
Collecting Visually-Grounded Dialogue with A Game Of Sorts
We address these concerns by introducing a collaborative image ranking task, a grounded agreement game we call "A Game Of Sorts".
Towards Objective Evaluation of Socially-Situated Conversational Robots: Assessing Human-Likeness through Multimodal User Behaviors
This paper tackles the challenging task of evaluating socially situated conversational robots and presents a novel objective evaluation approach that relies on multimodal user behaviors.
Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs
In any system that uses structured knowledge graph (KG) data as its underlying knowledge representation, KG-to-text generation is a useful tool for turning parts of the graph data into text that can be understood by humans.
Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis
Turn-taking is a fundamental aspect of human communication where speakers convey their intention to either hold, or yield, their turn through prosodic cues.
Response-conditioned Turn-taking Prediction
Treating the turn-prediction and response-ranking as a one-stage process, our findings suggest that our model can be used as an incremental response ranker, which can be applied in various settings.
What makes a good pause? Investigating the turn-holding effects of fillers
Filled pauses (or fillers), such as "uh" and "um", are frequent in spontaneous speech and can serve as a turn-holding cue for the listener, indicating that the current speaker is not done yet.
The Open-domain Paradox for Chatbots: Common Ground as the Basis for Human-like Dialogue
There is a surge in interest in the development of open-domain chatbots, driven by the recent advancements of large language models.
How "open" are the conversations with open-domain chatbots? A proposal for Speech Event based evaluation
To clarify the boundaries of "openness", we conduct two studies: First, we classify the types of "speech events" encountered in a chatbot evaluation data set (i. e., Meena by Google) and find that these conversations mainly cover the "small talk" category and exclude the other speech event categories encountered in real life human-human communication.
How Much Does Prosody Help Turn-taking? Investigations using Voice Activity Projection Models
Turn-taking is a fundamental aspect of human communication and can be described as the ability to take turns, project upcoming turn shifts, and supply backchannels at appropriate locations throughout a conversation.
Voice Activity Projection: Self-supervised Learning of Turn-taking Events
The modeling of turn-taking in dialog can be viewed as the modeling of the dynamics of voice activity of the interlocutors.
CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings
This paper presents CoLLIE: a simple, yet effective model for continual learning of how language is grounded in vision.
TurnGPT: a Transformer-based Language Model for Predicting Turn-taking in Spoken Dialog
Syntactic and pragmatic completeness is known to be important for turn-taking prediction, but so far machine learning models of turn-taking have used such linguistic information in a limited way.
Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees
In dialogue, speakers continuously adapt their speech to accommodate the listener, based on the feedback they receive.
Using Lexical Alignment and Referring Ability to Address Data Sparsity in Situated Dialog Reference Resolution
Referring to entities in situated dialog is a collaborative process, whereby interlocutors often expand, repair and/or replace referring expressions in an iterative process, converging on conceptual pacts of referring language use in doing so.
Multimodal Continuous Turn-Taking Prediction Using Multiscale RNNs
To design spoken dialog systems that can conduct fluid interactions it is desirable to incorporate cues from separate modalities into turn-taking models.
Investigating Speech Features for Continuous Turn-Taking Prediction Using LSTMs
The continuous predictions represent generalized turn-taking behaviors observed in the training data and can be applied to make decisions that are not just limited to end-of-turn detection.
Towards a General, Continuous Model of Turn-taking in Spoken Dialogue using LSTM Recurrent Neural Networks
Previous models of turn-taking have mostly been trained for specific turn-taking decisions, such as discriminating between turn shifts and turn retention in pauses.
