Search Results for author:
Found 21 papers, 12 papers with code
Encoding of lexical tone in self-supervised models of spoken language
Interpretability research has shown that self-supervised Spoken Language Models (SLMs) encode a wide variety of features in human speech from the acoustic, phonetic, phonological, syntactic and semantic levels, to speaker characteristics.
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Transformers have become a key architecture in speech processing, but our understanding of how they build up representations of acoustic and linguistic structure is limited.
Wave to Syntax: Probing spoken language models for syntax
Understanding which information is encoded in deep models of spoken and written language has been the focus of much research in recent years, as it is crucial for debugging and improving these architectures.
Quantifying Context Mixing in Transformers
Self-attention weights and their transformed variants have been the main source of information for analyzing token-to-token interactions in Transformer-based models.
Learning English with Peppa Pig
In the real world the coupling between the linguistic and the visual modality is loose, and often confounded by correlations with non-semantic aspects of the speech signal.
Discrete representations in neural models of spoken language
The distributed and continuous representations used by neural networks are at odds with representations employed in linguistics, which are typically symbolic.
Learning to Understand Child-directed and Adult-directed Speech
Speech directed to children differs from adult-directed speech in linguistic aspects such as repetition, word choice, and sentence length, as well as in aspects of the speech signal itself, such as prosodic and phonemic variation.
Analyzing analytical methods: The case of phonology in neural models of spoken language
Given the fast development of analysis techniques for NLP and speech processing systems, few systematic studies have been conducted to compare the strengths and weaknesses of each method.
Bootstrapping Disjoint Datasets for Multilingual Multimodal Representation Learning
However, we do find that using an external machine translation model to generate the synthetic data sets results in better performance.
Correlating neural and symbolic representations of language
Analysis methods which enable us to better understand the representations and functioning of neural models of language are increasingly needed as deep learning becomes the dominant approach in NLP.
Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language.
Lessons learned in multilingual grounded language learning
Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language.
Revisiting the Hierarchical Multiscale LSTM
Hierarchical Multiscale LSTM (Chung et al., 2016a) is a state-of-the-art language model that learns interpretable structure from character-level input.
On the difficulty of a distributional semantics of spoken language
In the domain of unsupervised learning most work on speech has focused on discovering low-level constructs such as phoneme inventories or word-like units.
Encoding of phonology in a recurrent neural model of grounded speech
We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech.
Representations of language in a model of visually grounded speech signal
We present a visually grounded model of speech perception which projects spoken utterances and images to a joint semantic space.
Representation of linguistic form and function in recurrent neural networks
We present novel methods for analyzing the activation patterns of RNNs from a linguistic point of view and explore the types of linguistic structure they learn.
Learning language through pictures
We propose Imaginet, a model of learning visually grounded representations of language from coupled textual and visual input.
