Search Results for author:
Found 39 papers, 8 papers with code
Criteria for Useful Automatic Romanization in South Asian Languages
This paper presents a number of possible criteria for systems that transliterate South Asian languages from their native scripts into the Latin script, a process known as romanization.
Extensions to Brahmic script processing within the Nisaba library: new scripts, languages and utilities
The Brahmic family of scripts is used to record some of the most spoken languages in the world and is arguably the most diverse family of writing systems.
Design principles of an open-source language modeling microservice package for AAC text-entry applications
We present MozoLM, an open-source language model microservice package intended for use in AAC text-entry applications, with a particular focus on the design principles of the library.
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages
We evaluate commonly used models on the benchmark.
Spelling convention sensitivity in neural language models
Using a set of probe words unique to either British or American English, we first establish that training corpora exhibit substantial (though not total) consistency.
Beyond Arabic: Software for Perso-Arabic Script Manipulation
This paper presents an open-source software library that provides a set of finite-state transducer (FST) components and corresponding utilities for manipulating the writing systems of languages that use the Perso-Arabic script.
Graphemic Normalization of the Perso-Arabic Script
Since its original appearance in 1991, the Perso-Arabic script representation in Unicode has grown from 169 to over 440 atomic isolated characters spread over several code pages representing standard letters, various diacritics and punctuation for the original Arabic and numerous other regional orthographic traditions.
Structured abbreviation expansion in context
Ad hoc abbreviations are commonly found in informal communication channels that favor shorter messages.
Finding Concept-specific Biases in Form--Meaning Associations
It is not a new idea that there are small, cross-linguistic associations between the forms and meanings of words.
Finite-state script normalization and processing utilities: The Nisaba Brahmic library
This paper presents an open-source library for efficient low-level processing of ten major South Asian Brahmic scripts.
Disambiguatory Signals are Stronger in Word-initial Positions
Psycholinguistic studies of human word processing and lexical access provide ample evidence of the preferred nature of word-initial versus word-final segments, e. g., in terms of attention paid by listeners (greater) or the likelihood of reduction by speakers (lower).
Processing South Asian Languages Written in the Latin Script: the Dakshina Dataset
This paper describes the Dakshina dataset, a new resource consisting of text in both the Latin and native scripts for 12 South Asian languages.
Phonotactic Complexity and its Trade-offs
We present methods for calculating a measure of phonotactic complexity---bits per phoneme---that permits a straightforward cross-linguistic comparison.
Language-agnostic Multilingual Modeling
Multilingual Automated Speech Recognition (ASR) systems allow for the joint training of data-rich and data-scarce languages in a single model.
Distilling weighted finite automata from arbitrary probabilistic models
Weighted finite automata (WFA) are often used to represent probabilistic models, such as n-gram language models, since they are efficient for recognition tasks in time and space.
Latin script keyboards for South Asian languages with finite-state normalization
The use of the Latin script for text entry of South Asian languages is common, even though there is no standard orthography for these languages in the script.
Rethinking Phonotactic Complexity
In this work, we propose the use of phone-level language models to estimate phonotactic complexity{---}measured in bits per phoneme{---}which makes cross-linguistic comparison straightforward.
Meaning to Form: Measuring Systematicity as Information
A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade?
What Kind of Language Is Hard to Language-Model?
Trying to answer the question of what features difficult languages have in common, we try and fail to reproduce our earlier (Cotterell et al., 2018) observation about morphological complexity and instead reveal far simpler statistics of the data that seem to drive complexity in a much larger sample.
Neural Models of Text Normalization for Speech Applications
One problem that has been somewhat resistant to effective machine learning solutions is text normalization for speech applications such as text-to-speech synthesis (TTS).
Approximating probabilistic models as weighted finite automata
Weighted finite automata (WFA) are often used to represent probabilistic models, such as $n$-gram language models, since they are efficient for recognition tasks in time and space.
Are All Languages Equally Hard to Language-Model?
For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on languages with differing typological profiles?
