Browse SoTA > Natural Language Processing > Language Modelling

Language Modelling

656 papers with code ยท Natural Language Processing

Language modeling is the task of predicting the next word or character in a document.

* indicates models using dynamic evaluation; where, at test time, models may adapt to seen tokens in order to improve performance on following tokens. (Mikolov et al., (2010), Kraus et al., (2017))

( Image credit: Exploring the Limits of Language Modeling )

Benchmarks

Latest papers without code

FastLR: Non-Autoregressive Lipreading Model with Integrate-and-Fire

6 Aug 2020

NAR lipreading is a challenging task that has many difficulties: 1) the discrepancy of sequence lengths between source and target makes it difficult to estimate the length of the output sequence; 2) the conditionally independent behavior of NAR generation lacks the correlation across time which leads to a poor approximation of target distribution; 3) the feature representation ability of encoder can be weak due to lack of effective alignment mechanism; and 4) the removal of AR language model exacerbates the inherent ambiguity problem of lipreading.

LANGUAGE MODELLING LIPREADING

6VecLM: Language Modeling in Vector Space for IPv6 Target Generation

5 Aug 2020

Fast IPv6 scanning is challenging in the field of network measurement as it requires exploring the whole IPv6 address space but limited by current computational power.

LANGUAGE MODELLING

Efficient MDI Adaptation for n-gram Language Models

5 Aug 2020

This paper presents an efficient algorithm for n-gram language model adaptation under the minimum discrimination information (MDI) principle, where an out-of-domain language model is adapted to satisfy the constraints of marginal probabilities of the in-domain data.

LANGUAGE MODELLING

Learning Visual Representations with Caption Annotations

4 Aug 2020

Starting from the observation that captioned images are easily crawlable, we argue that this overlooked source of information can be exploited to supervise the training of visual representations.

IMAGE CAPTIONING LANGUAGE MODELLING

AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting

3 Aug 2020

Unlike previous works that merely employed visual features for text detection, this work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter), which learns both visual and linguistic features to significantly reduce ambiguity in text detection.

LANGUAGE MODELLING SCENE TEXT TEXT SPOTTING

On Learning Universal Representations Across Languages

31 Jul 2020

Recent studies have demonstrated the overwhelming advantage of cross-lingual pre-trained models (PTMs), such as multilingual BERT and XLM, on cross-lingual NLP tasks.

CONTRASTIVE LEARNING CROSS-LINGUAL NATURAL LANGUAGE INFERENCE LANGUAGE MODELLING MACHINE TRANSLATION

Improving NER's Performance with Massive financial corpus

31 Jul 2020

Training large deep neural networks needs massive high quality annotation data, but the time and labor costs are too expensive for small business.

LANGUAGE MODELLING

Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing

31 Jul 2020

In this paper, we challenge this assumption by showing that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.

LANGUAGE MODELLING NAMED ENTITY RECOGNITION

TweepFake: about Detecting Deepfake Tweets

31 Jul 2020

To help the research in this field, we collected a dataset of real Deepfake tweets.

FACE SWAPPING LANGUAGE MODELLING TEXT GENERATION

A Study on Effects of Implicit and Explicit Language Model Information for DBLSTM-CTC Based Handwriting Recognition

31 Jul 2020

Deep Bidirectional Long Short-Term Memory (D-BLSTM) with a Connectionist Temporal Classification (CTC) output layer has been established as one of the state-of-the-art solutions for handwriting recognition.

HANDWRITING RECOGNITION LANGUAGE MODELLING