Search Results for author:
Found 4 papers, 1 papers with code
High probability or low information? The probability–quality paradox in language generation
When generating natural language from neural probabilistic models, high probability does not always coincide with high quality.
On the probability-quality paradox in language generation
Specifically, we posit that human-like language should contain an amount of information (quantified as negative log-probability) that is close to the entropy of the distribution over natural strings.
On Decoding Strategies for Neural Text Generators
For example, the nature of the diversity-quality trade-off in language generation is very task-specific; the length bias often attributed to beam search is not constant across tasks.
Locally Typical Sampling
Automatic and human evaluations show that, in comparison to nucleus and top-k sampling, locally typical sampling offers competitive performance (in both abstractive summarization and story generation) in terms of quality while consistently reducing degenerate repetitions.
