Search Results for author:
Found 4 papers, 1 papers with code
Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains
Inspired by the Markovianity of natural languages, we model the data as a Markovian source and utilize this framework to systematically study the interplay between the data-distributional properties, the transformer architecture, the learnt distribution, and the final model performance.
Batch Universal Prediction
Large language models (LLMs) have recently gained much popularity due to their surprising ability at generating human-like English sentences.
LASER: Linear Compression in Wireless Distributed Optimization
On the latter, we obtain $50$-$64 \%$ improvement in perplexity over our baselines for noisy channels.
Mismatched decoding reliability function at zero rate
We derive an upper bound on the reliability function of mismatched decoding for zero-rate codes.
Information Theory Information Theory
