Search Results for author:
Found 3 papers, 2 papers with code
Do Llamas Work in English? On the Latent Language of Multilingual Transformers
Tracking intermediate embeddings through their high-dimensional space reveals three distinct phases, whereby intermediate embeddings (1) start far away from output token embeddings; (2) already allow for decoding a semantically correct next token in the middle layers, but give higher probability to its version in English than in the input language; (3) finally move into an input-language-specific region of the embedding space.
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia
Yet the mechanisms underlying this contextual grounding remain unknown, especially in situations where contextual information contradicts factual knowledge stored in the parameters, which LLMs also excel at recalling.
PaSS: Parallel Speculative Sampling
As an alternative, we propose to use parallel decoding as a way to draft multiple tokens from a single model with no computational cost, nor the need for a second model.
