Search Results for author:
Found 9 papers, 8 papers with code
Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models
In this paper, we deeply explore the mechanisms employed by Transformer-based language models in factual recall tasks.
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models
In reasoning tasks, even a minor error can cascade into inaccurate results, leading to suboptimal performance of large language models in such domains.
Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning
In this paper, by treating in-context learning (ICL) as a meta-optimization process, we explain why LLMs are sensitive to the order of ICL examples.
Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use
Specifically, the crucial information in the context will be potentially overlooked by model when it is positioned in the trough zone of the attention waveform, leading to decreased performance.
Ranked #2 on
Trajectory Planning
on ToolBench
Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse
Recent studies have highlighted a phenomenon in large language models (LLMs) known as "the reversal curse," in which the order of knowledge entities in the training data biases the models' comprehension.
DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations
In open-domain dialogue generation tasks, contexts and responses in most datasets are one-to-one mapped, violating an important many-to-many characteristic: a context leads to various responses, and a response answers multiple contexts.
GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework
Our proposed representation, coupled with the non-autoregressive generative model, empowers GETMusic to generate music with any arbitrary source-target track combinations.
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
These characteristics cannot be well handled by neural generation models that learn lyric-to-melody mapping in an end-to-end way, due to several issues: (1) lack of aligned lyric-melody training data to sufficiently learn lyric-melody feature alignment; (2) lack of controllability in generation to better and explicitly align the lyric-melody features.
Target-Side Data Augmentation for Sequence Generation
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens.
