no code implementations • 14 Feb 2024 • Xinyun Chen, Ryan A. Chi, Xuezhi Wang, Denny Zhou
We first examine the effect of premise ordering on deductive reasoning on a variety of LLMs, and our evaluation shows that permuting the premise order can cause a performance drop of over 30%.