no code implementations • 7 May 2024 • Tianyi Zhang, Jonah Yi, Zhaozhuo Xu, Anshumali Shrivastava
We observe that distinct channels of a key/value activation embedding are highly inter-dependent, and the joint entropy of multiple channels grows at a slower rate than the sum of their marginal entropies.
1 code implementation • 29 Aug 2023 • Gaurav Gupta, Jonah Yi, Benjamin Coleman, Chen Luo, Vihan Lakshman, Anshumali Shrivastava
With the surging popularity of approximate near-neighbor search (ANNS), driven by advances in neural representation learning, the ability to serve queries accompanied by a set of constraints has become an area of intense interest.