Discovering Universal Geometry in Embeddings with ICA

1 code implementation • 22 May 2023 • Hiroaki Yamagiwa, Momose Oyama, Hidetoshi Shimodaira

This study utilizes Independent Component Analysis (ICA) to unveil a consistent semantic structure within embeddings of words or images.

Norm of Word Embedding Encodes Information Gain

no code implementations • 19 Dec 2022 • Momose Oyama, Sho Yokoi, Hidetoshi Shimodaira

Distributed representations of words encode lexical semantic information, but what type of information is encoded and how?

Informativeness Keyword Extraction +2

A Mechanism of Implicit Regularization in Deep Learning

no code implementations • 25 Sep 2019 • Masayoshi Kubo, Genki Sugiura, Kenta Shinzato, Momose Oyama

Despite a lot of theoretical efforts, very little is known about mechanisms of implicit regularization by which the low complexity contributes to generalization in deep learning.

Generalization Bounds