2 code implementations • 12 Mar 2023 • Xiaojun Guo, Yifei Wang, Tianqi Du, Yisen Wang
Instead of characterizing oversmoothing from the view of complete collapse in which representations converge to a single point, we dive into a more general perspective of dimensional collapse in which representations lie in a narrow cone.
Ranked #8 on Node Property Prediction on ogbn-arxiv
1 code implementation • 8 Mar 2023 • Yifei Wang, Qi Zhang, Tianqi Du, Jiansheng Yang, Zhouchen Lin, Yisen Wang
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics.
no code implementations • 2 Mar 2023 • Xuyang Zhao, Tianqi Du, Yisen Wang, Jun Yao, Weiran Huang
Moreover, we show that contrastive learning fails to learn domain-invariant features, which limits its transferability.