On Representation Knowledge Distillation for Graph Neural Networks

1 code implementation • 9 Nov 2021 • Chaitanya K. Joshi, Fayao Liu, Xu Xun, Jie Lin, Chuan-Sheng Foo

Past work on distillation for GNNs proposed the Local Structure Preserving loss (LSP), which matches local structural relationships defined over edges across the student and teacher's node embeddings.

Contrastive Learning Knowledge Distillation