GraphMix: Improved Training of GNNs for Semi-Supervised Learning

25 Sep 2019  ·  Vikas Verma, Meng Qu, Kenji Kawaguchi, Alex Lamb, Yoshua Bengio, Juho Kannala, Jian Tang ·

We present GraphMix, a regularization method for Graph Neural Network based semi-supervised object classification, whereby we propose to train a fully-connected network jointly with the graph neural network via parameter sharing and interpolation-based regularization. Further, we provide a theoretical analysis of how GraphMix improves the generalization bounds of the underlying graph neural network, without making any assumptions about the "aggregation" layer or the depth of the graph neural networks. We experimentally validate this analysis by applying GraphMix to various architectures such as Graph Convolutional Networks, Graph Attention Networks and Graph-U-Net. Despite its simplicity, we demonstrate that GraphMix can consistently improve or closely match state-of-the-art performance using even simpler architectures such as Graph Convolutional Networks, across three established graph benchmarks: Cora, Citeseer and Pubmed citation network datasets, as well as three newly proposed datasets: Cora-Full, Co-author-CS and Co-author-Physics.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Node Classification Bitcoin-Alpha GraphMix (GCN) F1-score 0.6534 # 1
Node Classification Bitcoin-OTC GraphMix (GCN) F1-score 0.6635 # 1
Node Classification Citeseer random partition GraphMix (GCN) Accuracy 76.45 ± 1.57 # 1
Node Classification CiteSeer with Public Split: fixed 20 nodes per class GraphMix(GCN) Accuracy 74.52 ± 0.59 # 6
Node Classification CiteSeer with Public Split: fixed 5 nodes per class GraphMix (GCN) Accuracy 58.55 ± 2.26 # 1
Node Classification Coauthor CS GraphMix (GCN) Accuracy 91.83 ± 0.51 # 15
Node Classification Coauthor Physics GraphMix (GCN) Accuracy 94.49 ± 0.84 # 7
Node Classification Cora: fixed 10 node per class GraphMix (GCN) Accuracy 79.3 # 2
Node Classification Cora: fixed 5 node per class GraphMix (GCN) Accuracy 71.99 ± 6.46 # 2
Node Classification Cora Full-supervised GraphMix (GCN) Accuracy 61.8% # 8
Node Classification Cora random partition GraphMix (GCN) Accuracy 82.07 ± 1.17 # 1
Node Classification Cora with Public Split: fixed 20 nodes per class GraphMix Accuracy 83.94 ± 0.57 # 12
Node Classification Cora with Public Split: fixed 20 nodes per class GraphMix (GCN) Accuracy 83.94 ± 0.57 # 12
Node Classification Pubmed random partition GraphMix (GCN) Accuracy 80.72 ± 1.08 # 1
Node Classification PubMed with Public Split: fixed 20 nodes per class GraphMix (GCN) Accuracy 80.98 ± 0.55 # 8
Node Classification PubMed with Public Split: fixed 20 nodes per class GCN(predicted-targets) Accuracy 80.42% # 12

Methods