Mitigating Degree Biases in Message Passing Mechanism by Utilizing Community Structures
This study utilizes community structures to address node degree biases in message-passing (MP) via learnable graph augmentations and novel graph transformers. Recent augmentation-based methods showed that MP neural networks often perform poorly on low-degree nodes, leading to degree biases due to a lack of messages reaching low-degree nodes. Despite their success, most methods use heuristic or uniform random augmentations, which are non-differentiable and may not always generate valuable edges for learning representations. In this paper, we propose Community-aware Graph Transformers, namely CGT, to learn degree-unbiased representations based on learnable augmentations and graph transformers by extracting within community structures. We first design a learnable graph augmentation to generate more within-community edges connecting low-degree nodes through edge perturbation. Second, we propose an improved self-attention to learn underlying proximity and the roles of nodes within the community. Third, we propose a self-supervised learning task that could learn the representations to preserve the global graph structure and regularize the graph augmentations. Extensive experiments on various benchmark datasets showed CGT outperforms state-of-the-art baselines and significantly improves the node degree biases. The source code is available at https://github.com/NSLab-CUK/Community-aware-Graph-Transformer.
PDF AbstractDatasets
Results from the Paper
Ranked #1 on Node Clustering on Pubmed (Conductance metric)
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Node Classification | Amazon Computers | CGT | Accuracy | 91.45±0.58 | # 2 | |
Node Clustering | Amazon Computers | CGT | Conductance | 10.13±1.30 | # 1 | |
Modularity | 88.07±1.32 | # 1 | ||||
Node Clustering | Amazon Photo | CGT | Conductance | 9.71±0.44 | # 1 | |
Modularity | 85.39±2.58 | # 1 | ||||
Node Classification | AMZ Photo | CGT | Accuracy | 95.73±0.84 | # 1 | |
Node Classification | Citeseer | CGT | Accuracy | 76.59±0.98 | # 13 | |
Node Clustering | Citeseer | CGT | Modularity | 68.19±0.39 | # 1 | |
Conductance | 5.40±1.21 | # 2 | ||||
Node Classification | Cora | CGT | Accuracy | 87.10±1.53 | # 18 | |
Node Clustering | Cora | CGT | Modularity | 69.28±0.32 | # 2 | |
Conductance | 9.84±0.76 | # 2 | ||||
Node Classification | Pubmed | CGT | Accuracy | 86.86±0.12 | # 18 | |
Node Clustering | Pubmed | CGT | Conductance | 7.66±1.52 | # 1 | |
Modularity | 89.51±4.19 | # 1 | ||||
Node Clustering | Wiki-CS | CGT | Conductance | 21.68±0.82 | # 1 | |
Modularity | 76.71±1.31 | # 1 | ||||
Node Classification | Wiki-CS | CGT | Accuracy | 84.61±0.53 | # 1 |