Heterogeneous Graph Transformer
Трансформер для неоднородных графов
2020-04-20
SCID: 54.1/yg77hrhw
Discuss with AI
HGSamplingHeterogeneous Graph TransformerOpen Academic Graphheterogeneous attentionheterogeneous graphs
Figures from the paper
Abstract (AI)
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges belong to the same types, making it infeasible to represent heterogeneous structures. In this paper, we present the Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To model heterogeneity, we design node- and edge-type dependent parameters to characterize the heterogeneous attention over each edge, empowering HGT to maintain dedicated representations for different types of nodes and edges. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm—HGSampling—for efficient and scalable training. Extensive experiments on the Open Academic Graph of 179 million nodes and 2 billion edges show that the proposed HGT model consistently outperforms all the state-of-the-art GNN baselines by 9–21 on various downstream tasks. The dataset and source code of HGT are publicly available at https://github.com/acbull/pyHGT.
Key Findings
1
HGT maintains dedicated representations for different node and edge types, enabling more effective modeling of heterogeneous relational structures.
2
On the 179-million-node, 2-billion-edge Open Academic Graph, HGT consistently outperforms state-of-the-art GNN baselines by 9–21 across downstream tasks.
3
The HGSampling algorithm provides efficient, scalable heterogeneous mini-batch training for very large graph data.
4
The HGT dataset and source code are publicly released, supporting reproducibility and further research.
5
The Heterogeneous Graph Transformer (HGT) models Web-scale heterogeneous graphs using node- and edge-type-dependent parameters in its attention mechanism.
Research Object
Web-scale heterogeneous graphs (e.g., Open Academic Graph with 179 million nodes and 2 billion edges)
Research Subject
Type-specific graph representation learning and scalable modeling performance on downstream tasks
Publication Details
Publication Date
2020-04-20
Journal
Publisher
ISSN
Cited by
2443
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai7
Deep Residual Learning for Image Recognition2016
Semi-Supervised Classification with Graph Convolutional Networks2016
Transformers: State-of-the-Art Natural Language Processing2020
Neural Message Passing for Quantum Chemistry2017
Heterogeneous Graph Attention Network2019
HuggingFace's Transformers: State-of-the-art Natural Language Processing2019
Self-Attention with Relative Position Representations2018
Cited by7
Graph neural networks: A review of methods and applications2020
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research2022
Graph Neural Networks for Natural Language Processing: A Survey2023
A Survey on Graph Representation Learning Methods2023
Federated Social Recommendation with Graph Neural Network2022
Temporal Heterogeneous Graph Contrastive Learning for Fraud Detection in Credit Card Transactions2025
Brain Network Transformer2022