Computing Graph Neural Networks: A Survey from Algorithms to Accelerators
Вычисления для графовых нейронных сетей: обзор от алгоритмов до ускорителей
2021-10-08
SCID: 54.1/jm9px6wj
Discuss with AI
GNN acceleratorsGraph neural networksGraph-aware computingHardware-software co-designSparse-dense operations
Figures from the paper
Abstract (AI)
Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. Such an ability has strong implications in a wide variety of fields whose data are inherently relational, for which conventional neural networks do not perform well. Indeed, as recent reviews can attest, research in the area of GNNs has grown rapidly and has lead to the development of a variety of GNN algorithm variants as well as to the exploration of ground-breaking applications in chemistry, neurology, electronics, or communication networks, among others. At the current stage research, however, the efficient processing of GNNs is still an open challenge for several reasons. Besides of their novelty, GNNs are hard to compute due to their dependence on the input graph, their combination of dense and very sparse operations, or the need to scale to huge graphs in some applications. In this context, this article aims to make two main contributions. On the one hand, a review of the field of GNNs is presented from the perspective of computing. This includes a brief tutorial on the GNN fundamentals, an overview of the evolution of the field in the last decade, and a summary of operations carried out in the multiple phases of different GNN algorithm variants. On the other hand, an in-depth analysis of current software and hardware acceleration schemes is provided, from which a hardware-software, graph-aware, and communication-centric vision for GNN accelerators is distilled.
Key Findings
1
Efficient GNN processing remains challenging because computation depends on input graph structure, combines dense and highly sparse operations, and may require scaling to huge graphs.
2
Graph Neural Networks are increasingly important for learning from relational data across applications including chemistry, neurology, electronics, and communication networks.
3
The paper provides an in-depth analysis of existing GNN software and hardware acceleration approaches.
4
The survey derives a hardware-software, graph-aware, and communication-centric perspective for designing GNN accelerators.
5
The survey presents a computing-oriented tutorial and review covering GNN fundamentals, field evolution over the last decade, and operations across multiple algorithm variants.
Research Object
Graph Neural Networks (GNNs) and their computation on graph-structured data
Research Subject
Efficient processing and hardware–software acceleration of GNNs, including algorithmic operations, scalability, and graph-aware communication
Publication Details
Publication Date
2021-10-08
Journal
Publisher
ISSN
Cited by
310
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai15
Deep Residual Learning for Image Recognition2016
Semi-Supervised Classification with Graph Convolutional Networks2016
A Comprehensive Survey on Graph Neural Networks2020
Non-local Neural Networks2018
The Graph Neural Network Model2008
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
Graph neural networks: A review of methods and applications2020
Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting2018
Neural Message Passing for Quantum Chemistry2017
Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting2019
Graph Convolutional Neural Networks for Web-Scale Recommender Systems2018
A Review of Relational Machine Learning for Knowledge Graphs2015
Deep Learning on Graphs: A Survey2020
EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs2020
Detecting Functionality-Specific Vulnerabilities via Retrieving Individual Functionality-Equivalent APIs in Open-Source Repositories2025