PyTorch: An Imperative Style, High-Performance Deep Learning Library
PyTorch: библиотека глубокого обучения с императивным стилем программирования и высокой производительностью
2019-12-03
SCID: 54.1/wdwew8er
Discuss with AI
GPU accelerationPyTorchPythonic programmingdeep learning libraryimperative programming
Figures from the paper
Abstract (AI)
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.
Key Findings
1
Every aspect of PyTorch operates as a regular Python program under user control, enabling flexible and transparent development.
2
Its imperative, Pythonic programming style treats code as a model, simplifies debugging, and aligns with scientific computing libraries.
3
PyTorch demonstrates that usability and high performance can coexist within a deep learning framework.
4
PyTorch supports hardware accelerators such as GPUs while maintaining efficient runtime execution through carefully integrated components.
5
Subsystem evaluations and common benchmarks demonstrate compelling overall speed and efficiency.
Research Object
PyTorch deep learning library
Research Subject
the library’s imperative Pythonic programming model, runtime architecture, usability, and computational performance, including GPU-accelerated benchmark efficiency
Publication Details
Publication Date
2019-12-03
Journal
Publisher
ISSN
Cited by
31471
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
Cited by20
Array programming with NumPy2020
A Survey of Large Language Models2026
Advances and Open Problems in Federated Learning2020
A Comprehensive Overview of Large Language Models2025
Point Transformer2021
Albumentations: Fast and Flexible Image Augmentations2020
On the Opportunities and Risks of Foundation Models2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet2021
Time-series forecasting with deep learning: a survey2021
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery2024
Artificial intelligence: A powerful paradigm for scientific research2021
SplitFed: When Federated Learning Meets Split Learning2022
Dense Nested Attention Network for Infrared Small Target Detection2022
A Unifying Review of Deep and Shallow Anomaly Detection2021
Decoupled Knowledge Distillation2022
CMT: Convolutional Neural Networks Meet Vision Transformers2022
Transformer in Transformer2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better2023
Self-Supervised Speech Representation Learning: A Review2022
Explainability in Graph Neural Networks: A Taxonomic Survey2022