DeepLearningExamples

NVIDIA Deep Learning Examples for Tensor Cores

Introduction

This repository provides State-of-the-Art Deep Learning examples that are easy to train and deploy, achieving the best reproducible accuracy and performance with NVIDIA CUDA-X software stack running on NVIDIA Volta, Turing and Ampere GPUs.

NVIDIA GPU Cloud (NGC) Container Registry

These examples, along with our NVIDIA deep learning software stack, are provided in a monthly updated Docker container on the NGC container registry (https://ngc.nvidia.com). These containers include:

• The latest NVIDIA examples from this repository
• The latest NVIDIA contributions shared upstream to the respective framework
• The latest NVIDIA Deep Learning software libraries, such as cuDNN, NCCL, cuBLAS, etc. which have all been through a rigorous monthly quality assurance process to ensure that they provide the best possible performance
• Monthly release notes for each of the NVIDIA optimized containers

Computer Vision

Models	Framework	A100	AMP	Multi-GPU	Multi-Node	TRT	ONNX	Triton	TF-TRT	NB
ResNet-50	PyTorch	Yes	Yes	Yes	-	Yes	-	Yes	-	-
ResNeXt-101	PyTorch	Yes	Yes	Yes	-	Yes	-	Yes	-	-
SE-ResNeXt-101	PyTorch	Yes	Yes	Yes	-	Yes	-	Yes	-	-
Mask R-CNN	PyTorch	Yes	Yes	Yes	-	-	-	-	-	Yes
SSD	PyTorch	Yes	Yes	Yes	-	-	-	-	-	Yes
ResNet-50	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
ResNeXt101	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
SE-ResNeXt-101	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
Mask R-CNN	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
SSD	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	Yes
U-Net Ind	TensorFlow	Yes	Yes	Yes	-	-	-	-	Yes	Yes
U-Net Med	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
U-Net 3D	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
V-Net Med	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
U-Net Med	TensorFlow2	Yes	Yes	Yes	-	-	-	-	-	-
Mask R-CNN	TensorFlow2	Yes	Yes	Yes	-	-	-	-	-	-
ResNet-50	MXNet	-	Yes	Yes	-	-	-	-	-	-

Natural Language Processing

Models	Framework	A100	AMP	Multi-GPU	Multi-Node	TRT	ONNX	Triton	TF-TRT	NB
BERT	PyTorch	Yes	Yes	Yes	Yes	-	-	Yes	-	-
TransformerXL	PyTorch	Yes	Yes	Yes	Yes	-	-	-	-	-
GNMT	PyTorch	Yes	Yes	Yes	-	-	-	-	-	-
Transformer	PyTorch	Yes	Yes	Yes	-	-	-	-	-	-
ELECTRA	TensorFlow2	Yes	Yes	Yes	Yes	-	-	-	-	-
BERT	TensorFlow	Yes	Yes	Yes	Yes	Yes	-	Yes	-	Yes
BioBert	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	Yes
TransformerXL	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
GNMT	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
Faster Transformer	Tensorflow	-	-	-	-	Yes	-	-	-	-

Recommender Systems

Models	Framework	A100	AMP	Multi-GPU	Multi-Node	TRT	ONNX	Triton	TF-TRT	NB
DLRM	PyTorch	Yes	Yes	Yes	-	-	Yes	Yes	-	Yes
NCF	PyTorch	Yes	Yes	Yes	-	-	-	-	-	-
Wide&Deep	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
NCF	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-
VAE-CF	TensorFlow	Yes	Yes	Yes	-	-	-	-	-	-

Speech to Text

Models	Framework	A100	AMP	Multi-GPU	Multi-Node	TRT	ONNX	Triton	TF-TRT	NB
Jasper	PyTorch	Yes	Yes	Yes	-	Yes	Yes	Yes	-	Yes
Hidden Markov Model	Kaldi	-	-	Yes	-	-	-	Yes	-	-

Text to Speech

| Models | Framework | A100 | AMP | Multi-GPU | Multi-Node | TRT | ONNX | Triton | TF-TRT | NB | | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |------------- |------------- |------------- |------------- |------------- | | FastPitch | PyTorch | Yes | Yes | Yes | - | - | - | - | - | - | | FastSpeech | PyTorch | - | Yes | Yes | - | Yes | - | - | - | - | | Tacotron 2 and WaveGlow | PyTorch | Yes | Yes | Yes | - | Yes | Yes | Yes | - | - |

NVIDIA support

In each of the network READMEs, we indicate the level of support that will be provided. The range is from ongoing updates and improvements to a point-in-time release for thought leadership.

Feedback / Contributions

We're posting these examples on GitHub to better support the community, facilitate feedback, as well as collect and implement contributions using GitHub Issues and pull requests. We welcome all contributions!

Known issues

In each of the network READMEs, we indicate any known issues and encourage the community to provide feedback.