This presentation is a high-level overview of the different types of training regimes that you'll encounter as you move from single GPU to multi GPU to multi node distributed training. It briefly describes where the computation happens, how the gradients are communicated, and how the models are updated and communicated.
Update June 5th 2020: OpenAI has announced a successor to GPT-2 in a newly published paper. Checkout our GPT-3 model overview [https://lambdalabs.com/blog/demystifying-gpt-3/]. OpenAI recently published a blog post on their GPT-2 language model [https://blog.openai.com/better-language-models/]. This tutorial shows you how to run the
A cost and speed comparison between the Lambda Hyperplane 8 V100 GPU Server and AWS p3 GPU instances. A very similar comparison to the DGX-1.
This blog tests how fast does ResNet9 (the fastest way to train a SOTA image classifier on Cifar10) run on Nvidia's Turing GPUs, including 2080 Ti and Titan RTX. We also include 1080 Ti as the baseline for comparison.
This tutorial demonstrates how to use a pre-trained model for transfer learning. The networks used in this tutorial include ResNet50, InceptionV4 and NasNet. The dataset is Stanford Dogs. Tensorflow implementation is provided.
This tutorial will walk you through the steps of building an image classification application with TensorFlow. We will also introduce you to a few building blocks for creating your own deep learning demos.
...