Chuan Li, PhD reviews GPT-3, the new NLP model from OpenAI. This paper empirically shows that language model performance scales as a power-law with model size, datataset size, and the amount of computation.
The Lambda Deep Learning Blog
Voltron Data Case Study: Why ML teams are using Lambda Reserved Cloud Clusters
November 01, 2022
How to fine tune stable diffusion: how we made the text-to-pokemon model at Lambda
September 28, 2022
Update June 5th 2020: OpenAI has announced a successor to GPT-2 in a newly published paper. Checkout our GPT-3 model overview [https://lambdalabs.com/blog/demystifying-gpt-3/]. OpenAI recently published a blog post on their GPT-2 language model [https://blog.openai.com/better-language-models/]. This tutorial shows you how to run the
BERT is Google's SOTA pre-training language representations. This blog is about running BERT with multiple GPUs. Specifically, we will use the Horovod framework to parrallelize the tasks. We ill list all the changes to the original BERT implementation and highlight a few places that will make or break the performance.