How to use FlashAttention-2 on Lambda Cloud, including H100 vs A100 benchmark results for training GPT-3-style models using the new model.
The Lambda Deep Learning Blog
Subscribe
Categories
- gpu-cloud (27)
- tutorials (24)
- benchmarks (22)
- announcements (21)
- NVIDIA H100 (14)
- lambda cloud (14)
- hardware (12)
- gpus (9)
- tensorflow (9)
- NVIDIA A100 (8)
- LLMs (7)
- company (7)
- gpu clusters (7)
- news (7)
- deep learning (6)
- hyperplane (6)
- training (6)
- CNNs (4)
- generative networks (4)
- machine learning (4)
- presentation (4)
- research (4)
- rtx a6000 (4)
- text generation (4)
Recent posts
In this blog, we will outline the benefits of our new Reserved Cloud Cluster and an example of how Voltron Data is using it to work with large datasets.
Published 11/01/2022 by Lauren Watkins
This post discusses the Total Cost of Ownership (TCO) for a variety of Lambda A100 servers and clusters. We calculate the TCO for individual Hyperplane-A100 servers, compare the cost with renting a AWS p4d.24xlarge instance, and walk through the cost of building and operating A100 clusters.
Published 09/22/2021 by Chuan Li
...