Boosting GPU Performance with CGX and FFCV: Faster Speeds, Lower Costs
Genesis Cloud is on a mission to make high-performance cloud computing efficient, accessible, and affordable for everyone. In our journey, we’ve been exploring two revolutionary tools that supercharge our GPU computing capabilities: CGX and FFCV. Individually, these tools offer impressive boosts in their respective domains. Now, imagine the potential when they join forces. In this blog post, we’re going to walk you through our exciting journey of integrating both FFCV and CGX into your deep learning multi-GPU training. We’ll discover together the promising outcomes this combination could potentially unlock. But first, let’s have a quick refresher on what each of these frameworks does.
CGX, a novel communication engine designed specifically for deep learning, has been developed in collaboration with researchers from the Institute of Science and Technology Austria (ISTA). CGX tackles the challenges of scaling up training processes across multiple consumer-grade GPUs like NVIDIA’s RTX 3090s offered by Genesis Cloud. Our recent research illustrates how CGX addresses the communication bandwidth bottleneck between GPUs, matching and potentially surpassing the performance of data-center grade GPUs like NVIDIA V100s.
Our detailed blog post on CGX provides a comprehensive tutorial on how to incorporate this innovative framework into your deep learning applications on Genesis Cloud. By leveraging CGX, you can now unlock significantly better performance-per-dollar on your multi-GPU training tasks.
Complementing CGX is FFCV - a built-in PyTorch-compatible library designed to supercharge the data loading process, consequently augmenting the throughput for model training. By integrating FFCV, we’ve witnessed impressive performance improvements and cost reductions compared to conventional PyTorch methods. In our previous post on I/O performance optimization, we documented the extensive benchmarks carried out on Genesis Cloud instances, particularly the newer CPU and memory-optimized RTX 3090 GPU instances. The results speak for themselves: FFCV delivers better performance faster and at a lower cost than vanilla PyTorch approaches.
CGX and FFCV: Better Together
But the real magic happens when CGX and FFCV are paired together. The combination of these two frameworks has proven to be a game-changer for GPU performance and cost-effectiveness. We recently released a guide on our Pytorch-CGX-FFCV repository, which details how to train CIFAR-10 using both CGX and FFCV.
Our tests have shown that this combination can cut the training time on an RTX 3090 Optimized provided by Genesis Cloud in half. This advancement has significant implications, particularly when compared to the cost of using other cloud providers. In our benchmarks, we noticed that training CIFAR-10 for 20 epochs to reach an accuracy of over 90% required only a quarter of the time taken by a vanilla code as you can see in detail in both the table and the graph below:
| Instance Type | Framework | GPUs | Price/h $ | Time (s) | Cost-to-Train | Δ Cost-to-Train |
|---|---|---|---|---|---|---|
| GC RTX 3090 Optimized | PyTorch | 4x RTX 3090 | $3.36 | 195 | $0.18 | baseline |
| GC RTX 3090 Optimized w/ CGX | CGX | 4x RTX 3090 | $3.36 | 63 | $0.06 | -67.90% |
| GC RTX 3090 Optimized w/ CGX+FFCV | CGX+FFCV | 4x RTX 3090 | $3.36 | 33 | $0.03 | -83.17% |
Training times on a Genesis Cloud RTX 3090 Optimized instance were halved, showcasing an astonishing near 85% reduction in both training time and cost-to-train. These remarkable advancements wouldn’t be possible without the combination of CGX and FFCV, and also the backing of Genesis Cloud’s newly revised pricing model which you can explore in more detail in our previous blog post. All these factors together drastically outperform other cloud providers like AWS and GCP, making Genesis Cloud a prime choice for users seeking peak efficiency and cost savings. For a more detailed look at these performance differences, we invite you to take a look at the figure below:
While CGX empowers consumer-grade GPUs like the RTX 3090 to match the performance of data center-grade GPUs, FFCV takes it a step further by speeding up the data loading process, maximizing the cost-effectiveness of our instances.
In conclusion, the integration of CGX and FFCV is emerging as a significant breakthrough in the realm of GPU computing. It’s a testament to Genesis Cloud’s commitment to make high-performance cloud computing more efficient and accessible.
Keep accelerating 🚀
The Genesis Cloud team
Never miss out again on Genesis Cloud news and our special deals: follow us on Twitter, LinkedIn, or Reddit. Sign up for an account with Genesis Cloud here and benefit from $15 in free credits. If you want to find out more, please write to [email protected].
Written on August 24th, 2023 by Marouane Khoukh