We are proud to introduce our new article series that will guide you on how to run state of the art deep learning models on Genesis Cloud infrastructure. These articles will be initially published as blog posts and will be added to our knowledge base after their release. Please note: The order of the articles is important as articles are written as a series and information contained in the initial articles might be required for understanding the subsequent articles.
In this series of articles we will use 1x RTX 3080 instance type on Genesis Cloud (our recommended GPU for inference use) and showcase four (4) different deployment strategies for deep learning inference using (a) PyTorch (TorchScript), (b) TensorRT, and (c) Triton.
For the models, we will focus on computer vision applications using the torchvision model collection. This collection will serve as an example and includes various pretrained versions of classic deep learning algorithms such as alexnet, densenet, mobilenet, resnet, shufflenet, and squeezenet.
In the early days of machine learning GPUs were mainly used for training deep learning models while inference could still be done on a CPU. While the field of machine learning progressed immensely in the past 10 years, the models have grown in both size and complexity, meaning that today the standard infrastructure setup for latency-sensitive deep learning applications are based on GPU cloud instances instead of CPU-only instances.
Rationale for using a GPU is not just performance but also cost. Compared to CPUs, GPUs are often two orders of a magnitude more efficient in processing deep neural networks. This means, that cost-savings can be achieved by switching to a GPU instance especially when operating with high throughput applications.
All you need are a Genesis Cloud GPU instance, a trained deep learning model, data to be processed, and the supporting software. We will show you how to master it all.
Each article will contain:
In case you aren’t using Genesis Cloud yet, get started here.
Now start accelerating on machine learning with Genesis Cloud 🚀
ImageNet dataset with pictures of dogs labelled by breed.Written on March 10th, 2022 by Stefan Ojanen