Inference with Torch-TensorRT Deep Learning Prediction for Beginners - CPU vs CUDA vs TensorRT

Hi everyone! 😀 In the last video we’ve seen how to accelerate the speed of our programs with Pytorch and CUDA - today we will take it another step further with Torch-TensorRT! We will focus on a Machine Learning process called Inference (which is when the model is trained, perfected and ready to make a prediction). For this we will load a state-of-the-art artificial neural network and we will use it to classify a picture of my cat! 🙀🙀🙀 Specifically - we will borrow ResNet50 for our little Pytorch experiment! 😉 We will also run a speed test comparing Pytorch models running on CPU, on CUDA and on Torch-TensorRT - which of these do you think is faster?? ⏲️ TIMESTAMPS ⏲️ ----------------------------------- 00:00 - intro 01:05 - clone Torch-TensorRT 01:40 - install and setup Docker 03:52 - install Nvidia Container Toolkit & Nvidia Docker 2 05:02 - Torch-TensorRT container (option #1) 07:22 - Torch-TensorRT Nvidia NGC container (option #2) 09:00 - import Pytorch 09:16 - load ResNet50 10:25 - load sample image 11