Model Optimization Onboarding

[Ishaan-Datta]

Nvidia Video Tutorials:

• Jetson AI Fundamentals - S3E2 - Image Classification Inference - YouTube
• Jetson AI Fundamentals - S3E4 - Object Detection Inference - YouTube
• Jetson AI Fundamentals - S3E5 - Training Object Detection Models - YouTube
• Inference Optimization with NVIDIA TensorRT - YouTube

Documentation:

• Nvidia Documentation
• Nvidia Basic Workflow (for making docs)
• Nvidia Guide for Benchmarking/Profiling
• Creating a guide on model optimization after doing research on how layer fusion operations work and doing some benchmarking
• Comprehensive Guide to TF-TRT background (still useful for pytorch)
• Python API Documentation

PyTorch Exporting:

• CUDA Graphs

Code Examples:

• ONNX TensorRT
• Nvidia Tutorial Example
• Updated Nvidia Tutorial
• Runtimes
• C++ TensorRT workflow

ROS2 Crash Course:

• ROS2 crash course

Optimization:

• ONNX Graphsurgeon
• ONNX graphsurgeon examples
• Nvidia Object Detection Examples
• Nvidia Example ONNX Deployment
• Nvidia Guide for Optimization
• Boosting Performance and Utilization with Multi-Instance GPU - YouTube
• Running Multiple Workloads on a Single A100 GPU - YouTube
• Ultralytics INT8 Calibration Explanation
• Ultralytics Model Validatio
• Ultralytics TensorRT Whitepaper

Investigate Nvidia DLA Support on Jetson (NEW!):

• Ultralytics Guide
• Nvidia Whitepaper
• Jetson Module Specs
• DLA Supported Modules
• More Nvidia DLA Documentation

Nvidia Libraries:

• NCCL
• RAPIDS
• DALI
• TensorRT
• cuDNN: library to accelerate deep neural network computations
• cuBLAS
• CUDA Toolkit
• RAPIDS
• Support Matrix
• Gstreamer
• DeepStream SDK
• DALI
• NVCC
• cuRAND
• cuFFT
• cuSolver
• cuSPARSELt: library to accelerate sparse matrix-matrix multiplication
• cuPTI
• NVML
• DLPack
• Thrust
• cuTensor