Model Optimization Tasks: Stage 2

[Ishaan-Datta]

Stage 2: Optimizing Engine Conversion

• Work with Kezia for model conversion/inference approaches

• ONNX to TRT Project

• Research different engine export methods and benchmark results/create performance report

• ex. PyTorch -> TensorRT (Torch-TRT or torchscript)

• ex. PyTorch -> ONNX -> TensorRT

• ex. Later: TensorFlow -> TensorRT (TF-TRT)

• ex. Later: Tensorflow -> ONNX -> TensorRT

• ex. Ultralytics conversion

• ex. trtexec CLI conversion

• Research different TensorRT runtimes

• Research different TensorRT Engine Plugins

• Investigate GStreamer

• Investigate DeepStream

• Nvidia Library Converter

• Basic Explanation Article

• Deploying Programs

• Quick Start

• Dynamo Export

• Engine Caching

• Compiling Exported Programs

• Exporting CUDA Graphs

Bonus:

• TensorFlow to ONNX Library (useful comparison for conversion backend)