Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this March 11, 2025.

This update includes:

• Model Support
  • Added STDiT for OpenSoRA text-to-video support. Refer to examples/stdit/README.md.
• Features
  • The C++ runtime is now open sourced.
  • Added batched tensor FP4 quantization support.
  • Added a /metrics endpoint for trtllm-serve to log iteration statistics.
  • PyToch workflow
    • Added a C++ based decoder, which added support for:
    • TopK / TopP.
    • Bad words.
    • Stop words.
    • Embedding bias.
• AutoDeploy
  • Updated MoE pattern matcher and fused MoE op for DeepSeek Support.
• Bug fixes
  • Fixed incorrect output for Llama-3.2-11B-Vision-Instruct. (Multimodal Cross-attention incorrect results in #2796)

Thanks,
The TensorRT-LLM Engineering Team