Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we have pushed an update to the development branch (and the Triton backend) this Sep 10, 2024.

This update includes:

• Features
  • Supported lookahead decoding (experimental), see docs/source/speculative_decoding.md.
  • Supported LoRA in ModelRunnerCpp.
  • Supported openai-gelu.
  • Supported head_size=48 cases for FMHA kernels.
  • Added FP8 examples for DiT models, see examples/dit/README.md.
  • Supported LoRA for the LLM class.
  • Supported decoder with encoder input features for the C++ executor API.
  • Supported engine building using dummy weights for the LLM class.
• API
  • [BREAKING CHANGE] multi_block_mode is enabled by default.
• Documentation
  • Update the latest news, thanks to the contribution from @Sherlock113 in Add blog for Tuning TensorRT-LLM for Optimal Serving #2169.

Thanks,
The TensorRT-LLM Engineering Team