Optimum supports AMDGPU ？

[taikai-zz]

Feature request

Onnxruntime supports AMD-ROCM ，
how to compile on optimum

Motivation

Our company is currently testing amdgpu and has learned that optim can accelerate inference on CUDA. We are not sure if it will support ROCM in the future?

Your contribution

none

[fxmarty]

Hi @taikai-zz, the ORTModel classes support ROCm GPUs for inference with ONNX Runtime, following this PR #1546. See as reference https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/amdgpu. However, from internal benchmarks, we did not see very large performance improvements compared to simply using PyTorch.

The status of the support of ROCm GPUs in Hugging Face libraries is summarized in https://huggingface.co/docs/optimum/main/en/amd/amdgpu/overview. Overall, the upstream support in PyTorch makes lots of things easy.