CM error: Extracted TensorRT folder does not seem proper - Version information missing!

[thehalfspace]

I'm getting this new error for tensorrt framework bert inference implementation on grace-hopper 200:

Here is the output:

Apptainer> cm run script "get tensorrt _dev" --tar_file=/my/path/TensorRT-10.1.0.27.Ubuntu-22.04.aarch64-gnu.cuda-12.4.tar.gz
INFO:root:* cm run script "get tensorrt _dev"
INFO:root:  * cm run script "detect os"
INFO:root:         ! cd /tmp/001/CM/repos/local/cache/e217b3c0efc44d75
INFO:root:         ! call /tmp/001/CM/repos/mlcommons@cm4mlops/script/detect-os/run.sh from tmp-run.sh
INFO:root:         ! call "postprocess" from /tmp/001/CM/repos/mlcommons@cm4mlops/script/detect-os/customize.py
INFO:root:  * cm run script "get python3"
INFO:root:       ! load /tmp/001/CM/repos/local/cache/5126da4b74e84019/cm-cached-state.json
INFO:root:Path to Python: /usr/bin/python3
INFO:root:Python version: 3.10.12
Untaring file - can take some time ...

CM error: Extracted TensorRT folder does not seem proper - Version information missing!
Apptainer> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:24:28_PDT_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0
Apptainer>

I intend to run this command:

cm run script --tags=run-mlperf,inference,_r4.1 \
   --model=bert-99 \
   --implementation=nvidia \
   --framework=tensorrt \
   --category=edge \
   --scenario=Offline \
   --execution_mode=valid \
   --device=cuda \
   --quiet \

Any hints?

[arjunsuresh]

@thehalfspace Nvidia implementation requires --docker option. Please follow this docs page for the latest commands.
https://docs.mlcommons.org/inference/benchmarks/language/bert/#__tabbed_50_1

But we haven't tested on GH200 yet. You might have to add --batch_size=64 if the given run command fails.