Skip to content

Commit

Permalink
remove comment
Browse files Browse the repository at this point in the history
Signed-off-by: Terry Kong <[email protected]>
  • Loading branch information
terrykong committed Nov 1, 2024
1 parent 8f080d6 commit 567b144
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions nemo/export/trt_llm/tensorrt_llm_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -511,9 +511,8 @@ def load_distributed(engine_dir, model_parallel_rank, gpus_per_node):
)
decoder = ModelRunner.from_engine(
engine=engine,
# rank=world_config.rank,
# We want the engine to have the mp_rank, but the python runtime to not resassign the device of the current process
# So we will set it to the current
# So we will set it to the current device
rank=torch.cuda.current_device(),
)

Expand Down

0 comments on commit 567b144

Please sign in to comment.