Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] #599

Closed
grpinto opened this issue Oct 17, 2023 · 2 comments
Closed

[Bug] #599

grpinto opened this issue Oct 17, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@grpinto
Copy link

grpinto commented Oct 17, 2023

Hi !!

I am trying to pass this command in google colab :

!CUDA_VISIBLE_DEVICES=0 PORT=29500 ./tools/dist_train.sh \ {/content/mmpose/configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/cspnext-m_udp_8xb64-210e_coco-wholebody-256x192.py}/group_fisher_{act}prune{cspnext-m_udp_8xb64-210e_coco-wholebody-256x192}.py 1
--work-dir '/content/mmrazor/work_dir'

and I am receiving this error :

/usr/local/lib/python3.10/dist-packages/torch/distributed/launch.py:181: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use-env is set by default in torchrun.
If your script expects --local-rank argument to be set, please
change it to read from os.environ['LOCAL_RANK'] instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

warnings.warn(
usage: train.py [-h] [--work-dir WORK_DIR] [--amp] [--auto-scale-lr] [--resume]
[--cfg-options CFG_OPTIONS [CFG_OPTIONS ...]]
[--launcher {none,pytorch,slurm,mpi}] [--local_rank LOCAL_RANK]
config
train.py: error: unrecognized arguments: --local-rank=0
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 2) local_rank: 0 (pid: 10358) of binary: /usr/bin/python3
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/launch.py", line 196, in
main()
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/launch.py", line 192, in main
launch(args)
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/launch.py", line 177, in launch
run(args)
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/run.py", line 785, in run
elastic_launch(
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/lib/python3.10/dist-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

./tools/train.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2023-10-17_14:58:52
host : 511a211dac06
rank : 0 (local_rank: 0)
exitcode : 2 (pid: 10358)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Did somebody had the same problem ?

Thank you for your attention

@grpinto grpinto added the bug Something isn't working label Oct 17, 2023
@grpinto
Copy link
Author

grpinto commented Oct 17, 2023

What I am trying to do is prunning an model from cpsnext called mmpose, is there any way to do this using a command ?

@LKJacky
Copy link
Collaborator

LKJacky commented Oct 18, 2023

It seems that you use torch >= 2.0. Please use torch <2.0

@grpinto grpinto closed this as completed Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants