Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: Options in training stage for recipes #628

Open
joeoct93 opened this issue Jul 2, 2022 · 8 comments
Open

KeyError: Options in training stage for recipes #628

joeoct93 opened this issue Jul 2, 2022 · 8 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@joeoct93
Copy link

joeoct93 commented Jul 2, 2022

🐛 Bug

When I tried to run a recipe (SuDORMRFNET and SuDORMRFImprovedNet), I always end up running into this problem in stage 2 like the following.

Stage 2: Training
Traceback (most recent call last):
File "/mnt/c/Project/asteroid/egs/librimix/SuDORMRFNet/train.py", line 140, in
arg_dic, plain_args = parse_args_as_dict(parser, return_plain_args=True)
File "/home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages/asteroid/utils/parser_utils.py", line 135, in parse_args_as_dict
args_dic["main_args"] = args_dic["optional arguments"]
KeyError: 'optional arguments'

To Reproduce

cd /asteroid/egs/librimix/SuDORMRFImprovedNet
. ./run.sh

or
python train.py

Expected behavior

The training commences

Environment

Package versions

Run asteroid-versions and paste the output here:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchoi1022/.local/bin/asteroid-versions", line 33, in
sys.exit(load_entry_point('asteroid', 'console_scripts', 'asteroid-versions')())
File "/home/cchoi1022/.local/bin/asteroid-versions", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py", line 5, in
import pytorch_lightning as pl
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py", line 30, in
from pytorch_lightning.callbacks import Callback # noqa: E402
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py", line 14, in
from pytorch_lightning.callbacks.base import Callback
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py", line 25, in
from pytorch_lightning.utilities.types import STEP_OUTPUT
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py", line 18, in
from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py", line 29, in
from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py", line 22, in
import pkg_resources
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3254, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3237, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3266, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pyYAML
pyYAML: command not found
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ pip install --upgrade pyyaml
Requirement already satisfied: pyyaml in /home/cchoi1022/cd/envs/asteroid/lib/python3.10/site-packages (5.4.1)
Collecting pyyaml
Using cached PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
Installing collected packages: pyyaml
Attempting uninstall: pyyaml
Found existing installation: PyYAML 5.4.1
Uninstalling PyYAML-5.4.1:
Successfully uninstalled PyYAML-5.4.1
Successfully installed pyyaml-6.0
(asteroid) cchoi1022@DESKTOP-Q470KUQ:/mnt/c/Project/asteroid/egs/librimix/SuDORMRFImprovedNet$ asteroid-versions
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 584, in _build_master
ws.require(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 901, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cchoi1022/.local/bin/asteroid-versions", line 33, in
sys.exit(load_entry_point('asteroid', 'console_scripts', 'asteroid-versions')())
File "/home/cchoi1022/.local/bin/asteroid-versions", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.8/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/mnt/c/Project/asteroid/asteroid/scripts/asteroid_versions.py", line 5, in
import pytorch_lightning as pl
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/init.py", line 30, in
from pytorch_lightning.callbacks import Callback # noqa: E402
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/init.py", line 14, in
from pytorch_lightning.callbacks.base import Callback
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/callbacks/base.py", line 25, in
from pytorch_lightning.utilities.types import STEP_OUTPUT
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/init.py", line 18, in
from pytorch_lightning.utilities.apply_func import move_data_to_device # noqa: F401
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py", line 29, in
from pytorch_lightning.utilities.imports import _compare_version, _TORCHTEXT_LEGACY
File "/home/cchoi1022/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/imports.py", line 22, in
import pkg_resources
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3254, in
def _initialize_master_working_set():
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3237, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 3266, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 586, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 599, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 792, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (PyYAML 5.3.1 (/usr/lib/python3/dist-packages), Requirement.parse('PyYAML>=5.4'), {'pytorch-lightning'})

Alternatively, if you cannot install Asteroid or have an old version that doesn't have the asteroid-versions script,
please output the output of:

pip freeze | egrep -i 'pytorch|torch|asteroid'

Additional info

Additional info (environment, custom script, etc...)

@joeoct93 joeoct93 added bug Something isn't working help wanted Extra attention is needed labels Jul 2, 2022
@mpariente
Copy link
Collaborator

Did you try solving the issue?

@joeoct93
Copy link
Author

joeoct93 commented Jul 6, 2022

I tried looking online for solutions, but could not find anything. Apparently this KeyError is telling me that "optional arguments" is not in the dictionary. But I simply installed asteroid, I don't understand why 'optional arguments' is not in the dictionary.

@miaris98
Copy link

miaris98 commented Nov 8, 2022

same here any tips?

@mpariente
Copy link
Collaborator

Sorry, I don't have any bandwidth to debug this.
My tip : have a look at the other recipes and look at what's different in the argument parsing, you'll probably find our error

@miaris98
Copy link

i also tried SuDORMRFImprovedNet recipe and it gives me the same error any more tips?

@jbartolewska
Copy link
Contributor

Key in argparse module for Python version above 3.10.10 changed from "optional arguments" into "options", hence two possible solutions:

  1. change in asteroid/utils/parser_utils.py args_dic["optional arguments"] to args_dic["options"]
  2. downgrade Python

@mpariente
Copy link
Collaborator

Thanks for the info, that was my idea !

We can also restrict argparse's version which is much less restrictive than the Python version.

@mpariente
Copy link
Collaborator

We can also restrict argparse's version which is much less restrictive than the Python version.

My bad, argparse is part of Python -_-

So, change to args_dic["options"] depending on python version here is probably the best. @jbartolewska would you like to do it ?

jbartolewska added a commit to jbartolewska/asteroid-original that referenced this issue Feb 22, 2023
YoshikiMas added a commit to YoshikiMas/asteroid that referenced this issue Mar 18, 2023
[src] Fix argparse key based on Python version (asteroid-team#628) (asteroid-team#657)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants