Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion error: self.metadata.channels == other.metadata.channels when updating lockfile with added channels #283

Open
klieret opened this issue Nov 15, 2022 · 4 comments

Comments

@klieret
Copy link
Contributor

klieret commented Nov 15, 2022

Edit: This only happens when running with the --lockfile option. It seems to be a bug that is triggered when updating a lock file that doesn't contain a new channel yet.

Workaround: Delete lock file and run without --lockfile option

What I ran:

conda-lock -p linux-64 -p win-64 -f default.yml --lockfile default.conda-lock.yml

Output:

Locking dependencies for ['linux-64', 'win-64']...
INFO:conda_lock.conda_solver:linux-64 using specs ['python 3.10', 'cudatoolkit 11.6.*', 'pytorch 1.12', 'pyg', 'ray-tune', 'optuna', 'wandb', 'numpy', 'pandas', 'scikit-learn', 'pip']
INFO:conda_lock.conda_solver:win-64 using specs ['python 3.10', 'cudatoolkit 11.6.*', 'pytorch 1.12', 'pyg', 'ray-tune', 'optuna', 'wandb', 'numpy', 'pandas', 'scikit-learn', 'pip']
Traceback (most recent call last):
  File "/scratch/gpfs/kl5675/miniconda3/bin/conda-lock", line 10, in <module>
    sys.exit(main())
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 1190, in lock
    lock_func(
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 949, in run_lock
    make_lock_files(
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/conda_lock/conda_lock.py", line 389, in make_lock_files
    lock_content = lock_content | create_lockfile_from_spec(
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/conda_lock/src_parser/__init__.py", line 150, in __or__
    return other.__ror__(self)
  File "/scratch/gpfs/kl5675/miniconda3/lib/python3.9/site-packages/conda_lock/src_parser/__init__.py", line 161, in __ror__
    assert self.metadata.channels == other.metadata.channels
AssertionError
make: *** [Makefile:2: default.conda-lock.yml] Error 1

The file default.yml:

name: gnn
channels:
  - pytorch
  - pyg
  - ostrokach-forge
  - conda-forge
  - scikit-learn
dependencies:
  - python==3.10
  - cudatoolkit=11.6
  - pytorch==1.12
  - pyg
  - ray-tune
  - optuna
  - wandb
  - numpy
  - pandas
  - scikit-learn
  - pip

Additional information using conda-lock --pdb:

(Pdb) self.metadata.channels
[Channel(url='pytorch'), Channel(url='pyg'), Channel(url='ostrokach-forge'), Channel(url='conda-forge'), Channel(url='scikit-learn')]
(Pdb) other.metadata.channels
[Channel(url='pytorch'), Channel(url='pyg'), Channel(url='ostrokach-forge'), Channel(url='conda-forge')]
@klieret klieret changed the title Assertion error: self.metadata.channels == other.metadata.channels Assertion error: self.metadata.channels == other.metadata.channels when updating lockfile Nov 15, 2022
@klieret klieret changed the title Assertion error: self.metadata.channels == other.metadata.channels when updating lockfile Assertion error: self.metadata.channels == other.metadata.channels when updating lockfile with added channels Nov 15, 2022
@mariusvniekerk
Copy link
Collaborator

Channel addition can ONLY be additive on the top.

In this case you should just delete your existing lockfile and relock with the new channels

@maresb
Copy link
Contributor

maresb commented Nov 15, 2022

Given things like this and #196, how would you feel about by default regenerating the lockfile from scratch. I don't have experience with other lockfile systems, but I find it very unintuitive that lockfiles are stateful.

@klieret
Copy link
Contributor Author

klieret commented Nov 15, 2022

The fact that there is additive behavior is also not clear from the readme or command line options. In fact, it only talks about using this option to supply a different output filename (and this is what I was looking for).

(Also, it generally seems to be discouraged to use assert to catch errors based on external inputs, because assert statements are switched off for optimizations)

@klieret
Copy link
Contributor Author

klieret commented Nov 15, 2022

Channel addition can ONLY be additive on the top.

But this is what happened here. It seems to me that any channel modification is vetoed by the assert statement. Or perhaps I misunderstood you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants