Add support for Poetry 1.5 lockfiles #7834
Conversation
deivid-rodriguez
force-pushed
the
deivid-rodriguez/poetry-1.5
branch
3 times, most recently
from
f9e3de4
to
3874bb3
Compare
jeffwidman
force-pushed
the
deivid-rodriguez/poetry-1.5
branch
from
3874bb3
to
cff72d3
Compare
|
@deivid-rodriguez I think the email might be the issue with the attribution
I think that should be:
|
deivid-rodriguez
force-pushed
the
deivid-rodriguez/poetry-1.5
branch
2 times, most recently
from
78bf3cc
to
78c3d48
Compare
|
There you go, thanks! |
Instead of parsing the lockfile directly, use `poetry show --only main`, and consider the dependencies there as "production", and the others as "development". This has the advantage of not having to parse the lockfiles directly, and as a result fixes an issue where we try to parse a "category" attribute in the lockfile which has been removed in the latest lockfile format. Ideally `poetry show` would support JSON output, but for now this should be good enough. Co-authored-by: Phillip Verheyden <[email protected]>
jeffwidman
force-pushed
the
deivid-rodriguez/poetry-1.5
branch
from
78c3d48
to
37c5a94
Compare
There was a problem hiding this comment.
👍 Looks good to me other than possibly the helper can be made more generic. But that can be handled in a follow-on PR, if appropriate.
I'm going to deploy and merge this to unblock all the folks affected by it.
Sorry for the delay here, many thanks to @phillipuniverse for pushing on this repeatedly.
I didn't have bandwidth to look at this til we got through dropping 3.6/3.7, but it does simplify things a bit now that we pin to solely to poetry==1.5.1.
BTW, researching all the background on this issue (I hadn't spent much time looking at poetry before today) resulted in some interesting dinner conversation:
My wife: "Honey, what'd you do today?"
Me: "Read a bunch about poetry"
Her: ???...
python/lib/dependabot/python/file_parser/pyproject_files_parser.rb
Outdated
Show resolved
Hide resolved
| File.write(lockfile.name, lockfile.content) | ||
|
|
||
| begin | ||
| output = Helpers.run_poetry_command("pyenv exec poetry show --only main") |
There was a problem hiding this comment.
👍 for using the poetry CLI rather than inspecting the file directly.
However, the docs for show indicate that --only main and --without dev are both workable here...
At first I was thinking that --without dev might be better here, as main is the default group so there may be other groups, until I saw this note here:
Dependency groups, other than the implicit
maingroup, must only contain dependencies you need in your development process. Installing them is only possible by using Poetry.
So I'm guessing docs/tests, etc are other groups that won't be considered part of main, but aren't part of dev either...
The 
concept of dep categories isn't quite so sophisticated, we tend to simply have "prod" vs "dev", and in this method solely want prod deps, not "non-dev deps", so --only main makes sense to me!
There was a problem hiding this comment.
Yeah, in general I tend to consider "dev", everything that's not production/runtime, that's why I went with --only main.
jeffwidman
force-pushed
the
deivid-rodriguez/poetry-1.5
branch
from
a903116
to
77fe93b
Compare
|
I tested in our staging environment using this new code against an old-style lockfile with the |
|
Hi, I'm not sure if I should open new issue for this, but seems that even after this released, Dependabot still removes I tried it today on one of the repos, and PR has all categories removed, example: |
|
@nantiferov this is expected behavior. Dependabot does not have a way to specify a version of Poetry to use (see #1556) and is currently using 1.5. When poetry.lock is re-created with Poetry 1.5, the category is always removed. The only fix really is to make sure that you are using Poetry 1.5 everywhere (locally, CI, etc) to ensure the category key stays removed. This PR was about resolving an issue where if that category key was removed, Dependabot could not parse it correctly. |
|
Ah, ok, now it's clear. Thank you! |
…oetry-1.5 Add support for Poetry 1.5 lockfiles
Since we upgraded to Poetry 1.5, we're failing to provide updates to any projects including Poetry 1.5 specific
poetry.lockfiles, which have removed the "category" attribute from locked dependencies.There's an open PR to fix this at #7418, however, I'm not fully convinced it's the right solution because:
I think this PR provides a better solution which applies one of the principles we're trying to apply recently: delegate to the package manager as much as possible. So, we let
poetry show --only mainlet us find the list of "production" dependencies.Also, the logic is moved to the parser. If we get the dependency type right there, then the proper information is already carried out as
Dependency#production?.Fixes #7389.
Fixes #7641.
Supersedes #7418.