poetry-plugin-up: init at 0.2.1 #211231

K900 · 2023-01-17T12:29:11Z

Description of changes

A plugin to update dependencies to latest even if pyproject.toml says not to.

Things done

mweinelt

Use pytestCheckHook to run the tests.

pkgs/tools/package-management/poetry-plugin-up/default.nix

dotlambda · 2023-01-17T12:51:13Z

pkgs/top-level/all-packages.nix

@@ -16383,6 +16383,8 @@ with pkgs;

  poetry = callPackage ../tools/package-management/poetry { };

+  poetry-plugin-up = callPackage ../tools/package-management/poetry-plugin-up { };


This should be in python-packages.nix.

Why? It's not going to be useful with any Python version but the one Poetry is built with.

It could also live at poetry.plugins or something like that.

I thought about that but I'm not sure if it's worth adding another scope when we only have one package to put under it.

It is. We can ensure that plugins use the same Python package set in two ways: the plugins argument becomes a function that is passed python.pkgs (see home-assistant) and plugins are packaged in python3Packages, or plugins live in poetry.plugins where they're automatically built with the correct Python package set.

K900 · 2023-01-17T12:55:45Z

Use pytestCheckHook to run the tests.

Tests won't run because we don't have a Poetry.

K900 · 2023-01-17T12:57:10Z

Applied all the style fixes.

mweinelt · 2023-01-17T12:58:29Z

Tests won't run because we don't have a Poetry.

Add poetry to checkInputs, disable doCheck, import the package itself, create an override in passthru.tests á la poetry-plugin-up.overridePythonAttrs (_: { doCheck = true; })

K900 · 2023-01-17T13:00:02Z

That's still not going to work because Poetry doesn't propagate its dependencies (see #209534).

SuperSandro2000 · 2023-01-17T15:58:22Z

pkgs/tools/package-management/poetry/default.nix

+    ] ++ lib.optionals (pythonOlder "3.8") [
+      backports-cached-property


We can remove that, we no longer support 3.8

K900 · 2023-01-22T12:09:53Z

Rebased to catch the nativeCheckInputs change.

Introduce a wrapper for Poetry instead of manually unpropagating things, which also allows using unwrapped Poetry as an input when necessary. Also add poetry-plugin-up as an example.

dotlambda

I think this is more complicated than it needs to be. I don't see why one type of wrapping shouldn't suffice and I'm not fond of the $PYTHONPATH hackery.
Thus I tried to achieve the same thing just using overridePythonAttrs and thought that it's maybe a good idea to put the actual expression for poetry in a separate file unwrapped.nix. See #213914 for my suggestion.

dotlambda · 2023-01-17T16:07:47Z

pkgs/tools/package-management/poetry/default.nix

+      passthru = {
+        inherit unwrapped python;
+        withPlugins = selector: let
+          pluginsSet = python.pkgs.callPackage ./plugins {


You can use import

K900 · 2023-02-01T06:55:08Z

The reason I went with a double wrapper here is mostly to avoid rebuilding poetry itself, which is not particularly slow but also kinda slow.

dotlambda · 2023-02-01T06:58:38Z

The reason I went with a double wrapper here is mostly to avoid rebuilding poetry itself, which is not particularly slow but also kinda slow.

That does make sense indeed. I guess I could copy the double wrapper in my approach if that's preferred.
I still think having poetry.plugins is a bit cleaner and makes sure all plugins use the same versions of all Python packages.

K900 · 2023-02-01T07:03:24Z

Cleaner than what? Sorry, might be missing something...

dotlambda · 2023-02-01T07:08:31Z

Cleaner than what? Sorry, might be missing something...

You're right, your approach is almost the same in this regard. So never mind.

dotlambda · 2023-02-01T07:21:05Z

pkgs/tools/package-management/poetry/default.nix

+
+    wrapWithPlugins = ''
+      buildPythonPath $plugins
+      makeWrapper ${unwrapped}/bin/poetry $out/bin/poetry --suffix PYTHONPATH : "$program_PYTHONPATH"


This is basically a no-op if $program_PYTHONPATH is empty. Is there a specific reason you're using a different wrapper in that case?

It's a tiny efficiency thing, probably not worth worrying about realistically.

mweinelt · 2023-02-02T13:48:26Z

#213914 looks simpler and is the more python-native approach using packageOverrides for the package set and overridePythonAttrs to propagate plugins into the poetry package.

I think I prefer it also because it relies less on a bespoke wrapping logic, like is the case here.

The downside of the override approach is that you get to rebuild poetry, which takes a minute on my desktop due to five very slow tests. We could however also just disable tests on the overridden derivation to get that penalty out of the way.

(formed this opinion while discussing the situation with K900 in a private chat)

K900 · 2023-02-02T13:50:52Z

I definitely like the packageOverrides approach and probably would have done the same thing if I remembered it existed at the time (lol). I'm still not 100% convinced on the wrapper versus adding inputs part, but it's also definitely not a blocker and either way should be fine.

dotlambda · 2023-02-02T17:13:26Z

We could however also just disable tests on the overridden derivation to get that penalty out of the way.

That's a good compromise.

I'm still not 100% convinced on the wrapper versus adding inputs part

The advantage of adding inputs is that it's going to be correct in the future even if wrapPython's logic changes. We also won't have to change the wrapper if files are addded to poetry's $out/share or whatever.

K900 · 2023-02-02T17:14:19Z

Then I'm just going to close this in favor of #213914.

K900 requested review from mweinelt and dotlambda January 17, 2023 12:39

ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 10.rebuild-linux: 1 labels Jan 17, 2023

mweinelt requested changes Jan 17, 2023

View reviewed changes

dotlambda reviewed Jan 17, 2023

View reviewed changes

K900 force-pushed the poetry-plugin-up branch from 940f57c to ef2b18c Compare January 17, 2023 12:56

K900 force-pushed the poetry-plugin-up branch from ef2b18c to 234dbed Compare January 17, 2023 15:25

ofborg bot requested review from jbaum98 and dotlambda January 17, 2023 15:45

SuperSandro2000 reviewed Jan 17, 2023

View reviewed changes

K900 requested a review from mweinelt January 19, 2023 09:55

K900 force-pushed the poetry-plugin-up branch from 234dbed to 265b7f6 Compare January 22, 2023 12:09

K900 force-pushed the poetry-plugin-up branch 3 times, most recently from 9db73c1 to 8dd486f Compare January 24, 2023 11:34

SuperSandro2000 added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 24, 2023

poetry: allow adding plugins

2c96102

Introduce a wrapper for Poetry instead of manually unpropagating things, which also allows using unwrapped Poetry as an input when necessary. Also add poetry-plugin-up as an example.

K900 force-pushed the poetry-plugin-up branch from 8dd486f to 2c96102 Compare January 26, 2023 14:13

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 26, 2023

dotlambda reviewed Feb 1, 2023

View reviewed changes

dotlambda mentioned this pull request Feb 1, 2023

poetryPlugins.poetry-plugin-up: init at 0.2.1 #213914

Merged

13 tasks

dotlambda reviewed Feb 1, 2023

View reviewed changes

K900 closed this Feb 2, 2023

dotlambda mentioned this pull request Feb 8, 2024

python311Packages.mdformat-*: propagate dependencies #287103

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

poetry-plugin-up: init at 0.2.1 #211231

poetry-plugin-up: init at 0.2.1 #211231

K900 commented Jan 17, 2023 •

edited

Loading

mweinelt left a comment

dotlambda Jan 17, 2023

K900 Jan 17, 2023

mweinelt Jan 17, 2023

K900 Jan 17, 2023

dotlambda Jan 17, 2023

K900 commented Jan 17, 2023

K900 commented Jan 17, 2023

mweinelt commented Jan 17, 2023

K900 commented Jan 17, 2023

SuperSandro2000 Jan 17, 2023

K900 commented Jan 22, 2023

dotlambda left a comment

dotlambda Jan 17, 2023

K900 commented Feb 1, 2023

dotlambda commented Feb 1, 2023

K900 commented Feb 1, 2023

dotlambda commented Feb 1, 2023

dotlambda Feb 1, 2023

K900 Feb 1, 2023

mweinelt commented Feb 2, 2023 •

edited

Loading

K900 commented Feb 2, 2023

dotlambda commented Feb 2, 2023

K900 commented Feb 2, 2023

		@@ -16383,6 +16383,8 @@ with pkgs;

		poetry = callPackage ../tools/package-management/poetry { };

		poetry-plugin-up = callPackage ../tools/package-management/poetry-plugin-up { };

		] ++ lib.optionals (pythonOlder "3.8") [
		backports-cached-property

poetry-plugin-up: init at 0.2.1 #211231

poetry-plugin-up: init at 0.2.1 #211231

Conversation

K900 commented Jan 17, 2023 • edited Loading

Description of changes

Things done

mweinelt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

K900 commented Jan 17, 2023

K900 commented Jan 17, 2023

mweinelt commented Jan 17, 2023

K900 commented Jan 17, 2023

Choose a reason for hiding this comment

K900 commented Jan 22, 2023

dotlambda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

K900 commented Feb 1, 2023

dotlambda commented Feb 1, 2023

K900 commented Feb 1, 2023

dotlambda commented Feb 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt commented Feb 2, 2023 • edited Loading

K900 commented Feb 2, 2023

dotlambda commented Feb 2, 2023

K900 commented Feb 2, 2023

K900 commented Jan 17, 2023 •

edited

Loading

mweinelt commented Feb 2, 2023 •

edited

Loading