jf poetry install runs poetry update

[abdsahin]

Describe the bug

I am trying to use jf cli to install the python dependencies. When I run the jf poetry install, it runs internally poetry update, which is not intended since it is used in the Pipeline and I don't want my lock file to be updated.

What I see is that there is always a call chain as follow
Run --> SetPypiRepoUrlWithCredentials --> ConfigPoetryRepo --> addRepoToPyprojectFile which runs a poetry update command everytime.

Current behavior

👾 install:ci | jf poetry-config --repo-resolve $JFROG_PLATFORM_PYPI_REPO
16:32:19 [Debug] JFrog CLI version: 2.38.4
16:32:19 [Debug] OS/Arch: linux/amd64
16:32:19 [Info] poetry build config successfully created.
👾 install:ci | jf poetry install --sync
16:32:19 [Debug] JFrog CLI version: 2.38.4
16:32:19 [Debug] OS/Arch: linux/amd64
16:32:19 [Debug] Preparing to read the config file /builds/test/folder/solution-teams/the-awesome-team/folder-awscdk-python-app-poetry2/.jfrog/projects/poetry.yaml
16:32:19 [Debug] Found resolver in the config file /builds/test/folder/solution-teams/the-awesome-team/folder-awscdk-python-app-poetry2/.jfrog/projects/poetry.yaml
16:32:19 [Info] Running Poetry install.
16:32:19 [Debug] Preparing build prerequisites...
16:32:19 [Debug] Saving build general details at: /tmp/jfrog/builds/1339a8cb9483d833fdde458e1b4402202949ab77a6328d843980fd78485f1d55/partials
16:32:19 [Info] Running Poetry config repositories.jfrog-server https://url-of-the-artifactory/artifactory/api/pypi/tat-pypi/simple
16:32:19 [Debug] Usage Report: Sending info...
16:32:19 [Debug] Sending HTTP GET request to: https://url-of-the-artifactory/artifactory/api/system/version
16:32:19 [Debug] Artifactory response: 200
16:32:19 [Debug] JFrog Artifactory version is: 7.59.9
16:32:19 [Debug] Sending HTTP POST request to: https://url-of-the-artifactory/artifactory/api/system/usage
16:32:19 [Debug] Usage Report: Usage info sent successfully. Artifactory response: 200
16:32:19 [Info] Running Poetry config ***
Using a plaintext file to store credentials
16:32:20 [Info] Added tool.poetry.source name:"jfrog-server" url:"https://url-of-the-artifactory/artifactory/api/pypi/tat-pypi/simple"
16:32:20 [Info] Running Poetry update
Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 22 installs, 0 updates, 0 removals

• Installing attrs (23.1.0)
• Installing exceptiongroup (1.1.1)
• Installing six (1.16.0)
• Installing cattrs (22.2.0)
• Installing importlib-resources (5.12.0)
• Installing python-dateutil (2.8.2)
• Installing typeguard (2.13.3)
• Installing publication (0.0.3)
• Installing typing-extensions (4.6.3)
• Installing iniconfig (2.0.0)
• Installing jsii (1.83.0)
• Installing packaging (23.1)
• Installing pluggy (1.0.0)
• Installing tomli (2.0.1)
• Installing aws-cdk-asset-awscli-v1 (2.2.189)
• Installing aws-cdk-asset-kubectl-v20 (2.1.1)
• Installing aws-cdk-asset-node-proxy-agent-v5 (2.0.163)
• Installing constructs (10.2.52)
• Installing pytest (7.3.1)
• Installing coverage (7.2.7)
• Installing aws-cdk-lib (2.83.1)
• Installing pytest-cov (4.1.0)
/root/.cache/pypoetry/virtualenvs/folder-awscdk-python-app-poetry2-2HAYbF5F-py3.10
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 0 updates, 0 removals, 22 skipped

• Installing attrs (23.1.0): Skipped for the following reason: Already installed
• Installing aws-cdk-asset-awscli-v1 (2.2.189): Skipped for the following reason: Already installed
• Installing aws-cdk-asset-kubectl-v20 (2.1.1): Skipped for the following reason: Already installed
• Installing aws-cdk-asset-node-proxy-agent-v5 (2.0.163): Skipped for the following reason: Already installed
• Installing aws-cdk-lib (2.83.1): Skipped for the following reason: Already installed
• Installing cattrs (22.2.0): Skipped for the following reason: Already installed
• Installing constructs (10.2.52): Skipped for the following reason: Already installed
• Installing exceptiongroup (1.1.1): Skipped for the following reason: Already installed
• Installing coverage (7.2.7): Skipped for the following reason: Already installed
• Installing importlib-resources (5.12.0): Skipped for the following reason: Already installed
• Installing pytest (7.3.1): Skipped for the following reason: Already installed
• Installing pluggy (1.0.0): Skipped for the following reason: Already installed
• Installing iniconfig (2.0.0): Skipped for the following reason: Already installed
• Installing pytest-cov (4.1.0): Skipped for the following reason: Already installed
• Installing tomli (2.0.1): Skipped for the following reason: Already installed
• Installing packaging (23.1): Skipped for the following reason: Already installed
• Installing typing-extensions (4.6.3): Skipped for the following reason: Already installed
• Installing six (1.16.0): Skipped for the following reason: Already installed
• Installing publication (0.0.3): Skipped for the following reason: Already installed
• Installing python-dateutil (2.8.2): Skipped for the following reason: Already installed
• Installing typeguard (2.13.3): Skipped for the following reason: Already installed
• Installing jsii (1.83.0): Skipped for the following reason: Already installed

Installing the current project: folder-awscdk-python-app-poetry2 (0.0.0)

Reproduction steps

No response

Expected behavior

No response

JFrog CLI-Core version

2.34.7

JFrog CLI version (if applicable)

2.38.4

Operating system type and version

mac, linux

JFrog Artifactory version

7.59.9

JFrog Xray version

No response

[abdsahin]

Any feedback here? To be honest, I wonder the reason why poetry update is run here.

When I run jf poetry update, poetry update is run twice and no build dependency is collected.

[dhtyler]

I am also impacted by this issue as well.

My pyproject.toml already specifies my artifactory server in tool.poetry.source so my lock file already references artifactory as the source for my packages.

This is a serious issue for applications that use version ranges in their pyproject.toml, but expect the versions specified in their lock files to be installed. By running "poetry update", the contents of any poetry.lock file are effectively ignored. This can causes dependencies to be installed at unexpected versions, since the poetry.lock file is used to track/lock down the specific versions which should be installed within the ranges specified with pyproject.toml.

If jfrog wants to continue to inject the modification the tools.poetry.source as a default behavior, could we get some sort of option to disable this for project that already specify their artifactory server in the pyproject.toml and need their lock files to be respected?

[gangefors]

jf poetry should not modify files in my checked out code unless I use a command from poetry that I expect modifies it.

jf poetry build completely rewrites my pyproject.toml. It shouldn't.
jf poetry build updates my depdendencies, rewriting poetry.lock. It shouldn't.

A build tool should not modify the code I intend to build from, unless I explicitly tell it to.

The wrapping of poetry commands is completely useless due to this behavior.

Can we get a reason for why jf cli modifies my code?

[gangefors]

I opened a support ticket with JFrog regarding the broken functionality in the jf poetry commands and I've quoted their answer below. Not exactly the one I hoped for, but understandable if the feature is broken and they don't want to/or have the bandwidth to fix it.

A community plugin might be the way to go here. All build info should be possible to collect from the lock file and building+deploying packages should not be that hard to wrap.
I need to learn how to code in Go.

We understand your concern regarding the lack of support for the Poetry in JFrog CLI for Artifactory. The issue you have mentioned (/issues/835 and /issues/1148) highlights the limitations of JFrog CLI when it comes to handling Poetry-managed ones.

To provide more context, JFrog CLI is designed to work seamlessly with popular package managers like pip for installation and python3 for package uploads to Artifactory. However, Poetry, while a popular choice for managing Python dependencies and packaging, is not officially supported by JFrog CLI for Artifactory.

In light of this limitation, the team had to remove all Poetry-related examples and support from JFrog CLI for Artifactory. This decision is based on the fact that Poetry isn't supported by JFrog CLI, and we want to ensure clarity and accuracy in our documentation and tooling.

We apologize for any confusion or inconvenience this may have caused. If you have any further questions/concerns, please feel free to reach out.

Thank you for your understanding and cooperation.

[BartSchuurmans]

I've made a PR that at least fixes the undesired version upgrades: #1235