Fix: publish to pypi build needs security enhancements

[lwasser]

Following this issue - we should make some changes to our release workflow to make it more robust.

Action items

• Update action versions https://github.com/pyOpenSci/pyosMeta/actions/runs/8067048660 (specifically, the summary page, not inside the job, below the graph view), there are annotations. They contain 2 deprecation warnings that come from actions/checkout@v3, actions/setup-python@v2 — these action versions are old and may stop working at some point so it is a good idea to upgrade them sooner rather than later.
• break up our build and publish sections as suggested here. This will create a more secure build.

an immediate recommendation — don't build your dists in the same job as publishing. Exposing the build machinery to the OIDC privilege is dangerous security-wise, as it may lead to privilege escalation in other systems through build dep poisoning. https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ demonstrates good job separation

• Remove test pypi from our build. i've published to testpypi in the past to make sure things still build and publish. However test pypi doesn't maintainer current versions of packages so installing from there will never work - ie pydantic is super old on test pypi and unreliable (anyone could have uploaded it)

Another route towards doing this is pre-releases. i just don't fully remember how to do that but have done it once for stravalib when we had a major API refactor / upgrade.

• consider sigstore signing (maybe this is a new issue?)

[willingc]

Great work @lwasser. We should add a thanks in the release note to the maintainers that helped you with the action.

[lwasser]

I'm 100% on board with acknowledging them both in release notes @willingc as well as ofek as he's helped me a lot w hatch things. I'm so appreciative of all of the support in our community!!

[webknjaz]

Since you're documenting follow-ups here, there's something I noticed but forgot to mention earlier. If you look at https://github.com/pyOpenSci/pyosMeta/actions/runs/8067048660 (specifically, the summary page, not inside the job, below the graph view), there are annotations. They contain 2 deprecation warnings that come from actions/checkout@v3, actions/setup-python@v2 — these action versions are old and may stop working at some point so it is a good idea to upgrade them sooner rather than later.

I also recommend making use of environments since they allow you to pause the workflow pre-publish and wait for a human to click the "approve" button (this can be configured to be required in the repo settings). Plus, setting a versioned PyPI URL makes it appear in a few places in the UI (it's rendered like this, on the job box: https://github.com/ansible-community/ansible-build-data/actions/runs/8067689405; and on the deployments page: https://github.com/ansible-community/ansible-build-data/deployments/pypi), which is a nice bonus.

Finally, there's a way to sign releases with Sigstore that's showcased @ https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#signing-the-distribution-packages (might need adapting to take your trigger into account). This is optional but with the support for uploading GPG signatures removed from the PyPI, it's probably going to be the future replacement so I decided to include a configuration example that also relies on OIDC but doesn't need much configuration.

[lwasser]

@all-contributors please add @webknjaz for code, review

@webknjaz thank you so much for this info. i do plan to carve out time to work on this!! but for now i'm adding you as a contributor as you've already helped me so much on this!

[allcontributors]

@lwasser

I've put up a pull request to add @webknjaz! 🎉

I couldn't determine any contributions to add, did you specify any contributions?
Please make sure to use valid contribution names.

I couldn't determine any contributions to add, did you specify any contributions?
Please make sure to use valid contribution names.

[webknjaz]

@lwasser thank you!

[lwasser]

Ok!! i've opened this pr #145 to modify our ci build to

✅ separate build from publish to enhance security!
✅ use environments in both sections. i'm still a bit fuzzy on exactly what the environment item does in the build but will read some more about it tonight!

i have NOT yet gone down the sigstore route but will open a new issue about implementing this as well. maybe in a sprint later!

[lwasser]

it worked!!

we now have our first release candidate released via our shiny new/cleaned up github action!! this will actually be great - it will fix all of the issues with test pypi (mostly related to dependencies) and simplify the ci build as well. 🚀 @webknjaz thank you AGAIN for all of the help with this. i'll dig into sigstore later. i have a friend who works there (well i think he still works there) so i think he will be excited if we use sigstore to sign our releases!