Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP621 and hatch scripts #202

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

PEP621 and hatch scripts #202

wants to merge 10 commits into from

Conversation

LecrisUT
Copy link
Contributor

@LecrisUT LecrisUT commented Aug 9, 2023

Here are various modernizaitons to the pyproject

  • Switched to PEP621
  • Switched to src-layout structure. Dropped as people are opposed to it
  • Changed the build backend to hatchling
  • Added some hatch environments and scripts to align with tmt a bit
  • Switched the bin/fmf file to a script entry-point
  • Functional change: Switched to dynamic version
  • Simplified the spec file
  • Removed .travis.yaml Small cleanups #237
  • python-coveralls dependency was removed. Not sure where that one was used
  • Switch PyPI publishing to trusted-publishing. This eliminates the use of the secret token. Closes Missing sdist in PyPI #224

@lukaszachy
Copy link
Collaborator

We need to be able to build on epel-9 which fails now (very likely contains too old macros/utils).

@psss WHat about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

@lukaszachy
Copy link
Collaborator

Makefile contains useful stuff as 'make (s)rpm', 'make clean'. IMO we need leave it (it can call hatch commands though).

@lukaszachy
Copy link
Collaborator

Switched to src-layout structure.

We moved AWAY from it in the past. Easier to use libraries locally when python path works out of the box.

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Aug 9, 2023

We need to be able to build on epel-9 which fails now (very likely contains too old macros/utils).

Indeed, I was just about to look into that. I was debugging locally and I hit an issue with it just now about dynamic version. It's weird that the same setup is used in packit. That needs a bit of debugging.

Makefile contains useful stuff as 'make (s)rpm', 'make clean'. IMO we need leave it (it can call hatch commands though).

Sure thing. I would still prefer the CI, packit and spec to use the direct commands though. Fine with that?

We moved AWAY from it in the past. Easier to use libraries locally when python path works out of the box.

Roger. It is a dev preference in the end

@LecrisUT LecrisUT force-pushed the pep621 branch 3 times, most recently from 059f6d9 to 4d79f9c Compare August 9, 2023 18:21
@LecrisUT
Copy link
Contributor Author

LecrisUT commented Aug 9, 2023

I am not sure why Rhel9 build is failing. I can't seem to debug it to check if the tmp folder includes .git_archival.txt. Any ideas on how to debug that?

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Aug 9, 2023

Seems like the error message in the JsonSchemaValidator is different on the other distros:

[31mFailed to load step data for DiscoverFmfStepData: Field 'DiscoverFmfStepData:ref' must be a string, 'int' found.�[0m
:: [ 18:35:04 ] :: [   FAIL   ] :: File '/var/tmp/rlRun_LOG.VEO4lNoT' should contain 'Failed to load step data for DiscoverFmfStepData: Field 'DiscoverFmfStepData:ref' must be unset or a string, 'int' found.' 

(note the lack of must be unset or...

And similar with tmt errors reported.

@psss
Copy link
Collaborator

psss commented Aug 10, 2023

@psss What about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

Yes, I'd say we should keep fmf and tmt aligned, let's stop providing updates for el-8.

@LecrisUT
Copy link
Contributor Author

Ok, this should be ok for this PR. @psss can someone look into the testing-farm failures? They are in the integration test and it should just need a more relaxed check of the fail message.

@psss
Copy link
Collaborator

psss commented Aug 10, 2023

@psss can someone look into the testing-farm failures? They are in the integration test and it should just need a more relaxed check of the fail message.

The testing farm failures were most probably caused by the fresh tmt not being in the stable repo. Should be good now.

@psss
Copy link
Collaborator

psss commented Aug 10, 2023

/packit test

@LecrisUT LecrisUT mentioned this pull request Aug 10, 2023
@LecrisUT LecrisUT force-pushed the pep621 branch 3 times, most recently from 8ccf9b6 to 8ca3f85 Compare August 11, 2023 09:41
@LecrisUT
Copy link
Contributor Author

LecrisUT commented Aug 11, 2023

@lukaszachy , @psss Ok, with this Epel 9 is fixed. Thanks to Nikola from packit for debugging the issue. I've also expanded the github tests, can someone trigger them? I've also enabled codecov, but on my project it did not work without a secret token. Maybe that needs to be setup here as well?

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Aug 11, 2023

/packit test

@LecrisUT
Copy link
Contributor Author

@lukaszachy, @psss, @happz Review for this one please?

@psss
Copy link
Collaborator

psss commented Sep 19, 2023

@psss What about epel-8 though, can we stop providing updates as we did with tmt? epel-8 has even older macros ;)

Yes, I'd say we should keep fmf and tmt aligned, let's stop providing updates for el-8.

Filed #212 to drop support for el8.

@psss
Copy link
Collaborator

psss commented Sep 19, 2023

@lukaszachy, @psss, @happz Review for this one please?

Sorry, had a bunch of stuff on the plate so I did no get to this. Will have a look, hopefully this week. Sorry for the delay.

@psss psss added this to the 1.4 milestone Sep 29, 2023
@psss
Copy link
Collaborator

psss commented Sep 29, 2023

Let's finish this as one of the first things for the 1.4 release.

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Sep 29, 2023

Oh, I need to rebase (after #215), need anything else than that?

@psss
Copy link
Collaborator

psss commented Oct 2, 2023

Yes, please rebase. No other changes needed right now (at least from the quick look through the changes).

@LecrisUT
Copy link
Contributor Author

LecrisUT commented Jun 5, 2024

@psss @lukaszachy @martinhoyer @happz What do you think of getting this in for 1.4, now that there has been quite some simplification to this PR. This PR is decoupled from most changes that require further discussion (e.g. #241), see the top-level comment for all changes in this PR. Right now it should not have any functional changes, other than the dynamic versioning, but @martinhoyer can you double-check it?

The current status is:

  • 1 unresolved discussion about tests optional dependency. @martinhoyer can we move this for another PR issue?
  • setting up trusted publishing

Addressing #224 (40023ee) would be very useful before pushing and update, maybe I can cherry-pick that one out if needed.

@martinhoyer
Copy link
Contributor

@LecrisUT Apologies, I'm afk for the rest of the week. Fwiw, I think it's in great state, but it might be a good idea to not rush it into 1.4 and instead maybe make 1.5 a dedicated release just for this? (thinking aloud)

@psss
Copy link
Collaborator

psss commented Jun 5, 2024

Thanks for putting this all together and for the nice summary! Had a quick look, sounds good, but it's not that little change and (also from the tmt experience and the packaging changes) I'd expect there will be a couple of bumps on the road. So let's not block 1.4 any more to get it out finally and let's try to finish this pull request among the first ones in 1.5.

Copy link
Contributor

@martinhoyer martinhoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments. One more thing I wonder about is how to generate manpage - if it make sense to have it same as https://github.com/teemtee/tmt/blob/54dfd29ede1e3a20bad98bb6bf8c79dc0da25e54/pyproject.toml#L183

.github/workflows/release.yml Outdated Show resolved Hide resolved
.packit.yaml Outdated Show resolved Hide resolved
.packit.yaml Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
fmf.spec Outdated Show resolved Hide resolved
fmf.spec Show resolved Hide resolved
pyproject.toml Show resolved Hide resolved
@LecrisUT LecrisUT force-pushed the pep621 branch 2 times, most recently from 5f38cb8 to b3bed5f Compare August 18, 2024 21:20
Copy link
Contributor

@martinhoyer martinhoyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ready to me, except for the 'test' subpackage.

@chenrui333
Copy link

any luck with this PR?

description = "Run scripts with multiple Python versions"

[[tool.hatch.envs.test.matrix]]
python = ["3.9", "3.11", "3.12"]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason the 3.10 is not in the matrix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for alignment with tmt. Iiuc the python versions tested are those from Fedora and EPEL, but these are not much maintained, 3.13 is not there yet. We should make use of them though and then maybe we can revisit which python versions to track

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imho 3.11 could be removed. 3.9, 3.12 and .13 when available.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3.13 should already be available, but let's keep it synced to tmt first, and get it aligned with ruff and other tooling as well. Then when we make changes of the infrastructure in tmt we just open equivalents here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Missing sdist in PyPI
6 participants