Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a cron job for testing third-party users of typing_extensions #206

Merged
merged 9 commits into from
May 31, 2023

Conversation

AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented May 27, 2023

Fixes #198.

As well as the packages suggested by Jelle in #198, I also added typeguard and cattrs. Both are popular packages that use typing_extensions extensively. typeguard, as you can see by the CI on this PR, is currently broken by the same change that broke pydantic, typing_inspect and typed-argument-parse (reimplementing Literal on 3.8 and 3.9, as well as 3.7). cattrs wasn't broken by v4.6.0, but given the way they're peering into our internals, it wouldn't surprise me if we broke them at some point.

Note that while the typeguard failures show the usefulness of having a CI job like this, the typed-argument-parser tests are currently passing despite us apparently having broken them. That shows the limits of a CI job like this: we're dependent on third-party projects having reasonable test coverage, or this won't help us much.

I didn't include beartype, as their CI is currently failing on their main branch, their CI setup is somewhat complicated, and they don't fully support typing_extensions anyway.

My thinking is that if a third-party test starts failing:

  • If we're not "at fault" (e.g. the Literal change), we immediately skip the test in CI, open an issue over at the third-party project, and unskip the test as soon as the issue is fixed in the third-party project. A release shouldn't necessarily be considered blocked if a test is skipped, but we should at least make the third-party project aware of the breakage before the release, and give them a reasonable amount of time to fix it before we cut a release.
  • If it is a regression on our side, we also immediately skip the test in CI, but a release is considered blocked until the regression is fixed and the test can be unskipped.

@AlexWaygood AlexWaygood force-pushed the cron-job branch 3 times, most recently from d368b9b to 284a6f4 Compare May 27, 2023 17:13
@AlexWaygood AlexWaygood changed the title Add a cron job for testing third-party dependencies Add a cron job for testing third-party packages May 27, 2023
@AlexWaygood AlexWaygood changed the title Add a cron job for testing third-party packages Add a cron job for testing third-party users of typing_extensions May 27, 2023
@JelleZijlstra
Copy link
Member

Thanks, this is great! I'll wait for some time in case the maintainers of these libraries have some feedback.

Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This is very useful.

.github/workflows/daily.yml Outdated Show resolved Hide resolved
.github/workflows/daily.yml Outdated Show resolved Hide resolved
.github/workflows/daily.yml Outdated Show resolved Hide resolved
@AlexWaygood AlexWaygood requested a review from srittau May 30, 2023 14:35
Copy link
Collaborator

@srittau srittau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I haven't reviewed the actual implementation.

@JelleZijlstra JelleZijlstra merged commit 295705f into python:main May 31, 2023
@AlexWaygood AlexWaygood deleted the cron-job branch May 31, 2023 18:30
JelleZijlstra added a commit that referenced this pull request Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a daily action to test typing-extensions against some downstream users
3 participants