-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setup "ecosystem CI" to avoid regressions for existing users #2677
Comments
Do you imagine this running on every PR? Or simply as a nightly scheduled build? |
Probably nightly to start. (I already do some of this manually before releasing, but it's very one-off -- if we have a new rule that I feel is sufficiently complex, I'll run it over Zulip, Bokeh, or a few other projects that I have checked-out locally.) |
As recent examples of the kinds of things I'd want to catch earlier:
|
Few examples from different Python projects running a primer on pull requests and commenting the results:
This is done as a diff between main branch and the pull request so if one finds reported changes acceptable and merges them, they will no longer be reported. |
(I'd be interested to set this up) |
I'd like to setup a CI job to run Ruff over some open-source projects that use it already (and are known to have "passing" Ruff configurations). For example, we could have a job that checks out a known-to-be-passing commit of Pandas, and runs latest Ruff over the codebase.
This would be helpful to spot regressions beyond Ruff's own test suite. Similar to https://github.com/vitejs/vite-ecosystem-ci.
Lots of details to work out. E.g, sometimes we'll see new lint failures due to improvements in Ruff (new rules, but also, bug fixes that eliminate false negatives), so this may need to be advisory rather than a blocker to release. We'll also need to pin commits for whichever projects we choose to include.
If anyone is interested in owning this, I'd love help.
The text was updated successfully, but these errors were encountered: