refactor: add gen-source scripts/binary to avoid pinned deps #3926
Conversation
github-actions
bot
added
arrow
Could we get a CI check to make sure things stay in sync, i.e. run gen sources and check git shows no changes? Something like this already exists, but assumes the build is running the generator |
| @@ -46,13 +46,19 @@ clap = { version = "4.1", default-features = false, features = ["std", "derive", | |||
| tracing-log = { version = "0.1", optional = true } | |||
| tracing-subscriber = { version = "0.3.1", default-features = false, features = ["ansi", "fmt"], optional = true } | |||
|
|
|||
| # Build dependencies | |||
| proc-macro2 = { version = "1.0.53", default-features = false, optional = true } | |||
There was a problem hiding this comment.
Perhaps we could split these into a separate child crate?
| @@ -46,13 +46,19 @@ clap = { version = "4.1", default-features = false, features = ["std", "derive", | |||
| tracing-log = { version = "0.1", optional = true } | |||
| tracing-subscriber = { version = "0.3.1", default-features = false, features = ["ansi", "fmt"], optional = true } | |||
|
|
|||
| # Build dependencies | |||
| proc-macro2 = { version = "1.0.53", default-features = false, optional = true } | |||
| prost-build = { version = "0.11.8", default-features = false, optional = true } | |||
There was a problem hiding this comment.
These still need to be pinned, as otherwise the vendored code can drift. I think splitting them into a child crate is the way to go here
There was a problem hiding this comment.
I think with the change we can use pinned version here with no harm. It won't affect runtime because it's gated by feature.
There was a problem hiding this comment.
I'm currently bashing out the sub-crate version, I would rather avoid adding a public feature if possible
There was a problem hiding this comment.
PR #3927 apologies for pre-empting you on this, trying to save some back and forth and really want this to make the release this afternoon 😄
There was a problem hiding this comment.
Never mind! I hope this can be released as soon as possible too because we are blocked by this issue.
Happy Friday afternoon release!
Which issue does this PR close?
Closes #3876.
Rationale for this change
As mentioned in #3876 , the pinned version of build dependencies are causing version conflicts when there is a newer version in the same project.
What changes are included in this PR?
This patch makes our source generation independent from cargo build process. It follows a great practice from https://stackoverflow.com/a/53728256 . In this patch, source code generation are moved to a separated binary
gen-source.When we want to update the source code, simply run script
./gen-source.sh. Since our previous update process is also manual, the developer experience should be similar or even better .Are there any user-facing changes?
No.