Add a mechanism to inform cargo not to rebuild the crate after running build script

[upsuper]

Build script is usually used to do code generating, and sometimes, even the build script is triggered to rerun, the output generated code may not change at all. In that case, rebuilding the whole crate (and all its dependants) doesn't make sense.

There should be a mechanism for build script to inform that a rebuild is not needed from it.

[upsuper]

I guess a new special key from build script's output should work?

[alexcrichton]

Does this happen often? I'd be curious about what percentage of builds rerun a build script but don't want to rebuild the crate...

[upsuper]

I think with binding generated from C++ headers, that could happen more frequently than normal, especially when you use whitelist to include only specific types, functions, etc. in the result.

[alexcrichton]

Hm I'm not sure I quite follow. Could you elaborate more on the example? Is the library being bound not being compiled into the crate? How is bindgen detecting that no changes are made? etc.

[upsuper]

For example, if I have a C++ header:

class A { /* ... */ };
class B { /* ... */ };

For some reason, we only want to have A in the generated Rust code (because, for example, B contains something bindgen cannot convert correctly, or we ask bindgen to replace it with a different class for simplification), then the generated Rust file would be something like:

struct A { /* ... */ }

Now, whatever you do to B wouldn't affect the generated Rust code, and thus nothing actually needs to be rebuilt.

In larger projects, like Gecko, there are complicated dependencies between headers, and thus tons of headers, which may or may not affect the result, are indirectly included. But we only whitelist a small fraction of the types / functions declared in those files, and thus majority of changes to those files wouldn't really change the generated Rust code.

Bindgen library itself is not able to detect whether there is any change, but the build script can check whether the generated code is different from what's currently inside the OUT_DIR, and if not, inform Cargo that nothing actually needs to happen.

FWIW, Gecko's build system has a mechanism to avoid updating generated file if its content is not changed, so that all its dependants would not be triggered to rebuild. (Gecko's build system requires a manifest to specify which file is generated by which script or function, so it can apply a generic mechanism for all generated files. This is different than the design of Cargo's build script, though.)

[upsuper]

Oh, and, when you are debugging the build script, you may not really want the other part to build repeatedly as well.

[alexcrichton]

@upsuper thanks for the explanation! That definitely makes sense to me!

[ehuss]

#5663 includes some other use cases for this.

[DanielKeep]

I have this issue with a project where build.rs is used to automatically generate test cases from a directory of test input/output files.

Specifically, it generates tests/black_box.rs which contains a #[test] function for each tests/black_box/*.json file. In order to ensure the build script is run if I add new tests, I emit a rerun-if-changed=tests/black_box line. This works for adding/removing *.json files, but it also re-runs any time I modify any of those files. As a note: the tests/black_box.rs file depends only on the list of test files, not on their contents.

So any change to any test input causes a full 10 second re-link of the program, despite it being completely unnecessary. The only alternative is to manually maintain the tests/black_box.rs file, but I automated it because I would sometimes add a bunch of tests at once and not notice I'd forgotten one.

As for why I do all this in build.rs and not some other script: build.rs is the only "hook" available which I can guarantee I don't forget to run.

[epage]

My main concern with targeting use cases related to test generation is that means all of your dependents now need to build those dependencies and run your test generator. #12552 would be an alternative solution though that would build on the same core as build.rs

EDIT: see also #4511

[epage]

If I'm understanding this issue correctly,build.rs are always writing out their data and cargo rebuilds because the mtime has changed.

In that case, users can workaround it by having a write_if_changed function. This doesn't work as well library dumps out a lot of files at once. Maybe a tempdir and a sync_dir function? We might be able to bundle some of this in #12432.

Longer term, it seems like #6529 would be the way to go.

Personally, I'd be concerned about allowing users to say "don't rebuild, trust me". Doing that requires a certain level of checking anyways, so why not use that checking to ensure the mtime is unchanged?

[bkchr]

If I'm understanding this issue correctly,build.rs are always writing out their data and cargo rebuilds because the mtime has changed.

Not sure about the author of the issue, however in my case the build.rs script checks if the output file is different to the one that it wants to write. If it isn't different, it doesn't try to write it. Actually this is the write_if_changed function you mentioned 😅

But the problem is that the moment the build.rs was running, because it was triggered by some rerun-if-changed request, the entire crate rebuild. It would be nice that if the build.rs doesn't change any generated file, to also not rebuild the crate and every crate that follows in the dependency chain.