need a way to run tests on dependencies

[metajack]

Currently cargo test only runs tests for the top level package. I need some way to run tests for some of the dependencies. In servo these would be at least the path dependencies (which we have to work around lack of multiple [lib] sections). But it would also be nice to run them for arbitrary dependencies.

[alexcrichton]

cc #482, one possible strategy

[brendanzab]

This also affects gfx-rs. Currently we have a Makefile that cds into the subdirectories, but that causes a huge amount of redundant recompilations of dependencies.

[brendanzab]

Could we have the ability to have something like:

cargo [build | test | bench | doc] [PATH...]

Then cargo would run the specified command using the Cargo.toml located under each PATH, but using a common target directory.

[ghost]

@alexcrichton What do you think about supporting running tests recursively, that is, for the package specified and all its dependencies? Say, cargo test --recursive? Should I open a separate issue if that sounds sensible?

[alexcrichton]

Hm, perhaps we could start of with accepting multiple -p flags? It's hard to say where --recursive should stop as you probably don't want to run tests on your entire dependency graph.

[jsgf]

I do!

[fschutt]

@alexcrichton well, I'd be for a --recursive function. I mean, if you don't want to test the entire dependency graph, you don't need to call it. It would be handy for larger releases, where you want to make sure nothing goes wrong.

[joe-askattest]

I believe there is a real problem here is when cargo is building dependencies. If people are suggesting --recursive as in all tests everywhere, then I don't think it's quite right. Why? If you have a monorepo style, then you can do something like cargo test --all. If you have split repos, then you can have the repo test before commit or publish. I just feel there are solutions to that heavy handed approach.

But there is a problem. That's with building local dependencies whilst developing. You run cargo test, and a local dependency has changed. So it builds it. It does not run it's tests. For me, this is the core issue. If cargo test causes a dependency to be built, then it should run it's tests too. After all there is no build for that dependency where the tests have been run.

I don't want to go with the cargo test --all solution I gave above, because if tests were run on build then I just don't need to. If the build hasn't changed, then it would be a waste of time.

So I'd love to see a --recursive that only ran the tests on dependencies that it's building, and skips the rest. Perhaps a --recursive-all for people who really did want it to test everything everywhere, regardless.

Tbh many other build tools allow you to have test as a part of the build. So if the build fails, the test fails. If you build a dependency, then it's tests are run too. Build and test go hand in hand. They don't in cargo. So perhaps there is a better third solution here.

[korczis]

My two cents.

cargo test --recursive 

Which equals to:

cargo test --recursive --depth=1

And optionally one of these for unlimited depth test:

cargo test --recursive --depth=-1
cargo test --recursive --depth=0
cargo test --recursive --depth=full
cargo test --recursive --depth=unlimited