Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharing target directories #482

Closed
alexcrichton opened this issue Aug 29, 2014 · 7 comments · Fixed by #1657
Closed

Sharing target directories #482

alexcrichton opened this issue Aug 29, 2014 · 7 comments · Fixed by #1657
Labels
A-configuration Area: cargo config files and env vars P-high Priority: High

Comments

@alexcrichton
Copy link
Member

Projects like servo have many dependencies, all of which need to be tested. It's much easier to set up CI for all of the dependencies once than separately. Right now it's the case that cargo test must be run once in each dependency, causing many libraries to be built many times. It would be much nicer if the target output directories could be shared amongst dependencies (somehow).

@SimonSapin
Copy link
Contributor

#436 (or its work around) might help?

@brendanzab
Copy link
Member

As I posted on #523, could we have something like cargo [build | test | bench | doc] [PATH...], where cargo would run the specified command using the Cargo.toml located under each PATH, but using a common target directory. This would have the benefit of supporting more complex project structures (although they would require something like a Makefile to compile easily).

@brendanzab
Copy link
Member

Example of a Makefile using this feature:

all: build doc

travis: test doc
    mv target/doc doc

build:
    cargo build src/*/

update:
    cargo update src/*/

clean:
    cargo clean

test:
    cargo test src/*/     # unit tests
    cargo test tests/*/   # integration tests

test-update: update
    cargo update tests/*/

doc:
    cargo doc src/*/

bench:
    cargo bench benches/*/

bench-update: update
    cargo update benches/*/

examples:
    cargo build examples/*/

examples-update: update
    cargo update examples/*/

And the Cargo.toml:

[package]

name = "gfx"
version = "0.1.0"
authors = [
]

build = "make build"

Note the separate update-* targets. This is to ensure that dependencies are not updated for the main build when they are not needed (like [dev_dependencies.*] currently does).

@SimonSapin
Copy link
Contributor

Could cargo test run tests in "path" dependencies? There is precedent of treating those specially: #612

(Servo might still want to run other tests, but this would be a first step.)

@alexcrichton
Copy link
Member Author

Certainly! That's what I would like to implement.

@metajack
Copy link

Is this on the radar at all?

@alexcrichton
Copy link
Member Author

@metajack not currently, but I can try to flesh something out if needed.

@alexcrichton alexcrichton added the P-high Priority: High label May 18, 2015
alexcrichton added a commit to alexcrichton/cargo that referenced this issue May 28, 2015
This commit adds support to allow specifying a custom output directory to Cargo.
First, the `build.target-dir` configuration key is checked, and failing that the
`CARGO_TARGET_DIR` environment variable is checked, and failing that the root
package's directory joined with the directory name "target" is used.

There are a few caveats to switching target directories, however:

* If the target directory is in the current source tree, and the folder name is
  not called "target", then Cargo may walk the output directory when determining
  whether a tree is fresh.
* If the target directory is not called "target", then Cargo may look inside it
  currently for `Cargo.toml` files to learn about local packages.
* Concurrent usage of Cargo will still result in badness (rust-lang#354), and this is now
  exascerbated because many Cargo projects can share the same output directory.
* The top-level crate is not cached for future compilations, so if a crate is
  built into directory `foo` and then that crate is later used as a dependency,
  it will be recompiled.

The naming limitations can be overcome in time, but for now it greatly
simplifies the crawling routines and shouldn't have much of a negative impact
other than some Cargo runtimes (which can in turn be negated by following the
"target" name convention).

Closes rust-lang#482
alexcrichton added a commit to alexcrichton/cargo that referenced this issue May 28, 2015
This commit adds support to allow specifying a custom output directory to Cargo.
First, the `build.target-dir` configuration key is checked, and failing that the
`CARGO_TARGET_DIR` environment variable is checked, and failing that the root
package's directory joined with the directory name "target" is used.

There are a few caveats to switching target directories, however:

* If the target directory is in the current source tree, and the folder name is
  not called "target", then Cargo may walk the output directory when determining
  whether a tree is fresh.
* If the target directory is not called "target", then Cargo may look inside it
  currently for `Cargo.toml` files to learn about local packages.
* Concurrent usage of Cargo will still result in badness (rust-lang#354), and this is now
  exascerbated because many Cargo projects can share the same output directory.
* The top-level crate is not cached for future compilations, so if a crate is
  built into directory `foo` and then that crate is later used as a dependency,
  it will be recompiled.

The naming limitations can be overcome in time, but for now it greatly
simplifies the crawling routines and shouldn't have much of a negative impact
other than some Cargo runtimes (which can in turn be negated by following the
"target" name convention).

Closes rust-lang#482
bors added a commit that referenced this issue Jun 4, 2015
This commit adds support to allow specifying a custom output directory to Cargo.
First, the `build.target-dir` configuration key is checked, and failing that the
`CARGO_TARGET_DIR` environment variable is checked, and failing that the root
package's directory joined with the directory name "target" is used.

There are a few caveats to switching target directories, however:

* If the target directory is in the current source tree, and the folder name is
  not called "target", then Cargo may walk the output directory when determining
  whether a tree is fresh.
* If the target directory is not called "target", then Cargo may look inside it
  currently for `Cargo.toml` files to learn about local packages.
* Concurrent usage of Cargo will still result in badness (#354), and this is now
  exascerbated because many Cargo projects can share the same output directory.
* The top-level crate is not cached for future compilations, so if a crate is
  built into directory `foo` and then that crate is later used as a dependency,
  it will be recompiled.

The naming limitations can be overcome in time, but for now it greatly
simplifies the crawling routines and shouldn't have much of a negative impact
other than some Cargo runtimes (which can in turn be negated by following the
"target" name convention).

Closes #482
@bors bors closed this as completed in #1657 Jun 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-configuration Area: cargo config files and env vars P-high Priority: High
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants