Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NX build command does not recreate cached target folder. #28

Open
yurikrupnik opened this issue Jan 13, 2023 · 1 comment
Open

NX build command does not recreate cached target folder. #28

yurikrupnik opened this issue Jan 13, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@yurikrupnik
Copy link

When running:

pnpm nx run-many --target=build --parallel --max-parallel=2 --prod

It recreates the dist folder for node and Golang projects from the cache.
This does not happen with rust projects on the target folder.

@dannymcgee dannymcgee added the bug Something isn't working label May 8, 2024
@dannymcgee
Copy link
Contributor

dannymcgee commented May 8, 2024

Hey there, sorry for the long silence. >_<

I have been noticing this too lately, it's pretty annoying. Unfortunately I'm not sure if there's a good "quick fix" at the moment. You can ask Nx to cache the artifacts produced by a build command, like this:

// project.json
{
  "name": "my-cargo-project",
  "targets": {
    "build": {
      "executor": "@nxrs/cargo:build",
      "options": {},
      "configurations": {
        "dev": {
          "profile": "dev"
        },
        "release": {
          "profile": "release"
        }
      },
      "defaultConfiguration": "local",
      "outputs": ["{workspaceRoot}/target/{{ options.profile }}"]
//    ^^^^^^^^^ This is the option you need to set
    }
    // ...
  }
}

But Nx will then store a copy of your entire target/<profile> directory in .nx/cache for every unique build output it sees. Rust's target directories can easily become pretty huge, and Nx keeps those cached outputs around indefinitely, so this can very quickly explode the disk space consumed by a project.

There's another problem with this too — the target/<profile> directories include all of your workspace's compiled dependencies, so you could end up in a situation like this:

  • project-a was built a long time ago but hasn't been modified since then
  • project-b was built more recently, with updated versions of some shared crates.io dependencies
  • project-a does not depend on project-b, so that old cache for project-a is still considered valid
  • You run nx build project-a which is a cache hit for that older build
  • The newer versions of your compiled dependencies are overwritten by the older cached outputs

This is likely fixable by adding those external crates.io dependencies to the dependency graph, but it looks like that API was not exposed until Nx v19.0.0, which just released yesterday, and I'd like to let that simmer and mature a bit before I upgrade all of my projects to it. This is on my radar though — thanks for the report!

In the meantime, this can be worked around by passing --skip-nx-cache to any build commands that may require a forced update. I know this is far from ideal, but cargo does have its own caching and incremental compilation system, so it shouldn't slow your builds down too much. But it does potentially make CI/CD kind of a pain to deal with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants