feat: npm workspace and better Deno workspace support #24334

dsherret · 2024-06-25T20:49:56Z

Adds much better support for the unstable Deno workspaces as well as support for npm workspaces. npm workspaces is still lacking in that we only install packages into the root node_modules folder. We'll make it smarter over time in order for it to figure out when to add node_modules folders within packages.

A root deno.json file can now specify workspace members like so:

{
  "workspace": ["./member-a", "./member-b"]
}

...these members are folders which can contain a deno.json, package.json, or both. They may or may not be a package. The root deno.json can also be a package itself by specifying "name", "version", and "exports". There are a lot of details here that will be laid out in documentation soon.

This includes a breaking change in config file resolution where we stop searching for config files on the first found package.json unless it's in a workspace. For the previous behaviour, the root deno.json needs to be updated to be a workspace by adding "workspace": ["./path-to-pkg-json-folder-goes-here"]. See details in denoland/deno_config#66

Note: this is missing wildcard support for npm workspaces (follow #24420 for updates)

Part of #22942

Closes #24340
Closes #24159
Closes #24161
Closes #22020
Closes #18546
Closes #16106
Closes #24160

- Get all node_modules in dir for deno compile w/ byonm - Maybe fix Windows CI, but probably won't fix

bartlomieju · 2024-07-03T22:13:58Z

cli/args/flags.rs

-        if *DENO_FUTURE {
-          Some(current_dir.to_path_buf())
-        } else {
-          None


What are implications of removing this condition?

deno_config just always resolved the Workspace now. That conditionally resolves the package.json based on whether package_json_discovery in the discover options is true. Right now, it only doesn't discover the package.json if --no-npm or --no-config is provided. I'm not sure it's worth making it more granular than that because all the perf sensitive commands need it anyway.

dsherret · 2024-07-03T22:19:34Z

cli/args/flags.rs

+      | Compile(CompileFlags {
+        source_file: script,
+        ..
+      }) => {


Note this change. We should be discovering the workspace based on the script provided to deno compile. Previously we were always using the cwd.

cli/args/mod.rs

bartlomieju · 2024-07-03T22:19:10Z

cli/args/mod.rs

+            // todo(dsherret): this should be false for nodeModulesDir: true
+            pkg_json_dep_resolution: if self.use_byonm() {
+              PackageJsonDepResolution::Disabled


Do you want to address that before landing?

No, that's a big change.

bartlomieju · 2024-07-03T22:20:59Z

cli/args/mod.rs

-        .as_ref()
-        .map(|c| c.has_unstable("sloppy-imports"))
-        .unwrap_or(false)
+      || self.workspace.has_unstable("sloppy-imports")


I must have missed it, but how are unstable features enabled in the workspace setting? I hope it's only at the top level?

Yeah, it's only at the top level. Otherwise it surfaces diagnostics in deno_config.

cli/args/package_json.rs

cli/standalone/binary.rs

cli/tools/registry/unfurl.rs

bartlomieju · 2024-07-03T22:29:42Z

cli/tools/task.rs

-    .unwrap_or_default();
+  let start_ctx = cli_options.workspace.resolve_start_ctx();
+  if !start_ctx.has_deno_or_pkg_json() {
+    bail!("deno task couldn't find deno.json(c). See https://deno.land/manual@v{}/getting_started/configuration_file", env!("CARGO_PKG_VERSION"))


Should it mention that we couldn't find package.json either?

I think it's better to encourage people to use a deno.json (this message is the same as before)

bartlomieju · 2024-07-03T22:31:23Z

cli/tools/vendor/mod.rs

@@ -48,10 +48,17 @@ pub async fn vendor(
  validate_options(&mut cli_options, &output_dir)?;
  let factory = CliFactory::from_cli_options(Arc::new(cli_options));
  let cli_options = factory.cli_options();
+  if cli_options.workspace.config_folders().len() > 1 {
+    bail!("deno vendor is not supported in a workspace. Set `\"vendor\": true` in the workspace deno.json file instead");
+  }


bartlomieju · 2024-07-03T22:31:54Z

cli/util/file_watcher.rs

+    if paths.is_empty() {
+      return Ok(());
+    }


What prompted this change?

I moved it in here instead of being done only sometimes conditionally in the higher level code.

…o_config's test suite

tungv · 2024-07-04T15:27:14Z

thanks: #24430

Adds much better support for the unstable Deno workspaces as well as support for npm workspaces. npm workspaces is still lacking in that we only install packages into the root node_modules folder. We'll make it smarter over time in order for it to figure out when to add node_modules folders within packages. This includes a breaking change in config file resolution where we stop searching for config files on the first found package.json unless it's in a workspace. For the previous behaviour, the root deno.json needs to be updated to be a workspace by adding `"workspace": ["./path-to-pkg-json-folder-goes-here"]`. See details in denoland/deno_config#66 Closes denoland#24340 Closes denoland#24159 Closes denoland#24161 Closes denoland#22020 Closes denoland#18546 Closes denoland#16106 Closes denoland#24160

dsherret added 30 commits June 17, 2024 19:27

Start on workspace refactor.

3660270

Some work on tasks

2c591ef

Switching more to workspace

81dc4a8

More.

271df70

Remove cli/args/package_json.rs

b155d9a

some work on deno compile

c1d52f8

tasks

c8dbbb1

update publishing

04fd3c6

Adding fmting

03b30f4

Merge branch 'main' into workspace_start

1e8b196

some work on linting

cb6b545

finish lint

5e9fd3f

Some work on lsp and installing from all the packages

4fe5441

separate out test options from workspace test options

19ad5c4

finish up test

6eed364

Add bench

c04afc7

compiling

49b1a23

handle base properly

76dcc18

fix slow types linting

4205667

Use SharedLocal and fix some vendor tests.

2033d89

Start making import map optional

297d191

Start investigating @types issue

8583651

Fix. Wasn't thinking.

df4e20e

fix info

a7b5857

fix watcher tests

c5df354

working npm tests

77c0977

some import specifier fixes

d38c216

Fix some lsp issues.

ffa9bb1

Fix test

e887d93

We understand the npm workspace now

64c20ac

dsherret added 3 commits July 2, 2024 22:23

Merge branch 'main' into workspace_start

43f1f5f

update tests

372ceed

fix copyright

0c98628

dsherret changed the title ~~feat: better workspace support~~ feat: npm workspace and better Deno workspace support Jul 3, 2024

dsherret added 3 commits July 3, 2024 13:03

add tests for vendor

51bed91

update test for pkg with no deps in lockfile in workspace

8ac81df

Merge branch 'main' into workspace_start

2a957e0

dsherret marked this pull request as ready for review July 3, 2024 17:09

- Fix private registries not working with auto-install in deno compile

650a43d

- Get all node_modules in dir for deno compile w/ byonm - Maybe fix Windows CI, but probably won't fix

bartlomieju reviewed Jul 3, 2024

View reviewed changes

dsherret commented Jul 3, 2024

View reviewed changes

bartlomieju reviewed Jul 3, 2024

View reviewed changes

dsherret added 2 commits July 3, 2024 19:00

maybe fix windows tests

ae7ebec

Updates from review

4d0d459

dsherret requested a review from bartlomieju July 3, 2024 23:48

bartlomieju approved these changes Jul 3, 2024

View reviewed changes

Merge branch 'main' into workspace_start

21673a3

dsherret enabled auto-merge (squash) July 3, 2024 23:54

Upgrade deno_unsync. Remove fatal diagnostic test because it's in den…

b282f75

…o_config's test suite

dsherret mentioned this pull request Jul 4, 2024

Support wildcards in npm workspaces #24420

Closed

dsherret merged commit 147411e into denoland:main Jul 4, 2024
17 checks passed

dsherret deleted the workspace_start branch July 4, 2024 01:19

This comment was marked as off-topic.

Sign in to view

lishaduck mentioned this pull request Jul 9, 2024

Support Deno. How to get about implementing platform-deno? Effect-TS/effect#1806

Open

dsherret mentioned this pull request Jul 9, 2024

workspace: specifier not supported for package.json dependency #18192

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: npm workspace and better Deno workspace support #24334

feat: npm workspace and better Deno workspace support #24334

dsherret commented Jun 25, 2024 •

edited

Loading

bartlomieju Jul 3, 2024

dsherret Jul 3, 2024

dsherret Jul 3, 2024 •

edited

Loading

bartlomieju Jul 3, 2024

dsherret Jul 3, 2024

bartlomieju Jul 3, 2024

dsherret Jul 3, 2024

bartlomieju Jul 3, 2024

dsherret Jul 3, 2024

bartlomieju Jul 3, 2024

bartlomieju Jul 3, 2024

dsherret Jul 3, 2024

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

tungv commented Jul 4, 2024

feat: npm workspace and better Deno workspace support #24334

feat: npm workspace and better Deno workspace support #24334

Conversation

dsherret commented Jun 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dsherret Jul 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

tungv commented Jul 4, 2024

dsherret commented Jun 25, 2024 •

edited

Loading

dsherret Jul 3, 2024 •

edited

Loading