module: ignore resolution failures for inspect-brk #30336

arcanis · 2019-11-08T17:44:10Z

The resolution for the main entry point may fail when the resolution requires
a preloaded module to be executed first (for example when adding new extensions
to the resolution process). Silently skipping such failures allow us to defer
the resolution as long as needed without having any adverse change (since the
main entry point won't resolve anyway if it really can't be resolved at all).

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

arcanis · 2019-11-11T21:03:06Z

Ping @guybedford? (I've seen that you made most recent changes in this file)

guybedford

This entire code path is a little messy, ideally it could be more like the process.mainModule === cjsModuleInstance comparison to avoid the need to double resolve at all.

If a refactoring could be made to avoid that resolution entirely that would be much much nicer (although would likely require another function argument), but otherwise this approach makes sense to me.

nodejs-github-bot · 2019-11-12T05:25:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/26510/

joyeecheung · 2019-11-12T05:47:21Z

test/sequential/test-resolution-inspect-brk-resolver.js

+// eslint-disable-next-line no-unused-vars
+const common = require('../common');
+
+require.extensions['.ext'] = require.extensions['.js'];


This file should be in test/fixtures instead. Ditto for the .ext file.

joyeecheung · 2019-11-12T05:56:16Z

lib/internal/modules/cjs/loader.js

@@ -1091,14 +1091,17 @@ Module.prototype._compile = function(content, filename) {
    if (!resolvedArgv) {
      // We enter the repl if we're not given a filename argument.
      if (process.argv[1]) {
-        resolvedArgv = Module._resolveFilename(process.argv[1], null, false);
+        // The resolution may fail if it requires a preload script


I am confused - this only fails if the preload script tries to extend the module resolver right? This comment seems to imply that --require alone can cause failures.

Also it would be better if we could assert the condition under which we allow this to fail. Array.isArray(getOptionValue('--require')) is part of that but there should be more.

Yes for the first point.

I'm not sure I understand what you mean in the second, can you detail?

I'm not sure I understand what you mean in the second, can you detail?

I meant we should at least assert(Array.isArray(getOptionValue('--require'))) in the catch block.

I'm not convinced - this codepath (where resolveArgv is empty and _resolveFilename fails but _compile is called) is only possible for the preloaded scripts anyway (since otherwise _compile won't be called at all if _resolvedFilename fails on the real main entry script).

That said, if you still think it's a blocker, I can add it.

..is only possible for the preloaded scripts anyway

Yes, exactly, and that's why it should be an assertion - to prevent people from changing this invariant unexpectedly in the future. That happens a lot when code are being moved/copy-pasted around.

I think this comment is redundant now?

joyeecheung · 2019-11-12T06:05:38Z

If a refactoring could be made to avoid that resolution entirely that would be much much nicer (although would likely require another function argument)

At this point, it seems we could specialize the handling of --require a bit since at that point the users cannot monkey-patch module yet and there are certain exceptions (like this one) that should be considered.

arcanis · 2019-11-14T14:51:57Z

If a refactoring could be made to avoid that resolution entirely that would be much much nicer (although would likely require another function argument), but otherwise this approach makes sense to me.

I tried to avoid changing the API as much as possible to avoid potential disruption. It's a fairly small change that can't break anything that was working before, which I think is safer in this situation than a full refactoring. What do you think?

This entire code path is a little messy, ideally it could be more like the process.mainModule === cjsModuleInstance comparison to avoid the need to double resolve at all.

I think we can't do this - it would fail for this kind of setup (but who does that, arguably):

node --inspect-brk --require ./foo.js ./foo.js

In this situation you will want to break inside foo.js during the preload step, since it won't get executed as main entry point (because it's in the cache). This makes it impossible to wait for the preload list to be fully executed, or to wait for mainModule or similar to be present.

Frankly it seems a super fringe use case, but I guess that's the rational behind the way the code is written otherwise I'm not sure why the ahead-of-time resolution was needed in the first place 🤔

The resolution for the main entry point may fail when the resolution requires a preloaded module to be executed first (for example when adding new extensions to the resolution process). Silently skipping such failures allow us to defer the resolution as long as needed without having any adverse change (since the main entry point won't resolve anyway if it really can't be resolved at all).

guybedford · 2019-11-15T20:34:07Z

I think we can't do this - it would fail for this kind of setup (but who does that, arguably):

Wait, doesn't the current logic also fail for this case?

guybedford · 2019-11-15T20:34:52Z

In the esm resolver we have an isMain flag which is true through the load operation for the module that corresponds to the main entry into Node.js only - so that's the sort of logic I mean as an alternative.

arcanis · 2019-11-16T20:07:47Z

Wait, doesn't the current logic also fail for this case?

No, because the break is execute inside _compile, so even for preloaded module.

In the esm resolver we have an isMain flag which is true through the load operation for the module that corresponds to the main entry into Node.js only - so that's the sort of logic I mean as an alternative.

There's also a isMain in the cjs loader (in _load). Still, the case described above wouldn't work anymore since the isMain field is only set when executing the main entry point, so not preloaded scripts. The resolution that happens in _compile is from my understanding a way to detect whether the script being compiled will be the main script - before it actually is.

guybedford · 2019-11-16T20:20:43Z

Ah so you suspect this code path was exactly to handle this edge case in the first place, where the main itself could be in the module cache. Although it does seem very much an edge case as opposed to one to be justifying the base-level semantics on, but I guess that is where things are indeed.

arcanis · 2019-11-25T19:10:11Z

Ping?

nodejs-github-bot · 2019-11-25T19:38:29Z

CI: https://ci.nodejs.org/job/node-test-pull-request/26974/

arcanis · 2019-12-02T14:54:56Z

I'm not sure if the failing tests are related to my diff (I'd assume they aren't, since they're passing most platforms) - can I get some help?

nodejs-github-bot · 2019-12-03T04:14:00Z

CI: https://ci.nodejs.org/job/node-test-pull-request/27294/

guybedford · 2019-12-03T04:14:04Z

It doesn't look related to me, I've re-triggered the failed builds.

guybedford · 2019-12-05T03:44:29Z

The test failure here seems to be in async hooks -

test.node-api/test_make_callback/test-async-hooks-gcable
crashed (-11)

will try again and see if it works this time.

guybedford · 2019-12-05T03:45:03Z

Actually no, the Travis report is wrong - this is all green.

The resolution for the main entry point may fail when the resolution requires a preloaded module to be executed first (for example when adding new extensions to the resolution process). Silently skipping such failures allow us to defer the resolution as long as needed without having any adverse change (since the main entry point won't resolve anyway if it really can't be resolved at all). PR-URL: #30336 Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

guybedford · 2019-12-05T04:31:02Z

Landed in 1549c8e.

The resolution for the main entry point may fail when the resolution requires a preloaded module to be executed first (for example when adding new extensions to the resolution process). Silently skipping such failures allow us to defer the resolution as long as needed without having any adverse change (since the main entry point won't resolve anyway if it really can't be resolved at all). PR-URL: #30336 Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Joyee Cheung <[email protected]>

nodejs/node#30336

* chore: bump node in DEPS to v12.16.0 * Fixup asar support setup patch nodejs/node#30862 * Fixup InternalCallbackScope patch nodejs/node#30236 * Fixup GN buildfiles patch nodejs/node#30755 * Fixup low-level hooks patch nodejs/node#30466 * Fixup globals require patch nodejs/node#31643 * Fixup process stream patch nodejs/node#30862 * Fixup js2c modification patch nodejs/node#30755 * Fixup internal fs override patch nodejs/node#30610 * Fixup context-aware warn patch nodejs/node#30336 * Fixup Node.js with ltcg config nodejs/node#29388 * Fixup oaepLabel patch nodejs/node#30917 * Remove redundant ESM test patch nodejs/node#30997 * Remove redundant cli flag patch nodejs/node#30466 * Update filenames.json * Remove macro generation in GN build files nodejs/node#30755 * Fix some compilation errors upstream * Add uvwasi to deps nodejs/node#30258 * Fix BoringSSL incompatibilities * Fixup linked module patch nodejs/node#30274 * Add missing sources to GN uv build libuv/libuv#2347 * Patch some uvwasi incompatibilities * chore: bump Node.js to v12.6.1 * Remove mark_arraybuffer_as_untransferable.patch nodejs/node#30549 * Fix uvwasi build failure on win * Fixup --perf-prof cli option error * Fixup early cjs module loading * fix: initialize diagnostics properly nodejs/node#30025 * Disable new esm syntax specs nodejs/node#30219 * Fixup v8 weakref hook spec nodejs/node#29874 * Fix async context timer issue * Disable monkey-patch-main spec It relies on nodejs/node#29777, and we don't override prepareStackTrace. * Disable new tls specs nodejs/node#23188 We don't support much of TLS owing to schisms between BoringSSL and OpenSSL. Co-authored-by: Shelley Vohr <[email protected]>

arcanis force-pushed the mael/debug-resolution branch from f55cb10 to a630fe2 Compare November 8, 2019 19:13

guybedford approved these changes Nov 12, 2019

View reviewed changes

joyeecheung reviewed Nov 12, 2019

View reviewed changes

arcanis force-pushed the mael/debug-resolution branch from a630fe2 to 93a01d6 Compare November 14, 2019 14:53

arcanis added 2 commits November 19, 2019 15:05

Update loader.js

3395d15

Update loader.js

9b8f623

joyeecheung approved these changes Nov 25, 2019

View reviewed changes

guybedford closed this Dec 5, 2019

guybedford mentioned this pull request Dec 5, 2019

module: fixup lint regression #30802

Closed

2 tasks

MylesBorins mentioned this pull request Dec 13, 2019

v13.4.0 proposal #30937

Merged

arcanis mentioned this pull request Jan 4, 2020

[Question] How to launch a debugger yarnpkg/berry#558

Closed

targos mentioned this pull request Jan 15, 2020

v12.15.0 release proposal #31368

Closed

MylesBorins mentioned this pull request Feb 8, 2020

v12.16.0 proposal #31691

Merged

codebytere added a commit to electron/electron that referenced this pull request Feb 12, 2020

Fixup context-aware warn patch

151f7e3

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 12, 2020

Fixup context-aware warn patch

d388b80

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 15, 2020

Fixup context-aware warn patch

85ab906

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 18, 2020

Fixup context-aware warn patch

82aedaa

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 21, 2020

Fixup context-aware warn patch

b54a5fd

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 21, 2020

Fixup context-aware warn patch

a07228e

nodejs/node#30336

codebytere added a commit to electron/electron that referenced this pull request Feb 24, 2020

Fixup context-aware warn patch

39ef174

nodejs/node#30336

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

module: ignore resolution failures for inspect-brk #30336

module: ignore resolution failures for inspect-brk #30336

arcanis commented Nov 8, 2019 •

edited

Loading

arcanis commented Nov 11, 2019

guybedford left a comment

nodejs-github-bot commented Nov 12, 2019

joyeecheung Nov 12, 2019

joyeecheung Nov 12, 2019

arcanis Nov 14, 2019

joyeecheung Nov 18, 2019

arcanis Nov 18, 2019

joyeecheung Nov 19, 2019 •

edited

Loading

arcanis Nov 19, 2019

joyeecheung Nov 20, 2019

joyeecheung commented Nov 12, 2019

arcanis commented Nov 14, 2019

guybedford commented Nov 15, 2019

guybedford commented Nov 15, 2019

arcanis commented Nov 16, 2019

guybedford commented Nov 16, 2019

arcanis commented Nov 25, 2019

nodejs-github-bot commented Nov 25, 2019

arcanis commented Dec 2, 2019

nodejs-github-bot commented Dec 3, 2019

guybedford commented Dec 3, 2019

guybedford commented Dec 5, 2019

guybedford commented Dec 5, 2019

guybedford commented Dec 5, 2019

module: ignore resolution failures for inspect-brk #30336

module: ignore resolution failures for inspect-brk #30336

Conversation

arcanis commented Nov 8, 2019 • edited Loading

Checklist

arcanis commented Nov 11, 2019

guybedford left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Nov 12, 2019

joyeecheung Nov 12, 2019

Choose a reason for hiding this comment

joyeecheung Nov 12, 2019

Choose a reason for hiding this comment

arcanis Nov 14, 2019

Choose a reason for hiding this comment

joyeecheung Nov 18, 2019

Choose a reason for hiding this comment

arcanis Nov 18, 2019

Choose a reason for hiding this comment

joyeecheung Nov 19, 2019 • edited Loading

Choose a reason for hiding this comment

arcanis Nov 19, 2019

Choose a reason for hiding this comment

joyeecheung Nov 20, 2019

Choose a reason for hiding this comment

joyeecheung commented Nov 12, 2019

arcanis commented Nov 14, 2019

guybedford commented Nov 15, 2019

guybedford commented Nov 15, 2019

arcanis commented Nov 16, 2019

guybedford commented Nov 16, 2019

arcanis commented Nov 25, 2019

nodejs-github-bot commented Nov 25, 2019

arcanis commented Dec 2, 2019

nodejs-github-bot commented Dec 3, 2019

guybedford commented Dec 3, 2019

guybedford commented Dec 5, 2019

guybedford commented Dec 5, 2019

guybedford commented Dec 5, 2019

arcanis commented Nov 8, 2019 •

edited

Loading

joyeecheung Nov 19, 2019 •

edited

Loading