flaky test-debugger-debug-brk

[BridgeAR]

https://ci.nodejs.org/job/node-test-commit-linux/15863/nodes=debian8-64/console

not ok 2017 sequential/test-debugger-debug-brk
  ---
  duration_ms: 0.172
  severity: fail
  stack: |-
    assert.js:72
      throw new AssertionError(obj);
      ^
    
    AssertionError [ERR_ASSERTION]: '/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/out/Release/node --inspect --debug-brk /home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/test/fixtures/empty.js' should not quit
        at ChildProcess.fail (/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/test/sequential/test-debugger-debug-brk.js:19:29)
        at ChildProcess.emit (events.js:131:13)
        at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
  ...

not ok 2044 sequential/test-inspector-async-hook-setup-at-signal
  ---
  duration_ms: 120.10
  severity: fail
  stack: |-
    timeout

not ok 2053 sequential/test-inspector-enabled
  ---
  duration_ms: 0.172
  severity: fail
  stack: |-
    Starting inspector on 127.0.0.1:9229 failed: address already in use

The two latter errors should be follow errors and there are more when looking at the output of that build.

[MylesBorins]

Happened again

https://ci.nodejs.org/job/node-test-commit-linux/15944/nodes=debian8-64/console

[apapirovski]

The most likely culprit here seems to be the V8 upgrade. This started failing shortly after that was pushed to master. The fact that it only seems to fail on one system isn't super helpful as far as debugging :(

[Trott]

sequential/test-debugger-debug-brk began failing for me locally on macOS and I couldn't figure out why for a few minutes.

Then I realized that there was a stale test process.

There is something in the Makefile that is supposed to terminate processes like that in CI, but perhaps it's not successfully terminating the process. Logging into test-rackspace-debian8-x64-1, sure enough there are week-old processes that need to be terminated. It was not being terminated because it was not originally from test-node-commit or whatever, so it isn't using the node that is being looked for (with Release in the path).

I just terminated the process. I bet this platform starts working now.

root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64# ./node test/sequential/test-debugger-debug-brk.js 
assert.js:72
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: '/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/out/Release/node --inspect --debug-brk /home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/test/fixtures/empty.js' should not quit
    at ChildProcess.fail (/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/test/sequential/test-debugger-debug-brk.js:19:29)
    at ChildProcess.emit (events.js:131:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64# /home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/out/Release/node --inspect --debug-brk /home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64/test/fixtures/empty.js
Starting inspector on 127.0.0.1:9229 failed: address already in use
root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64# ps -ef | grep 9229
root     14713 14491  0 05:28 pts/0    00:00:00 grep 9229
iojs     29220     1  0 Jan24 ?        00:00:00 /home/iojs/build/workspace/node-inspect-continuous-integration/MACHINE/debian8-64/node-v7.9.0-linux-x64/bin/node --inspect --debug-brk=9229 examples/backtrace.js
root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64# kill 29220
root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64# ps -ef | grep 9229
root     14768 14491  0 05:29 pts/0    00:00:00 grep 9229
root@test-rackspace-debian8-x64-1:/home/iojs/build/workspace/node-test-commit-linux/nodes/debian8-64#

/cc @nodejs/build

[Trott]

First green CI on that host in a long time. https://ci.nodejs.org/job/node-test-commit-linux/nodes=debian8-64/16016/console

Closing.

[apapirovski]

This is what happens when someone doesn't read far enough up the page... Ignore me @Trott. 😆

[apapirovski]

@Trott failed again today on https://ci.nodejs.org/job/node-test-commit-linux/16052/nodes=ubuntu1604-32/tapResults/ — could you have a look by any chance and see if the fix above applies here too?

[Trott]

Unless someone or something else fixed it in the intervening 8 hours, it was something else. Since subsequent inspector tests didn't also fail, that's also a sign this is a different cause.

[BridgeAR]

Happened again:

https://ci.nodejs.org/job/node-test-commit-linux-containered/2210/nodes=ubuntu1604_sharedlibs_openssl110_x64/console

[ryzokuken]

failed for #20045

23:55:31 not ok 2150 sequential/test-debugger-debug-brk
23:55:31   ---
23:55:31   duration_ms: 0.540
23:55:31   severity: fail
23:55:31   exitcode: 1
23:55:31   stack: |-
23:55:31     assert.js:77
23:55:31       throw new AssertionError(obj);
23:55:31       ^
23:55:31     
23:55:31     AssertionError [ERR_ASSERTION]: '/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/out/Release/node --inspect --debug-brk /usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/test/fixtures/empty.js' should not quit
23:55:31         at ChildProcess.fail (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/test/sequential/test-debugger-debug-brk.js:19:29)
23:55:31         at ChildProcess.emit (events.js:182:13)
23:55:31         at Process.ChildProcess._handle.onexit (internal/child_process.js:222:12)

[Trott]

OS in that previous report from @ryzokuken was freebsd10-64. (I usually include the link when I'm reporting these things even though they go stale fast because the URL contains that kind of information. The link here is https://ci.nodejs.org/job/node-test-commit-freebsd/17162/nodes=freebsd10-64/console.)

FreeBSD on CI tends more quickly than some other platforms when tests are fully safe to run in parallel. So that might be something to look at.

[Trott]

FreeBSD on CI tends more quickly than some other platforms when tests are fully safe to run in parallel. So that might be something to look at.

Oh, except that the test is already in sequential. 🙃

[maclover7]

Failing again on FreeBSD 10:

04:38:39 not ok 2199 sequential/test-debugger-debug-brk
04:38:39   ---
04:38:39   duration_ms: 1.468
04:38:39   severity: fail
04:38:39   exitcode: 1
04:38:39   stack: |-
04:38:39     assert.js:80
04:38:39       throw new AssertionError(obj);
04:38:39       ^
04:38:39     
04:38:39     AssertionError [ERR_ASSERTION]: '/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/out/Release/node --inspect --debug-brk /usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/test/fixtures/empty.js' should not quit
04:38:39         at ChildProcess.fail (/usr/home/iojs/build/workspace/node-test-commit-freebsd/nodes/freebsd10-64/test/sequential/test-debugger-debug-brk.js:19:29)
04:38:39         at ChildProcess.emit (events.js:182:13)
04:38:39         at Process.ChildProcess._handle.onexit (internal/child_process.js:237:12)
04:38:39   ...

[Trott]

https://ci.nodejs.org/job/node-test-commit-linux/22169/nodes=fedora-last-latest-x64/console

12:03:08 not ok 2303 sequential/test-debugger-debug-brk
12:03:08   ---
12:03:08   duration_ms: 0.425
12:03:08   severity: fail
12:03:08   exitcode: 1
12:03:08   stack: |-
12:03:08     assert.js:126
12:03:08       throw err;
12:03:08       ^
12:03:08     
12:03:08     AssertionError [ERR_ASSERTION]: '/home/iojs/build/workspace/node-test-commit-linux/nodes/fedora-last-latest-x64/out/Release/node --inspect --debug-brk /home/iojs/build/workspace/node-test-commit-linux/nodes/fedora-last-latest-x64/test/fixtures/empty.js' should not quit
12:03:08         at ChildProcess.fail (/home/iojs/build/workspace/node-test-commit-linux/nodes/fedora-last-latest-x64/test/sequential/test-debugger-debug-brk.js:19:29)
12:03:08         at ChildProcess.emit (events.js:182:13)
12:03:08         at Process.ChildProcess._handle.onexit (internal/child_process.js:240:12)
12:03:08   ...

[Trott]

Seems to have vanished again. Probably was fixed by fa5e097.