assert: multiple improvements #21628

BridgeAR · 2018-07-03T00:25:31Z

~~I am going to split this into smaller commits. Therefore it is WIP.~~

The following changes are in here:

Switched + / - | red / green (by request). It does seem to feel a bit more natural this way. Let's hope that most people agree with this.
Short primitives do not use the diff anymore (by request). Especially short numbers can be read well like 1 !== 2.
Improved error descriptions. It was not really clear how things worked. It should now be clear what everything is about.
Added a position indicator for single lines that fit into the tty.
~~Simple assert will now rarely bail out when checking the call site and the output got improved.~~
~~Some performance optimizations~~

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2018-07-03T00:25:32Z

@BridgeAR build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/201/pipeline

jasnell · 2018-07-10T19:37:51Z

lib/assert.js

@@ -128,8 +128,10 @@ function getBuffer(fd, assertLine) {
  let startBuffer = 0; // Start reading from that char on
  const bytesPerRead = 8192;
  const buffers = [];
+  // Use a single buffer up front that is reused until the call site is found.
+  let buffer = Buffer.allocUnsafe(bytesPerRead);


jasnell · 2018-07-10T19:39:11Z

lib/assert.js

+      nodes = parseExpressionAt(code, start);
+      if (nodes.type !== 'CallExpression') {
+        // Walk the tree to find the call expression.
+        const walk = require('internal/deps/acorn/dist/walk');


Perhaps make this a more "traditional" lazy load by having a top level let walk; then here walk = walk || require('internal/deps/acorn/dist/walk') or something similar?

devsnek · 2018-07-10T19:45:37Z

it might help to just remove the +/- and use colors. i generally try to just focus on the colors because the +/- add another dimension that confuses the heck outta me.

BridgeAR · 2018-07-12T15:13:25Z

@devsnek using only colors is not an option in case people do not have a TTY with color support. In the end the color is just the same as + / - and only a different notation for the same thing.

BridgeAR · 2018-07-14T01:54:30Z

I moved the simple assert changes out of this PR. I guess the rest is fine to be in one commit.

So PTAL.

Trott · 2018-07-14T02:30:34Z

@nodejs/tsc @nodejs/testing

targos

Diff changes tested and approved :)

Just a few questions on the messages

targos · 2018-07-14T11:38:30Z

lib/internal/assert.js

-  strictEqual: 'Input A expected to strictly equal input B',
-  notStrictEqual: 'Input A expected to strictly not equal input B'
+const kReadableOperator = {
+  deepStrictEqual: 'Expected input to be strictly deep-equal',


suggestion: "inputs" instead of "input"

I am good either way as it seems that both would work. @Trott @vsemozhetbyt do you have a preference?

For me, "inputs" seems more clear (no confusing "input is deep-equal — to what?"), but I am not a native speaker)

I agree with @targos and @vsemozhetbyt for the reason offered by @vsemozhetbyt (using "inputs" removes potential ambiguity).

targos · 2018-07-14T11:39:58Z

lib/internal/assert.js

-  notStrictEqual: 'Input A expected to strictly not equal input B'
+const kReadableOperator = {
+  deepStrictEqual: 'Expected input to be strictly deep-equal',
+  notDeepStrictEqual: 'Expected "actual" not to be strictly deep-equal',


why use "actual" here?

It is the value of the input argument "actual". I had hoped that this is actually clearer than the version before. Do you have a idea or a suggestion for to make it even better?

I'm wondering why it's not input or inputs like with the sibling method

It is a combination of this message including the whole other output. Without the further output I agree with you.

If we do a equal comparison the output would look like:

assert.deepStrictEqual([1, 2, 3], [1, 2, 2]); // Expected inputs to be strictly deep-equal: // + actual - expected // // // [ // 1, // 2, // + 3 // - 2 // ]

While a not equal diff looks like:

assert.notDeepStrictEqual([1, 2, 3], [1, 2, 3]); // Expected "actual" not be strictly deep-equal // // [ // 1, // 2, // 3 // ]

So the first part compares two different inputs (actual & expected) while the latter says that "actual" should not be equal to the afterwards printed value.

I see. Thanks for the explanation, this seems fine.

BridgeAR · 2018-07-14T15:37:30Z

Comment addressed.

CI https://ci.nodejs.org/job/node-test-pull-request/15877/

mcollina

LGTM with a nit.

mcollina · 2018-07-16T14:03:30Z

lib/internal/assert.js

+          (actual !== 0 || expected !== 0)) { // -0 === +0
+        return `${kReadableOperator[operator]}:\n\n` +
+            `${actualLines[0]} !== ${expectedLines[0]}\n`;
+      }


Can you please add a comment explaining this block of logic?

BridgeAR · 2018-07-16T15:45:04Z

CI https://ci.nodejs.org/job/node-test-pull-request/15896/

addaleax · 2018-07-27T11:00:17Z

lib/internal/assert.js

+    // If the stdout is a tty and the input length is lower than the current
+    // columns per line, add a mismatch indicator below the output.
+    } else if (process.stdout.isTTY &&
+               inputLength < process.stdout.columns) {


I’d prefer not to add a dependency on process.stdout here … there’s no guarantee that this is where the error output ends up, and it makes assert depend on something it shouldn’t depend on (or makes it more so, at least).

Can we pick a value, e.g. 80 or 100 or so?

I am aware that there is no guarantee that this is where the error output ends up. It is also the reason why I only added the indicator if it is a tty (and yes, even then the error could end up somewhere else but that is unlikely).

If I understand you correct you would actually like to have the indicator all the time? In that case I could add a default of 80 in case it is not a tty and check stdout otherwise?

That all aside: I would actually like to add a proper diffing algorithm at some point with which it would be possible to use colors to highlight the differences instead. But that is a bit more work and I got quite some pushback when it comes to improvements to assert, so I am not really sure if that is something worth trying.

1) Switched + / - and red / green in diffs. It seems like that style is more natural to most people. 2) Short primitives do not use the diff anymore. Especially short numbers can be read well like 1 !== 2. Cases that can not be displayed like that (e.g., -0 and +0) use the regular diff output. 3) Improved error descriptions. It was not always clear what the messages stood for. That should now be resolved. 4) Added a position indicator for single lines in case a tty is used and the line is shorter than the visual columns.

BridgeAR · 2018-07-27T13:42:17Z

@addaleax I added a default of 80 for all cases where stdout is not a tty.

@targos I also added a "to" to make things even clearer (it addresses #21628 (comment)).

I also moved some code up to remove a intermediate variable and to shorten the code path in that case.

CI https://ci.nodejs.org/job/node-test-pull-request/16029/

Trott

Seems good to me at a high level.

BridgeAR · 2018-07-27T22:01:29Z

I just changed two more small things:

Instead of checking stdout, it will now check stderr as it is more likely that the error ends up there.
The indicator is ignored for the first three characters.

BridgeAR · 2018-07-27T22:02:59Z

CI https://ci.nodejs.org/job/node-test-pull-request/16038/

BridgeAR · 2018-07-29T00:36:49Z

I just pushed the doc changes that were missing. The CI was green after resuming as it can be seen by the green check mark next to the commit before the docs. Since the doc changes do not need a full CI, I did not start anything else anymore.

PTAL. I think this is ready otherwise and could land.

BridgeAR · 2018-08-02T13:20:00Z

PTAL. It would be nice to get LG for the last changes.

BridgeAR · 2018-08-03T21:58:18Z

@nodejs/documentation PTAL

vsemozhetbyt

Doc changes LGTM)

vsemozhetbyt · 2018-08-03T22:21:52Z

Just a nit question. Sometimes in the error logs we have AssertionError [ERR_ASSERTION]: ... and sometimes just AssertionError: .... Is there some system in this?

BridgeAR · 2018-08-03T22:28:31Z

The line length was to long when adding the ERR_ASSERTION every time. Therefore it was some times removed.

1) Switched + / - and red / green in diffs. It seems like that style is more natural to most people. 2) Short primitives do not use the diff anymore. Especially short numbers can be read well like 1 !== 2. Cases that can not be displayed like that (e.g., -0 and +0) use the regular diff output. 3) Improved error descriptions. It was not always clear what the messages stood for. That should now be resolved. 4) Added a position indicator for single lines in case a tty is used and the line is shorter than the visual columns. 5) Color detection is now done by checking stderr instead of stdout. PR-URL: nodejs#21628 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]>

BridgeAR · 2018-08-04T08:05:38Z

Thanks a lot.

Landed in 0518b9e 🎉

1) Switched + / - and red / green in diffs. It seems like that style is more natural to most people. 2) Short primitives do not use the diff anymore. Especially short numbers can be read well like 1 !== 2. Cases that can not be displayed like that (e.g., -0 and +0) use the regular diff output. 3) Improved error descriptions. It was not always clear what the messages stood for. That should now be resolved. 4) Added a position indicator for single lines in case a tty is used and the line is shorter than the visual columns. 5) Color detection is now done by checking stderr instead of stdout. PR-URL: nodejs#21628 Reviewed-By: Michaël Zasso <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Vse Mozhet Byt <[email protected]>

BridgeAR added wip Issues and PRs that are still a work in progress. semver-major PRs that contain breaking changes and should be released in the next major version. labels Jul 3, 2018

BridgeAR requested review from mcollina, jasnell, Trott, addaleax, targos and joyeecheung July 3, 2018 00:25

BridgeAR requested review from devsnek and a team July 3, 2018 00:25

nodejs-github-bot added the assert Issues and PRs related to the assert subsystem. label Jul 3, 2018

jasnell reviewed Jul 10, 2018

View reviewed changes

BridgeAR force-pushed the improve-assert-things branch from 5a7225c to dfb54bf Compare July 13, 2018 19:20

BridgeAR removed the wip Issues and PRs that are still a work in progress. label Jul 13, 2018

BridgeAR requested a review from a team July 14, 2018 01:54

targos reviewed Jul 14, 2018

View reviewed changes

targos approved these changes Jul 14, 2018

View reviewed changes

BridgeAR requested a review from a team July 16, 2018 09:30

mcollina approved these changes Jul 16, 2018

View reviewed changes

BridgeAR requested a review from a team July 16, 2018 15:46

BridgeAR mentioned this pull request Jul 18, 2018

assert: reverse diff direction #20929

Closed

4 tasks

BridgeAR requested a review from jasnell July 27, 2018 01:45

addaleax approved these changes Jul 27, 2018

View reviewed changes

BridgeAR added 4 commits July 27, 2018 13:50

fixup: address comment

671a3cc

fixup: address comment

511f8a1

fixup

e789901

BridgeAR force-pushed the improve-assert-things branch from e0dec36 to e789901 Compare July 27, 2018 13:19

jasnell approved these changes Jul 27, 2018

View reviewed changes

Trott reviewed Jul 27, 2018

View reviewed changes

fixup: use stderr instead of stdout

4587e5d

fixup: improve docs

c00e63d

BridgeAR requested a review from vsemozhetbyt August 3, 2018 21:58

vsemozhetbyt approved these changes Aug 3, 2018

View reviewed changes

BridgeAR closed this Aug 4, 2018

oyyd mentioned this pull request Sep 26, 2018

[v10.x] Backport assert PRs #23095

Closed

2 tasks

BridgeAR deleted the improve-assert-things branch January 20, 2020 11:34

assert: multiple improvements #21628

assert: multiple improvements #21628

Conversation

BridgeAR commented Jul 3, 2018 • edited Loading

Checklist

nodejs-github-bot commented Jul 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devsnek commented Jul 10, 2018

BridgeAR commented Jul 12, 2018

BridgeAR commented Jul 14, 2018

Trott commented Jul 14, 2018

targos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented Jul 14, 2018

mcollina left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented Jul 16, 2018

Choose a reason for hiding this comment

BridgeAR Jul 27, 2018 • edited Loading

Choose a reason for hiding this comment

BridgeAR commented Jul 27, 2018 • edited Loading

Trott left a comment

Choose a reason for hiding this comment

BridgeAR commented Jul 27, 2018

BridgeAR commented Jul 27, 2018

BridgeAR commented Jul 29, 2018

BridgeAR commented Aug 2, 2018

BridgeAR commented Aug 3, 2018

vsemozhetbyt left a comment

Choose a reason for hiding this comment

vsemozhetbyt commented Aug 3, 2018

BridgeAR commented Aug 3, 2018

BridgeAR commented Aug 4, 2018

BridgeAR commented Jul 3, 2018 •

edited

Loading

BridgeAR Jul 27, 2018 •

edited

Loading

BridgeAR commented Jul 27, 2018 •

edited

Loading