url.parse now encodes components

[mojodna]

This behavior changed between Node-0.10 and 0.12/iojs and does not appear to be documented anywhere.

io.js v2.3.3:

> url.parse("http://example.com/{z}/{x}/{y}.png#foo{}")
{ protocol: 'http:',
  slashes: true,
  auth: null,
  host: 'example.com',
  port: null,
  hostname: 'example.com',
  hash: '#foo%7B%7D',
  search: null,
  query: null,
  pathname: '/%7Bz%7D/%7Bx%7D/%7By%7D.png',
  path: '/%7Bz%7D/%7Bx%7D/%7By%7D.png',
  href: 'http://example.com/%7Bz%7D/%7Bx%7D/%7By%7D.png#foo%7B%7D' }

node.js v0.10.39:

> url.parse("http://example.com/{z}/{x}/{y}.png#foo{}")
{ protocol: 'http:',
  slashes: true,
  auth: null,
  host: 'example.com',
  port: null,
  hostname: 'example.com',
  hash: '#foo{}',
  search: null,
  query: null,
  pathname: '/{z}/{x}/{y}.png',
  path: '/{z}/{x}/{y}.png',
  href: 'http://example.com/{z}/{x}/{y}.png#foo{}' }

This also means that URIs no longer roundtrip:

iojs v2.3.3

> url.format(url.parse("http://example.com/{z}/{x}/{y}.png#foo{}"))
'http://example.com/%7Bz%7D/%7Bx%7D/%7By%7D.png#foo%7B%7D'

node.js v0.10.39:

> url.format(url.parse("http://example.com/{z}/{x}/{y}.png#foo{}"))
'http://example.com/{z}/{x}/{y}.png#foo{}'

[rvagg]

This should go on to https://github.com/joyent/node/wiki/API-changes-between-v0.10-and-v0.12 but does anyone have a git ref that can explain the context around this?

[Fishrock123]

@mojodna any idea if this difference is also between 0.12 and 0.10?

[mojodna]

@Fishrock123 it is: nodejs/node-v0.x-archive#25636

[brendanashworth]

I believe we have our lucky winner/s: 17a379e, 881ef7c

found with guessing / version bisecting with v0.11.0 and v0.11.1

[Fishrock123]

Interesting. See nodejs/node-v0.x-archive#5474 and linked issues.

[brendanashworth]

@Fishrock123 hmm, seems like they put two patches up (38149bb, 17a379e) to fix the same bug and reverted the one in http (7124387), leaving url. Full backwards compatibilty?.. nahh ¯\_(ツ)_/¯

[mojodna]

The inability to cleanly round-trip is my biggest complaint about the regression, as it leads to code like this (plus, I'm probably forgetting to decode some components):

if (semver.satisfies(process.version, ">=0.11.0")) {
  // Node 0.12 changes the behavior of url.parse such that components are
  // url-encoded
  uri.hash = decodeURIComponent(uri.hash);
  uri.pathname = decodeURIComponent(uri.pathname);
  uri.path = decodeURIComponent(uri.path);
  uri.href = decodeURIComponent(uri.href);
}

(from mojodna/tilelive-http@a74871c#diff-168726dbe96b3ce427e7fedce31bb0bcR90)

[Fishrock123]

Crap we should probably actually fix this for 4.0.0.. hmmm

[Fishrock123]

cc @bnoordhuis since he reverted the http patch.

[rvagg]

@Fishrock123 put it on the 4.0.0 milestone if you feel strongly enough about it and think it needs to be done

[Fishrock123]

something needs to be done, I'm not sure if I should revert it or document it..

[mojodna]

My preference (for whatever it's worth) would be to revert it, solely on the basis of round-tripping not working.

[Fishrock123]

The argument @isaacs made for it was that it is how chrome handles parsing, which is true.

This is also on 0.12, so there is that to consider.

[rvagg]

@nodejs/tsc we need a position on this, it's been tagged for 4.0.0 but the current trajectory suggests it's not going to get in and we'll be stuck with current behaviour into LTS.

[trevnorris]

Just jumping in because of the tsc mention. IMO whichever opinion is the path of least surprise. Meaning, whatever more closely aligns with browser functionality.

[Fishrock123]

Meaning, whatever more closely aligns with browser functionality.

For posterity, what we now do is what chrome does.

[trevnorris]

Misspoke in my last post. We have been attempting to bring the URL parser to match the specification (https://url.spec.whatwg.org), and Chrome simply aligns to that.

Since it always returns an encoded string you can likewise always decode the return value. I believe this is fairly standard among all browser implementations.

[targos]

there is one difference between our implementation and Chrome though:

io.js v3.2.0

> url.parse('http://example.com/{z}/{x}/{y}.png#foo{}').hash
'#foo%7B%7D'

Chrome

new URL('http://example.com/{z}/{x}/{y}.png#foo{}').hash
"#foo{}"

[Fishrock123]

there is one difference between our implementation and Chrome though

Ummmm. Huh. I don't know enough about the URL specs to understand.

[rvagg]

We have to make a call on this asap, I'm inclined to leave it as it is and just make sure it's listed in the breaking changes. The hash encoding is an interesting one. I can understand why browsers don't encode it (there's really no need to if you're not sending it to a server), but does it make any more sense in Node? If nothing gets done in the next couple of days then this is going to be baked in to v4 through to LTS.

[jasnell]

At this point I'd rather us not make a major change this close to cutting
the release. It does need to be resolved but leaving it as is for v4.x
seems to be the best option.
On Aug 31, 2015 6:11 AM, "Rod Vagg" [email protected] wrote:

We have to make a call on this asap, I'm inclined to leave it as it is and
just make sure it's listed in the breaking changes. The hash encoding is
an interesting one. I can understand why browsers don't encode it (there's
really no need to if you're not sending it to a server), but does it make
any more sense in Node? If nothing gets done in the next couple of days
then this is going to be baked in to v4 through to LTS.

—
Reply to this email directly or view it on GitHub
#2113 (comment).

[trevnorris]

@targos Good point on the JS API. Looks to be the same in Firefox as well.

Anyone know of any API's that accept a URL are automatically encoded? I'm pretty sure they're aren't. If that's the case it does seem inconsistent to only encode the URL with this API.

[Fishrock123]

Fixed in 47e5cf7