Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow connection header in request #1829

Merged
merged 21 commits into from
Jan 5, 2023

Conversation

metcoder95
Copy link
Member

@metcoder95 metcoder95 commented Dec 22, 2022

@metcoder95 metcoder95 marked this pull request as draft December 22, 2022 21:11
@codecov-commenter
Copy link

codecov-commenter commented Dec 22, 2022

Codecov Report

Base: 90.12% // Head: 90.03% // Decreases project coverage by -0.08% ⚠️

Coverage data is based on head (42914fe) compared to base (f376a12).
Patch coverage: 54.54% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1829      +/-   ##
==========================================
- Coverage   90.12%   90.03%   -0.09%     
==========================================
  Files          66       66              
  Lines        5815     5813       -2     
==========================================
- Hits         5241     5234       -7     
- Misses        574      579       +5     
Impacted Files Coverage Δ
lib/core/request.js 96.55% <37.50%> (-2.85%) ⬇️
lib/client.js 97.53% <100.00%> (+<0.01%) ⬆️
lib/fetch/util.js 86.49% <0.00%> (-0.03%) ⬇️
lib/fetch/body.js 97.15% <0.00%> (+0.21%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

lib/client.js Outdated Show resolved Hide resolved
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@metcoder95 metcoder95 force-pushed the feat/allow_autoclose branch 3 times, most recently from f481f65 to deb3835 Compare December 26, 2022 20:50
@metcoder95
Copy link
Member Author

Sorry for the noise, finger issue 😅
I'll add the documentation soon, in the meanwhile is ready for review. Thanks 🙂

@metcoder95 metcoder95 marked this pull request as ready for review December 26, 2022 20:51
Copy link
Member

@ronag ronag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't it enough to have reset on the request? I don't think you need a reset option on the client.

@metcoder95
Copy link
Member Author

Why isn't it enough to have reset on the request? I don't think you need a reset option on the client.

Is it true that the Client does not need it at all and for instance, the parent implementations neither as well, my rationing was to allow the implementation side to have granular control over it.
Meaning that if they want to set it as the default behavior or just for a couple of requests, they have the chance to do so without having to keep track of appending the reset option on every request if they want to make this the default one.

e.g.

Circuit Breaker that transitions into half-open so it starts assessing the connection by sending requests with connection: close; and transitions to closed if the connection becomes stable.

But happy to revert it if everybody agrees this might be an overhead 🙂

@ronag
Copy link
Member

ronag commented Dec 27, 2022

You can set pipelining to 0 to achieve the same effect.

lib/client.js Outdated
@@ -115,6 +117,10 @@ class Client extends DispatcherBase {
throw new InvalidArgumentError('unsupported keepAlive, use pipelining=0 instead')
}

if (reset != null && typeof reset !== 'boolean') {
throw new InvalidArgumentError('invalid reset')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in order to have a consistent error message, this should be the same as the previous error message reset must be a boolean

lib/client.js Outdated
Comment on lines 1392 to 1385
if (reset) {
socket[kReset] = true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can move this to line 1388

lib/client.js Outdated
if (blocking) {
socket[kBlocking] = true
}

const shouldReset = request.reset != null ? request.reset : client[kClientResetValue]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: request.reset ?? client[kClientResetValue]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this would drop support for v12 and you'd need to remove it from the ci.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's true. I recommend doing it in a new pull request, if we are going to drop it (and I think we should)

lib/pool.js Outdated
@@ -40,6 +41,10 @@ class Pool extends PoolBase {
throw new InvalidArgumentError('invalid connections')
}

if (reset != null && reset.constructor !== Boolean) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not typeof reset !== 'boolean'?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was mostly about a way of working, but reverted it to keep in sync with previous @KhafraDev suggestion as well 👍

@metcoder95 metcoder95 force-pushed the feat/allow_autoclose branch 2 times, most recently from 6dea99e to 2a3ea3c Compare January 2, 2023 12:43
@metcoder95
Copy link
Member Author

I addressed the changes requested and reverted the extension to Client to support the reset property. Now, this is only accessible when executing any call that promotes the DispatchOptions-like object. I also added the documentation on top. 🙂

cc: @ronag @KhafraDev @anonrig

lib/client.js Outdated Show resolved Hide resolved
bodyTimeout,
reset,
throwOnError
} = options
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the reset?

Mostly to validate the reset value and carry over the setting during the request lifecycle, as it is attached to a single request.

Another option is to pass it as a third-parameter to the write function on lib/client. What are your thoughts? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just add reset to the previous list. I don't understand why you moved out options.

if (val.toLowerCase() === 'close') {
request.reset = true
}
request.headers += `${key}: ${val}\r\n`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not set the header here...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to do it in Client or where exactly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead, just switching the value on the reset will do what's needed, and just check for either one of the close or keep-alive options. The settings usually come in the keep-alive header if custom behavior is desired. Thoughts? 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just remove request.headers += ${key}: ${val}\r\n`` and make sure val is either keep-alive or close.

docs/api/Dispatcher.md Outdated Show resolved Hide resolved
lib/client.js Outdated Show resolved Hide resolved
lib/pool.js Outdated Show resolved Hide resolved
test/request.js Outdated Show resolved Hide resolved
lib/core/request.js Outdated Show resolved Hide resolved
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina mcollina merged commit 998a593 into nodejs:main Jan 5, 2023
@metcoder95 metcoder95 deleted the feat/allow_autoclose branch January 5, 2023 12:24
@@ -192,6 +192,7 @@ Returns: `Boolean` - `false` if dispatcher is busy and further dispatch calls wo
* **origin** `string | URL`
* **path** `string`
* **method** `string`
* **reset** `boolean` (optional) - Default: `false` - Indicates whether the request should attempt to create a long-living connection by sending the `connection: keep-alive` header, or close it immediately after response by sending `connection: close`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metcoder95 This is phrased in a confusing way. Documentation implies that "true" means creating long-lived connection, while code seems to indicate the opposite.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, let me fix it opening a new PR, thanks for the heads up 🙇

anonrig pushed a commit to anonrig/undici that referenced this pull request Apr 4, 2023
kodiakhq bot pushed a commit to X-oss-byte/Canary-nextjs that referenced this pull request Sep 18, 2023
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [undici](https://undici.nodejs.org) ([source](https://togithub.com/nodejs/undici)) | [`5.14.0` -> `5.19.1`](https://renovatebot.com/diffs/npm/undici/5.14.0/5.19.1) | [![age](https://developer.mend.io/api/mc/badges/age/npm/undici/5.19.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/undici/5.19.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/undici/5.14.0/5.19.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/undici/5.14.0/5.19.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

### GitHub Vulnerability Alerts

#### [CVE-2023-23936](https://togithub.com/nodejs/undici/security/advisories/GHSA-5r9g-qh6m-jxff)

### Impact

undici library does not protect `host` HTTP header from CRLF injection vulnerabilities.

### Patches

This issue was patched in Undici v5.19.1.

### Workarounds

Sanitize the `headers.host` string before passing to undici.

### References

Reported at https://hackerone.com/reports/1820955.

### Credits

Thank you to Zhipeng Zhang ([@&#8203;timon8](https://hackerone.com/timon8)) for reporting this vulnerability.

---

### Release Notes

<details>
<summary>nodejs/undici (undici)</summary>

### [`v5.19.1`](https://togithub.com/nodejs/undici/releases/tag/v5.19.1)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.19.0...v5.19.1)

#### ⚠️ Security Release ⚠️

-   [Regular Expression Denial of Service in Headers](https://togithub.com/nodejs/undici/security/advisories/GHSA-r6ch-mqf9-qc9w) with CVE-2023-24807
-   [CRLF Injection in Nodejs ‘undici’ via host](https://togithub.com/nodejs/undici/security/advisories/GHSA-5r9g-qh6m-jxff) with CVE-2023-23936

This release is part of the Node.js security release train: https://nodejs.org/en/blog/vulnerability/february-2023-security-releases/

### [`v5.19.0`](https://togithub.com/nodejs/undici/releases/tag/v5.19.0)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.18.0...v5.19.0)

#### What's Changed

-   fix(fetch): raise AbortSignal max event listeners by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1910
-   fix: content-disposition header parsing by [@&#8203;climba03003](https://togithub.com/climba03003) in [nodejs/undici#1911
-   fix: remove test by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1916
-   feat: add Headers.prototype.getSetCookie by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1915
-   fix(headers): clone getSetCookie list & add getSetCookie type by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1917
-   doc(mock): update out-of-date reply documentation by [@&#8203;p9f](https://togithub.com/p9f) in [nodejs/undici#1913
-   fix(types): add missing keepAlive params by [@&#8203;SkeLLLa](https://togithub.com/SkeLLLa) in [nodejs/undici#1918
-   Make the fetch() abort test pass locally, on Linux and Mac, Node 18/19. by [@&#8203;mcollina](https://togithub.com/mcollina) in [nodejs/undici#1927

#### New Contributors

-   [@&#8203;climba03003](https://togithub.com/climba03003) made their first contribution in [nodejs/undici#1911
-   [@&#8203;p9f](https://togithub.com/p9f) made their first contribution in [nodejs/undici#1913

**Full Changelog**: nodejs/undici@v5.18.0...v5.19.0

### [`v5.18.0`](https://togithub.com/nodejs/undici/releases/tag/v5.18.0)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.17.1...v5.18.0)

##### What's Changed

-   Add ability to set TCP keepalive by [@&#8203;xconverge](https://togithub.com/xconverge) in [nodejs/undici#1904
-   use faster timers by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1908
-   fix: ensure header value is a string by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1899

**Full Changelog**: nodejs/undici@v5.17.1...v5.18.0

### [`v5.17.1`](https://togithub.com/nodejs/undici/releases/tag/v5.17.1)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.17.0...v5.17.1)

#### What's Changed

-   fix: bad buffer slice (nodejs/undici@d2be675)

**Full Changelog**: nodejs/undici@v5.17.0...v5.17.1

### [`v5.17.0`](https://togithub.com/nodejs/undici/releases/tag/v5.17.0)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.16.0...v5.17.0)

#### What's Changed

-   fix(wpts): Blob is a global getter in >=v19.x.x by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1880
-   doc: fix anchor links dispatcher.stream by [@&#8203;RafaelGSS](https://togithub.com/RafaelGSS) in [nodejs/undici#1881
-   wpt: make runner more resilient by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1884
-   Make test pass in v19.x by [@&#8203;mcollina](https://togithub.com/mcollina) in [nodejs/undici#1879
-   Correct the type of DispatchOptions\["headers"] by [@&#8203;pan93412](https://togithub.com/pan93412) in [nodejs/undici#1896
-   perf(content-type parser): faster string collector by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1894
-   feat: expose content-type parser by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1895
-   fix(types): Update DispatchOptions type for missing "blocking" by [@&#8203;xconverge](https://togithub.com/xconverge) in [nodejs/undici#1889
-   fix(types): update error type definitions by [@&#8203;rafaelcr](https://togithub.com/rafaelcr) in [nodejs/undici#1888
-   fix: ensure connection header is a string by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1900
-   fix: throw if invalid content-type header by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1901
-   fix(fetch): use semicolon for Cookie header delimiter by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1906
-   Use FastBuffer by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1907

#### New Contributors

-   [@&#8203;pan93412](https://togithub.com/pan93412) made their first contribution in [nodejs/undici#1896
-   [@&#8203;rafaelcr](https://togithub.com/rafaelcr) made their first contribution in [nodejs/undici#1888

**Full Changelog**: nodejs/undici@v5.16.0...v5.17.0

### [`v5.16.0`](https://togithub.com/nodejs/undici/releases/tag/v5.16.0)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.15.2...v5.16.0)

#### What's Changed

-   Add feature to specify custom headers for proxies by [@&#8203;Sebmaster](https://togithub.com/Sebmaster) in [nodejs/undici#1877

#### New Contributors

-   [@&#8203;Sebmaster](https://togithub.com/Sebmaster) made their first contribution in [nodejs/undici#1877

**Full Changelog**: nodejs/undici@v5.15.2...v5.16.0

### [`v5.15.2`](https://togithub.com/nodejs/undici/compare/9d5f23177408dc16d3d4cbb8cebf463081c54e16...9457c9719029945ef9ff36b71d58557443730942)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.15.1...v5.15.2)

### [`v5.15.1`](https://togithub.com/nodejs/undici/releases/tag/v5.15.1)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.15.0...v5.15.1)

#### What's Changed

-   fix(websocket): simplify typedarray copying by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1854
-   fix: wpts on node v18.13.0+ by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1859
-   perf: allow keep alive for HEAD requests by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1858
-   fix: flaky abort test by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1863

**Full Changelog**: nodejs/undici@v5.15.0...v5.15.1

### [`v5.15.0`](https://togithub.com/nodejs/undici/releases/tag/v5.15.0)

[Compare Source](https://togithub.com/nodejs/undici/compare/v5.14.0...v5.15.0)

#### What's Changed

-   \[types] update ProxyAgent Options (timeout) by [@&#8203;sosoba](https://togithub.com/sosoba) in [nodejs/undici#1801
-   feat: implement websockets by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1795
-   feat(websocket): handle ping/pong frames & fix fragmented frames by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1809
-   docs: add basic fetch & company docs by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1810
-   make formdata body immutable and encode it only once by [@&#8203;jimmywarting](https://togithub.com/jimmywarting) in [nodejs/undici#1814
-   test: add regression test for [#&#8203;1814](https://togithub.com/nodejs/undici/issues/1814) by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1815
-   feat(websocket): only consume necessary bytes by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1812
-   websocket: use Buffer.allocUnsafe by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1817
-   build(deps-dev): bump [@&#8203;sinonjs/fake-timers](https://togithub.com/sinonjs/fake-timers) from 9.1.2 to 10.0.2 by [@&#8203;dependabot](https://togithub.com/dependabot) in [nodejs/undici#1819
-   fix(websocket): deprecation warning & 64-bit unsigned int body length by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1818
-   Use nodejs.stream.destroyed symbol by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1816
-   fetch: removal of redundant condition by [@&#8203;debadree25](https://togithub.com/debadree25) in [nodejs/undici#1821
-   fix(request): request headers array by [@&#8203;jd-carroll](https://togithub.com/jd-carroll) in [nodejs/undici#1807
-   fix(websocket): validate payload length received by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1822
-   fix(websocket): run parser in loop, instead of recursively by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1828
-   fix(fetch): weaker refs by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1824
-   websocket: add tests for opening handshake by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1831
-   websocket: add tests for constructor, close, and send by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1832
-   websocket: more test coverage by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1833
-   fix(WPTs): flaky abort test by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1835
-   wpt: add test by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1836
-   fix: don't send keep-alive if we want reset by [@&#8203;ronag](https://togithub.com/ronag) in [nodejs/undici#1846
-   fetch: update body consume to match spec by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1847
-   feat: allow connection header in request by [@&#8203;metcoder95](https://togithub.com/metcoder95) in [nodejs/undici#1829
-   feat: add cookie parsing ability by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1848
-   fix(cookie): add docs & expose in node v16 by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1849
-   fix(cookies): work with global Headers by [@&#8203;KhafraDev](https://togithub.com/KhafraDev) in [nodejs/undici#1850
-   docs(Dispatcher): adjust documentation for reset flag by [@&#8203;metcoder95](https://togithub.com/metcoder95) in [nodejs/undici#1852
-   Fix broken interceptor test by [@&#8203;mcollina](https://togithub.com/mcollina) in [nodejs/undici#1853

#### New Contributors

-   [@&#8203;sosoba](https://togithub.com/sosoba) made their first contribution in [nodejs/undici#1801
-   [@&#8203;debadree25](https://togithub.com/debadree25) made their first contribution in [nodejs/undici#1821
-   [@&#8203;jd-carroll](https://togithub.com/jd-carroll) made their first contribution in [nodejs/undici#1807

**Full Changelog**: nodejs/undici@v5.14.0...v5.15.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/sammyfilly/Canary-nextjs).
@robhogan
Copy link

robhogan commented Nov 1, 2023

Looks like this incorrectly closed #1305 - it only partly addresses it. It still doesn't appear to be possible to customise sec-fetch-mode.

@metcoder95
Copy link
Member Author

Please, feel free to open a new issue to reevaluate and see further actions 🙂

@robhogan
Copy link

robhogan commented Nov 1, 2023

Will do, but also, please don't mark issues closed in a PR unless the issue is actually fully closed - it just creates unnecessary churn and fragmentation.

@KhafraDev
Copy link
Member

setting a custom sec-fetch-mode is probably a wontfix anyways.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
8 participants