Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: dump interceptor #3118

Merged
merged 34 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6d975ea
feat: add dump interceptor
metcoder95 Apr 14, 2024
5024ae1
fix(types): interceptor definitions
metcoder95 Apr 14, 2024
befe478
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 14, 2024
da70cbd
docs: update maxSize description
metcoder95 Apr 16, 2024
62a4cde
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 16, 2024
d82b51b
refactor: leftover
metcoder95 Apr 16, 2024
33b7126
docs: adjust
metcoder95 Apr 16, 2024
e3190f1
feat: abort on dumped
metcoder95 Apr 17, 2024
2fd173d
fix: return on header
metcoder95 Apr 23, 2024
85af461
refactor: apply suggestions
metcoder95 Apr 25, 2024
f5cc824
feat: extend from decorator handler
metcoder95 Apr 25, 2024
675c261
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 25, 2024
c43c80c
fix: missing handler
metcoder95 Apr 25, 2024
8548c8e
feat: add dumpOnAbort
metcoder95 Apr 25, 2024
84c0d8b
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 25, 2024
13ba7e0
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 26, 2024
bf75a5b
test: adjust test
metcoder95 Apr 26, 2024
8607411
fix: bad consumer
metcoder95 Apr 28, 2024
428a953
Merge branch 'main' into feat/dump_interceptor
metcoder95 Apr 28, 2024
8fae866
refactor: tweaks
metcoder95 May 1, 2024
4492ab7
Merge branch 'main' into feat/dump_interceptor
metcoder95 May 8, 2024
cc42e2d
test: simplify
metcoder95 May 8, 2024
c60ee90
test: disable on windows
metcoder95 May 9, 2024
38b24fb
Merge branch 'main' into feat/dump_interceptor
metcoder95 May 10, 2024
fb4ccd8
refactoor Apply suggestions from code review
metcoder95 May 10, 2024
0d67095
chore: dump on abort by default
metcoder95 May 12, 2024
e60add7
fix: typo
metcoder95 May 13, 2024
3a68023
fix: test
metcoder95 May 13, 2024
3e3295f
refactor: Apply suggestions from code review
metcoder95 May 13, 2024
2265a19
fix: lint
metcoder95 May 13, 2024
ae685cf
fix: refactor
metcoder95 May 13, 2024
85ab3d6
Merge branch 'main' into feat/dump_interceptor
metcoder95 May 13, 2024
fb5af55
fix: cleanup leftovers
metcoder95 May 15, 2024
ab67b1b
Merge branch 'main' into feat/dump_interceptor
metcoder95 May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/docs/api/Dispatcher.md
Original file line number Diff line number Diff line change
Expand Up @@ -952,6 +952,38 @@ const client = new Client("http://example.com").compose(
);
```

##### `dump`
mcollina marked this conversation as resolved.
Show resolved Hide resolved

The `dump` interceptor enables you to dump the response body from a request upon a given limit.

**Options**
- `maxSize` - The maximum size (in bytes) of the response body to dump. If the size exceeds this value then the connection will be closed. Default: `1048576`.
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved

> The `Dispatcher#options` also gets extended with the option `dumpMaxSize` which can be used to set the default `maxSize` at a request-per-request basis.

**Example - Basic Dump Interceptor**

```js
const { Client, interceptors } = require("undici");
const { dump } = interceptors;

const client = new Client("http://example.com").compose(
dump({
maxSize: 1024,
})
);

// or
client.dispatch(
{
path: "/",
method: "GET",
dumpMaxSize: 1024,
},
handler
);
```

## Instance Events

### Event: `'connect'`
Expand Down
3 changes: 2 additions & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ module.exports.RedirectHandler = RedirectHandler
module.exports.createRedirectInterceptor = createRedirectInterceptor
module.exports.interceptors = {
redirect: require('./lib/interceptor/redirect'),
retry: require('./lib/interceptor/retry')
retry: require('./lib/interceptor/retry'),
dump: require('./lib/interceptor/dump')
}

module.exports.buildConnector = buildConnector
Expand Down
109 changes: 109 additions & 0 deletions lib/interceptor/dump.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
'use strict'

const util = require('../core/util')
const { InvalidArgumentError, RequestAbortedError } = require('../core/errors')

class DumpHandler {
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
res = null
maxSize = 1024 * 1024
ronag marked this conversation as resolved.
Show resolved Hide resolved

#abort = null
#aborted = false
#size = 0
#contentLength = 0
#reason = null
#handler = null

constructor ({ maxSize }, handler) {
if (maxSize != null && (!Number.isFinite(maxSize) || maxSize < 1)) {
throw new InvalidArgumentError('maxSize must be a number greater than 0')
}

this.maxSize = maxSize ?? this.maxSize
this.#handler = handler

// Handle possible onConnect duplication
this.#handler.onConnect(reason => {
this.#aborted = true
if (this.#abort != null) {
this.#abort(reason)
} else {
this.#reason = reason
}
})
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
}

onConnect (...args) {
const [abort] = args
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
if (this.#aborted) {
abort(this.#reason)
return
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (this.#aborted) {
abort(this.#reason)
return
}
if (this.#aborted) {
abort(this.#reason)
return
}

This can't happen?


this.#abort = abort
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
}

onResponseStarted () {
this.#handler.onResponseStarted?.()
}

onBodySent () {
this.#handler.onBodySent?.()
}

onUpgrade (statusCode, headers, socket) {
this.#handler.onUpgrade?.(statusCode, headers, socket)
}
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved

// TODO: will require adjustment after new hooks are out
onHeaders (statusCode, rawHeaders, resume, statusMessage) {
ronag marked this conversation as resolved.
Show resolved Hide resolved
const headers = util.parseHeaders(rawHeaders)
const contentLength = headers['content-length']

if (contentLength > this.maxSize) {
this.#reason = new RequestAbortedError(
`Response size (${contentLength}) larger than maxSize (${this.maxSize})`
)

this.#abort(this.#reason)
return
}

this.#contentLength = contentLength
this.#handler.onHeaders(statusCode, rawHeaders, resume, statusMessage)
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
}

onError (err) {
this.#handler.onError(err)
}

onData (chunk) {
ronag marked this conversation as resolved.
Show resolved Hide resolved
this.#size = this.#size + chunk.length

if (this.#size < this.maxSize) {
return true
}

// TODO: shall we forward the rest of the data to the handler or better to abort?
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestions? I might be had it wrong here 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you must keep the data flowing from the request or error/abort it. The latter will destroy the connection. It should be a user choice, but I would error/abort the socket by default.

}

onComplete (trailers) {
this.#handler.onComplete(trailers)
}
}

function createDumpInterceptor (
{ maxSize: defaultMaxSize } = { maxSize: 1024 * 1024 }
) {
return dispatch => {
return function Intercept (opts, handler) {
const { dumpMaxSize = defaultMaxSize } = opts

const redirectHandler = new DumpHandler({ maxSize: dumpMaxSize }, handler)

return dispatch(opts, redirectHandler)
metcoder95 marked this conversation as resolved.
Show resolved Hide resolved
}
}
}

module.exports = createDumpInterceptor
Loading
Loading