Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add optional charset option to multipart body parser #934

Merged
merged 5 commits into from
Nov 13, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/http-multipart-body-parser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ npm install --save @middy/http-multipart-body-parser
## Options

- `busboy` (object) (default `{}`): it can be used to pass extraparameters to the internal `busboy` instance at creation time. Checkout [the official documentation](https://www.npmjs.com/package/busboy#busboy-methods) for more information on the supported options.
- `charset` (string) (default `utf-8`): it can be used to change default charset.

**Note**: this middleware will buffer all the data as it is processed internally by `busboy`, so, if you are using this approach to parse significantly big volumes of data, keep in mind that all the data will be allocated in memory. This is somewhat inevitable with Lambdas (as the data is already encoded into the JSON in memory as Base64), but it's good to keep this in mind and evaluate the impact on you application.
If you really have to deal with big files, then you might also want to consider to allowing your users to [directly upload files to S3](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html)
Expand Down
3 changes: 2 additions & 1 deletion packages/http-multipart-body-parser/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ interface Options {
parts?: number
headerPairs?: number
}
}
},
charset?: string
}

export type Event = Omit<APIGatewayEvent, 'body'> & {
Expand Down
10 changes: 6 additions & 4 deletions packages/http-multipart-body-parser/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ const fieldnamePattern = /(.+)\[(.*)]$/

const defaults = {
// busboy options as per documentation: https://www.npmjs.com/package/busboy#busboy-methods
busboy: {}
busboy: {},
charset: 'utf8'
}

const httpMultipartBodyParserMiddleware = (opts = {}) => {
Expand All @@ -19,7 +20,7 @@ const httpMultipartBodyParserMiddleware = (opts = {}) => {

if (!mimePattern.test(contentType)) return

return parseMultipartData(request.event, options.busboy)
return parseMultipartData(request.event, options)
.then((multipartData) => {
// request.event.rawBody = body
request.event.body = multipartData
Expand All @@ -41,9 +42,10 @@ const httpMultipartBodyParserMiddleware = (opts = {}) => {

const parseMultipartData = (event, options) => {
const multipartData = {}
const charset = event.isBase64Encoded ? 'base64' : options.charset
// header must be lowercase (content-type)
const busboy = BusBoy({
...options,
...options.busboy,
headers: {
'content-type':
event.headers['Content-Type'] ?? event.headers['content-type']
Expand Down Expand Up @@ -90,7 +92,7 @@ const parseMultipartData = (event, options) => {
.on('close', () => resolve(multipartData))
.on('error', (e) => reject(e))

busboy.write(event.body, event.isBase64Encoded ? 'base64' : 'utf8')
busboy.write(event.body, charset)
busboy.end()
})
}
Expand Down
3 changes: 2 additions & 1 deletion packages/http-multipart-body-parser/index.test-d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ middleware = multipartBodyParser({
parts: 100,
headerPairs: 100
}
}
},
charset: 'utf-8'
})
expectType<middy.MiddlewareObj<Event>>(middleware)
1 change: 1 addition & 0 deletions website/docs/middlewares/http-multipart-body-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ npm install --save @middy/http-multipart-body-parser
## Options

- `busboy` (`object`) (optional): defaults to `{}` and it can be used to pass extraparameters to the internal `busboy` instance at creation time. Checkout [the official documentation](https://www.npmjs.com/package/busboy#busboy-methods) for more information on the supported options.
- `charset` (string) (default `utf-8`): it can be used to change default charset.

**Note**: this middleware will buffer all the data as it is processed internally by `busboy`, so, if you are using this approach to parse significantly big volumes of data, keep in mind that all the data will be allocated in memory. This is somewhat inevitable with Lambdas (as the data is already encoded into the JSON in memory as Base64), but it's good to keep this in mind and evaluate the impact on you application.
If you really have to deal with big files, then you might also want to consider to allowing your users to [directly upload files to S3](https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html)
Expand Down