Version 4

[willfarrell]

PR: #908

Planned

• Drop Node v14, Add support fro Node v18
  • Update engines to "node": ">=16"
  • Update github workflows node-version
  • Update swc to use es2021 (https://github.com/tsconfig/bases/tree/main/bases)
• event-normalizer: review for any new events from AWS docs and event list
• http-security-headers: review against helmet, MDN, ASVS
• AWS SDK v3 Full ESM Support #584
  • Update packages
  • Update tests
  • Update docs
  • TypeScript
  • Update to use async iterators for pagination in ssm, secrets-manager, etc (https://aws.amazon.com/blogs/developer/pagination-using-async-iterators-in-modular-aws-sdk-for-javascript/)
• http-urlencode-body-parser throw 422 when unable to parse
• http-content-encoding pull stream piping into new middleware stream-transform-response
• http-router parse path params. no, can be done in v3
• http-router REST and HTTP route paths should be the same (http-router needs stage name in path for routes declaration #873). No, see issue.
• validator export compileSchema and require it be passed in to allow for better tree-shaking
• Allow top level await within middy core when adding middleware .use(await middleware())
• Update bundling docs

>= 15

• use replaceAll in event-normalizer [skip - slower than current]

remove polyfills

• core
  • Remove AbortControler polyfill
  • remove timers/promises polyfill
• http-content-encoding: remove stream/promises polyfill
• util update tests to use timers/promises for delay

>= 16

Error {cause} >= 16.9

• util update createError to use class HttpError extends Error
• http-json-body-parser: use proper new Error(message, {cause}) notation
• http-multipart-body-parser: use proper new Error(message, {cause}) notation
• util: use proper new Error(message, {cause}) notation
• util: use proper super(message, {cause}) notation in HttpError
• validator: use proper new Error(message, {cause}) notation

deprecate

• deprecate cloudwatch-metrics for @aws-lambda-powertools/metrics (uses EMF)? @fredericbarthelet
• validator: deprecate inputSchema and outputSchema options
• event.rawBody in http-json-body-parser

[vicary]

@willfarrell Does .handler() means it inherits all the event types from the .use() before it and I no longer have to chain up middleware events in my resolver?

My team is currently repeating the following style in hundreds of resolvers. Every night, I dream of making all the middleware events private and self-contained.

const resolver = async (event: FooEvent & BarEvent & BazEvent) => { ... };
export const handler = middy(resolver).use(...).use(...).use(...);

[willfarrell]

@vicary this will be in v3. What happens is whatever is passed into .handler() replace the first arguments to middy() (likely nothing), then end the chaining.

It means you can do export const handler = middy().use(...).use(...).handler(resolver) instead.

Which mean it's easier to split up repeated middlewares:

// middlewares.js
export default middy().use(...).use(...)

// lambda.js
import wrapper from 'middleware.js'

const resolver = async (event: FooEvent & BarEvent & BazEvent) => { ... };
export const handler = wrapper.use(validator({...})).handler(resolver)

Technically you can do this in v2.x but you have to wrap it in a function, which isn't as easy to use.

[vicary]

@willfarrell Not sure if I get it.

I specifically wants to remove the necessity to chain FooEvent & BarEvent & BazEvent from my resolver() because they are events in the middlewares which add something to the end result. Can I do the following in v3?

import type { AppSyncResolverEvent } from "aws-lambda";

export const handler = middy()
   .use(foo()) // does TEvent & FooEvent inside
   .use(bar()) // does TEvent & BarEvent inside
   .use(baz()) // does TEvent & BazEvent inside
   .handler<AppSyncResolverEvent>((event) => {
      // Can I automatically get all the followings?
      event.foo; // type FooEvent = { foo: number; [key: string]: any };
      event.bar; // type BarEvent = { bar: string; [key: string]; any };
      event.baz; // type BazEvent = { baz: PartialDeep<BaseEntity>; [key: string]: any };
      event.arguments; // AppSyncResolverEvent
   });

[willfarrell]

middy is just a light framework to order repeated operation before/after a handler execution. What is in those middlewares is completely up to you to fit your needs.

All of the following are the same:

export const handler = middy().use(foo()).use(bar()).use(baz()).handler(...)
export const handler = middy().use([foo(), bar(), baz()]).handler(...)
export const handler = middy().use(foobarbaz()).handler(...)

import middlewares from '...'
export const handler = middlewares.handler(...)

Hope that help in your planning on how to best use middy.

Your above example looks like it will run just fine in v3. If you move the handler into middy(...) it can run in v2.

[vicary]

@willfarrell Main problem is TypeScript implementation does not carry/merge event types down the chain, it should be possible in the DT for .use() without too much effort.

My second code won't pass type checking because the handler type has no knowledge about FooEvent, BarEvent and BazEvent, it only knows the AppSyncResolverEvent I specifically give it.

I don't want to manually import all the intermediate middleware types in my handler (event: A & B & C & D & E ........) => {}, I want them to automatically be carried down when I call .use().

[willfarrell]

Ooohhhh, this is a typescript questions. Sorry. I personally don't use typescript so I can't advise how to best use middy with typescript. I know you're not the first to run into this issue, I'm unsure how other have resolved this. I'm open to feedback on how to this can be improved within middy, if even possible.

[vicary]

It's mildly troubling, but somehow breaking how types work. Not sure if I have the time and energy to whip up a PR for the already launching v3, so I am speaking in this v4 thread.

[willfarrell]

v3 is only in early alpha, it wont release until after node v16 is supported in lambda (likely Apr). So lots of time to discuss any breaking changes, especially for TypeScript. Otherwise they'll have to wait a year.

[vicary]

That's great! I'll see what I can do next month.

[dbartholomae]

@vicary You might want to take a look at https://github.com/dbartholomae/lambda-middleware which I created out of a discussion around how to better use TypeScript types in middy. It attacks the problem by using functional composition, so middlewares can define their input and output types.

[devlsh]

@dbartholomae Looks great - I recently recreated the middleware pattern for TypeScript too (https://github.com/evilkiwi/lamware), but went a different route with chaining .use() to have pass-through typing for a custom state that the handler gets given

[bilalq]

I love the documentation and ideas behind middy, but the TypeScript developer experience is broken right now.

The middleware implementations that @dbartholomae and @oyed put out look great, but now we have a problem of fragmentation. Middy has the biggest body of plugins, but TypeScript usage is dangerously unsafe. I see that the lambda-middleware package has a helper for adapting middy-based ones, but it's still a hassle to have to manually type that all out. It'd be much nicer if the internal types of middy middlewares themselves behaved more like the other two middleware orchestrators.

On a loosely related note, it'd be great to use named exports only and drop the default exports. The benefit of this isn't limited to just TypeScript users, since VSCode intellisense completion works pretty well for JS as well. This would be a breaking change, but it should improve the developer experience a fair bit.

[willfarrell]

@bilalq Thanks for the TS feedback. None of the core maintainers know TS, so all of our defs are community generated. If someone from the TS community would like to audit and update all middleware in a PR, I'd be happy to review.

We can easily have named exports alongside the default without a breaking change if it makes that big of a difference for developers. I would like to better understand why you think this would improve the developer experience before making this change. Is there an article / blog post you can share explaining this?

[vicary]

@willfarrell I can start from what I know, if you're OK with the approach in #589 (comment).

In the same PR I'll also fix inline middlewares, because they are broken in a way that their TEvent are always any.

[willfarrell]

I look forward to reviewing the PR.

[nponeccop]

deprecate use of qs in http-urlencode-body-parser for node:querystring

Shouldn't it be new URLSearchParams? It is both W3C-standardized and supported by node.

[codyfrisch]

Ooohhhh, this is a typescript questions. Sorry. I personally don't use typescript so I can't advise how to best use middy with typescript. I know you're not the first to run into this issue, I'm unsure how other have resolved this. I'm open to feedback on how to this can be improved within middy, if even possible.

Yeah, TypeScript is important for me, and the fact its poorly implemented here is why I have chucked the last two days work trying to refactor to use Middy and am searching for a more competent solution.

[willfarrell]

@codyfrisch Sorry to hear Middy TypeScript definitions are not meeting your needs. A PR to help improve them is welcome if you'd like to contribute back to the community. I've talked with other OSS maintainers about supporting TypeScript well, nearly all are also struggling with it. This article was shared with me that I hope will shed light on this topic: https://erock.prose.sh/typescript-terrible-for-library-developers

[bilalq]

@willfarrell FWIW, the #883 improved the situation a ton. Really appreciate the work you and @vicary did to get that merged in! Honestly, at this point, I have no reservations about using Middy with TS. It's been pretty stellar.

Regarding your point about difficulty in supporting TypeScript as a library author: I agree that it is annoying if your source code itself isn't authored in TypeScript. However, I've found maintaining libraries and frameworks a lot easier when they're actually written in TS compared to what it was like back in the pure JS days. The article you linked to is definitely right about how the heavy need for complex type expressions in libraries vs end apps basically forces you to build up expertise on the type system if you want to have a smooth developer experience.

[vicary]

@bilalq To be fair, it's impractical to overhaul a project if the authors are not already using TypeScript everyday.

But if refactoring is happening by any chance, count me in for the help and long term maintenance.

[naorpeled]

Hey @willfarrell,
would love to help with some tasks,
I can start with http-router - parse path params,
if you have another task(s) in mind let me know :)

[willfarrell]

@naorpeled Thanks for the offer! I've been rethinking whether to include this in v4 or just add to v3. Depending on how it's implemented I think it could be included into v3 as a minor update and not have to wait for v4 to release (date unknown - waiting on AWS). I was imagining it would parse by default with an option to disable for APIG use. Please feel free to open a PR for this off of the main branch.

[willfarrell]

From a feature standpoint, #908, is ready for an alpha release. Because NodeJS 18 runtime has not released yet, additional change may be applied later. Please take a look and give feedback.

If someone could review and update the TypeScript to accommodate the changes to support AWS SDK v3, that would be greatly appreciated.

Thanks

[naorpeled]

From a feature standpoint, #908, is ready for an alpha release. Because NodeJS 18 runtime has not released yet, additional change may be applied later. Please take a look and give feedback.

If someone could review and update the TypeScript to accommodate the changes to support AWS SDK v3, that would be greatly appreciated.

Thanks

Hey,
I'd love to take this.

Regarding the task we've discussed earlier here, I did some digging into the event payloads in the different AWS services but I feel I need to look further into it, was on a spontaneous vacation so didn't have time to look further into it, I think I'll be able to work on it during this weekend.

[willfarrell]

@naorpeled No worries, we all need vacation. I already knocked out the feature for parsing pathParams. I woke up with an idea on how to do it cleanly, so I just did it. It just needs to be released, something for tomorrow.

[naorpeled]

@naorpeled No worries, we all need vacation. I already knocked out the feature for parsing pathParams. I woke up with an idea on how to do it cleanly, so I just did it. It just needs to be released, something for tomorrow.

👑

[willfarrell]

an initial alpha is now released for testing

[willfarrell]

nodejs18.x runtime is finally here. v4 will release soon after some additional testing.