Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Express Forward #160

Closed
wesleytodd opened this issue Jan 29, 2024 · 33 comments
Closed

Express Forward #160

wesleytodd opened this issue Jan 29, 2024 · 33 comments

Comments

@wesleytodd
Copy link
Member

Hey Everyone! I hope this issue finds you doing well, it has been a few years since I last posted in the project but it has been on my mind for a while that I wanted to do this. Thanks to some great help from @sheplu and @UlisesGascon (triage team members and folks active in the larger Node.js Ecosystem) we wanted to put forward a plan for the future of the Express project.

As we all know, the project has been in more of a maintenance mode for a long time now. Since the 5.x branch has not shipped despite a concrete plan to do so in 2020 we believe that in order to prevent the ecosystem from having to deal with more drastic measures, that we should make an effort to revive the project starting with a renewed look at the governance to help bring new contributors into the project.

Ideally this plan is uncontroversial and can be quickly acted upon. To do that, we thought that we should schedule a TC meeting with at least the folks last listed as active TC members (@dougwilson @LinusU @blakeembrey @crandmck @wesleytodd) and interested community members to discuss and commit to a direction. So below is the plan we worked out to get us back onto a healthy track, after kicking off this discussion here I would like to open an issue to schedule a TC meeting for some time in the next few weeks.

Plan

The plan comes in phases and focuses on direct tactical steps as opposed to strategic goals. Despite this, most of the tactical goals are backed up by larger strategic goals which we are omitting for brevity. Additionally there are some todo's and areas of ambiguity here. Ideally these would be filled in by collaborators as we move along and should not be blockers to agreeing on the general direction and goals.

Immediate Needs

These are things that we propose be done immediately upon approval of this plan.

  • Review and Adopt the Repo Captains Proposal
    • Review and make any last changes to wording, then merge
    • Immediately give commit access
    • Immediately give publish rights on npm
    • Setup a fast follow for onboarding more repo captains
  • Reform Active TC and open up new TC seats
    • Goal of 5 active members at all times going forward
  • Add Foundation accounts to all orgs, repos and npm packages

Express 5.0

  • Publish bit on all packages shared with the Foundation
  • Address any remaining issues in patch/minor release as a fast follow
  • Change Express-Generator to use express 5 as a default
  • Blog and announcements
    • This should be shown and shared to have the community aware of the changes and need to update

Express 6.0

  • Evaluate Node.js version support
    • Better align with Node.js LTS schedules
    • This helps accelerate the ability for Express and Node.js to move forward
    • It helps folks have a more predictable upgrade schedule for breaking changes
    • It helps plan for when folks need to land breaking changes on Express
    • This doesn’t mean we would make breaking changes for the sake of it, just that we wouldn’t test or explicitly support non-LTS versions on new major releases made past their EOL
  • Automated release process
    • Move release auth into GH permissions
    • Automate releases across all repositories
  • Remove unused middleware
    • TODO: assemble list
  • Replace middleware by node.js core solution
    • TODO: assemble list
  • Remove template and rendering from Express
    • Move to middleware
    • Express as an API first class citizen
  • Restructure Github orgs to clarify ownership and focus
  • @express scope on npm for sub packages
    • All new packages under the scope
    • Asses if we want to move any existing packages
  • Stop using Node.js internals / monkey-patching Node.js
  • Officially maintained TypeScript types
    • Generated and tested, with docs
  • Express Generator revamp
    • Target for API only (remove templating for HTML render`)

Express 7.0

  • Ecosystem collaboration
    • Work with node core on future for many of the express owned modules
    • Web Platform things like Request/Response and URL
    • Rewrite middleware to be added to node core / rewrite for performance
    • Collaborate in the Web Server Frameworks Team
    • Stop using Node.js internals / monkey-patching Node.js
  • LTS Plan and Release schedule
    • Mirror Node.js LTS and setup major release schedule
  • Modernization effort
    • Directly support meta frameworks, Graphql, and GRPC style usage
    • New website

All of this is up for discussion, the goal here is more about rallying people together to help progress happen but we thought coming with a concrete list of items would be more productive so please feel free to discuss individual points. If we need to we can break the discussion up but use this issue as a hub. And ideally the first order of business is getting the TC meeting scheduled and Express 5 out the door, so let's not let perfect be the enemy of the good here and commit to starting and keeping the progress flowing.

@dougwilson
Copy link
Contributor

👏 thank you so much @wesleytodd for this!

@crandmck
Copy link
Member

I've been MIA for quite some time, but I'd love to see this project pick up some steam again.

I will try to attend a TC meeting if I am able. LMK if there are candidate proposed times. My schedule is often full during PT business hours, so 5pm-9pm PT is likely to work better for me, but I will try to make any time that is chosen.

@wesleytodd
Copy link
Member Author

I created an issue to get a meeting organized: #161

@broofa
Copy link

broofa commented Feb 1, 2024

As a bit of an outsider (albeit one who's been using express for over a decade), the only part of this I find of real interest is how you plan on changing the organization and development processes to allow for regular releases.

I say this because Express has needed a new major release for at least 7 years, since async-await dropped in node@7 (circa 2017). V5 should have been that overhaul but it's just... never... dropped. The reasons for that are important because they're still pretty clearly in evidence.

Until these issues are addressed I question how much value there is in discussing a v5 release, let alone v6 or v7.

If I had to guess at the root cause it's that with a community as large as Express', dropping a new major release will involve a significant amount of follow-on work: issues to investigate, PRs to review, middleware projects to help upgrade, etc. But none of the current maintainers have time for that.

That makes for a certain lack of interest in actually clicking the "Publish" button.

Anyhow, that's my high-level take on this. With that in mind, some suggestions ...

💡 I like the idea of onboarding new team captains and TC members. I'd suggest making this the primary focus for the foreseeable future. This project needs new, fresh faces to help carry it forward.

💡 I like the expectation that team captains and TC members be "active". Do not underestimate how much passive team members can drag a project down simply by doing nothing. I'd make a point of starting the TC meetings with an informal and supportive poll to get a sense of everyone's level of availability and involvement.

😱 Awkward aside: The existing TC has not met for 3+ years. It has allowed v5 to go unpublished for 8-9 years. Is there a definition of "active" that is consistent with this? If not, what does that imply...?

💡 Does the TC have a chairperson? If not, it should. This new "push" is a non-trivial effort. It will require someone designated with spearheading it, and to insure the new TC and process(es) that get put in place stick. Whoever steps into this role should not do so lightly. It will be a significant amount of work.

💡 Use the v5 release to vet the new team structure, especially when it comes to release and support resources.

  • Have a "captain" for CI and release tooling.
  • Have a "captain" for support
  • Have a "captain" for media and community engagement

💡 I would also look at adding support for sponsorships. Recruiting maintainers is one of the hardest parts of running an opensource project. Providing a means of compensation will help with that.

My apologies if this ruffles anyone's feathers. If I'm misinformed or ill-informed, please do let me know. It'd be great to see this project move forward and continue to evolve.

@dougwilson
Copy link
Contributor

@broofa you should consider joining ❤️ 😃

@wesleytodd
Copy link
Member Author

the only part of this I find of real interest is how you plan on changing the organization and development processes to allow for regular releases.

Until these issues are addressed I question how much value there is in discussing a v5 release, let alone v6 or v7.

That is explicitly why the "immediate needs" are what they are! Getting the right structures in place to operate the project in a more healthy way as nearly always been my goal since I started participating in the project and I think the support in this issue shows that maybe we are aligned on that goal.

If I had to guess at the root cause, it's that with a community as large as Express' dropping a new major release will involve a significant amount of follow-on work: issues to investigate, PRs to review, middleware projects to help upgrade, etc. But none of the current maintainers have time for that.

We have had no lack of volunteers over the years. What we didn't have was the structure in place to foster them and get them to be strong active contributors. This is what I hope to fix by opening this issue and calling for these things to be addressed. The reason for even including the concrete list of steps for 5/6/7 was to put some clarity behind this that that work is required.

The existing TC has not met for 3+ years. It has allowed v5 to go unpublished for 8-9 years. Is there a definition of "active" that is consistent with this? If not, what does that imply...?

I don't want to get too in the weeds here, but personally I stepped back because lack of progress caused me to burn out. If we cannot get the governance in place with a truly active group in the near future then I will personally be stepping down from the TC. We have verbage for moving members from active to inactive in the charter, but it is not enough. This is part of what I meant we needed to address under the "reform the tc with 5 active members".

Does the TC have a chairperson?

I tend to agree we should have this, but we would likely need to ammend the charter to make that happen. I am on board if we want to add that to the agenda.

I would also look at adding support for sponsorships

I have opened issues for this in the past in this repo. I agree we should do this. Lets figure out how to slot those into the plan.

My apologies if this ruffles anyone's feathers.

Not at all!!! The whole point of this was to draw out folks like yourself who had good ideas on how to unstick the project. Thanks for taking the time to write this all up and please think about coming to our meeting to help us achieve these goals.

@broofa
Copy link

broofa commented Feb 1, 2024

please think about coming to our meeting

Of course. If nothing else, it's a chance to put some faces and voices to names I've interacted with numerous times over the years. (No promises about getting involved, though! 😉 )

@dougwilson
Copy link
Contributor

I agree with @wesleytodd said above. We probably need to have a complete, cold hand over to others and hope they can get up and running. I just no longer have the time or energy to carry such a large project, just code wise, let alone anything else. As I mentioned to @wesleytodd for the past, probably couple years the endless flow of flase vulnerability reports and threads of filing CVEs, trying to argue with people, many times have never even used javascript, just sucked away all my time. If I didn't tend to it, probably every module in express would be security blacklisted for no reason. I am honestly kinda done with all this non sense Express seems to attract, and of course we now have the endless SPAM PRs on the main repo I cannot get to stop. Some awesome folks really would be awesome to step in and figure this all out 🙏

@wesleytodd
Copy link
Member Author

and hope

Other than this part, I agree. I think we need to do more than hope and I am personally volunteering to help make sure we do more than hope. I have time to help, just not the time or will to do it alone or with a group too small to achieve the goals. Luckily I don't think we will lack volunteers if we have the right setup for them to succeede.

@dougwilson
Copy link
Contributor

dougwilson commented Feb 1, 2024

Haha, fair. I pretty much only have at best 1 hour a day to work on any OSS any more, so hopefully that puts into perspective how constrained I have been. I feel like @wesleytodd hinted at, I think the best thing we can do for the project is pretty much an accounting of all the stuff, get everything moved to the foundation accounts and stuff, and I can disappear into the night 😃 bc honestly I probably need like a year break from this OSS stuff, like a sabbatical.

Edit: and I don't want my very limited availability to stop whatever momentum that comes from all this!

@wesleytodd
Copy link
Member Author

wesleytodd commented Feb 1, 2024

Burnout is REAL, and OSS can create an environment where it is easy to hit. I think we can all say we really really appreciate all the work you have done to keep things afloat. And now hopefully we can get a group of folks who can help take that off your shoulders (or help you find a better future balance with the project after you take that well deserved break)!

@dwelch2344
Copy link

Hey all! Just wanted to throw my hat in the ring here: would love to help contribute where I can. Also want to look at sponsorship, along with how our team could commit cycles on specific efforts!

I'm going to bring this up internally with our team tomorrow, but personally I'd love to get started on the CI/CD front. I haven't dug into the repo yet to see what's in place, but I'm a huge fan of Commit -> GH Actions -> Release pipelines as a starting point – and given some of the items above it sounds like that could be of use?

I'm not sure I could commit to being a captain or anything, but I'd love to be workhorse if someone wants to point me in the right direction. Docs, tests, builds, whatever :)

@ljharb
Copy link
Contributor

ljharb commented Feb 5, 2024

I'd also love to help out; I don't have time to write code, but I'm happy to join the TC or be a repo captain, or anything similar that would help.

@wesleytodd
Copy link
Member Author

Thanks @dwelch2344 and @ljharb! We absolutely need folks in all sorts of capacities to deliver on all the things on the list above, so everything from setting up and maintaining better shared CICD workflows for the many repos to helping provide technical leadership and direction for the community will be very welcome! If you haven't yet, check out #161 and add your intent to attend (with your TZ) so we can find a time which works for most people.

After that meeting we will likely have to revive a few issues and reach alignment on the way forward before we start pointing folks in direction for work, but hopefully that shouldn't take long once we get started.

@DominusKelvin
Copy link

Hey @wesleytodd thanks for putting this together.

I just had the time to take a thorough read and it does look good.

I'm happy to contribute in any capacity I can as we at Sails Core team 💜 Express.

@s1moe2
Copy link

s1moe2 commented Feb 11, 2024

Throwing my hat in for anything that is needed.
Like many others, I've been using Express for many years but I've never really contributed.

I'm open to contribute with active development, DevOps, or simply repo and docs maintenance.

There's also a chance I can bring in sponsorship, but would need to know more about how a sponsor can help before presenting this internally.

I would love to see this project moving forward :)

@mcollina

This comment was marked as off-topic.

@wesleytodd

This comment was marked as off-topic.

@collinsnji
Copy link

I would love to contribute as well, with a focus on documentation. I can also contribute to the code, so put me to work :)

@OAtulA

This comment was marked as off-topic.

@wesleytodd

This comment was marked as off-topic.

@expressjs expressjs deleted a comment from safarli Feb 13, 2024
@davericher
Copy link

How does one join said committee

@wesleytodd
Copy link
Member Author

How does one join said committee

We have some docs in the express repo about this. One of the things we are going to do in tonight's meeting is refine some of the language and try to make it more clear for the future. I doubt there will be a lot changed in the general process though, so this is the gist:

  1. start contributing to the project (answering questions, helping review issues/prs, learn the ropes)
  2. join the triage team, continue doing step 1
  3. dig into one or more of the packages more deeply, land some features or docs or whatever
  4. follow the contributing docs and get someone on the TC to recommend you be a new TC member

@davericher

This comment was marked as off-topic.

@davericher

This comment was marked as off-topic.

@UlisesGascon UlisesGascon pinned this issue Feb 17, 2024
@polRk

This comment was marked as off-topic.

@gerardolima
Copy link

where can I volunteer?

@wesleytodd
Copy link
Member Author

@gerardolima We will be spinning up the triage group again soon which will be the best way to get started. See the docs in the repo for that. Also see my above comments about getting involved.

@wesleytodd
Copy link
Member Author

@sheplu #167 is much more robust than this issue. Think we can close this and use yours as the tracking issue?

@MahyarGdz
Copy link

I would like to contribute 💪🏻💪🏻

@wesleytodd
Copy link
Member Author

Ok, closing this issue! REALLY want to thank everyone for the great work so far getting this stuff off the ground. Excited to see where we can do with this!

@Zorono
Copy link

Zorono commented Apr 6, 2024

regarding 'immediate needs', giving commit access and publish rights (in case i understood this enough) is a dangerous decision as it could be used for pushing malicious changes (like xz nightmare)

@UlisesGascon
Copy link
Member

regarding 'immediate needs', giving commit access and publish rights (in case i understood this enough) is a dangerous decision as it could be used for pushing malicious changes (like xz nightmare)

Thanks for bringing your concerns to our attention, @Zorono! 😊

The immediate needs served as the starting point for this initiative. We wanted to ensure that our focus from the beginning was on re-enabling the TC team and repo captains to properly operate with the expected autonomy (as any project at this scale).

Currently, Express is undergoing a security audit with OSTIF (see: issue tracker), as other key libraries in the ecosystem have recently done (see: Audits completed).

Additionally, we've established a Security WG to handle all security-related matters (see: current responsibilities).

As a direct result of this effort, we recently addressed an open redirect vulnerability (more info) and plan to continue improving the project over time (see: current initiatives).

If you're interested in helping the project and contributing, you're more than welcome to participate in the Security WG discussions and initiatives.

@UlisesGascon UlisesGascon unpinned this issue Apr 16, 2024
RobinTail added a commit to RobinTail/express-zod-api that referenced this issue Sep 10, 2024
Instead of #374 but non-breaking

The express 5 beta 3 is fully operational and tested by community, but
its release is delayed.
The current issue is that it's also outdated comparing to v4.18.2,
having nested dependencies of older versions.

latest tracking issue:
```
expressjs/discussions#233
```

previous one:
```
expressjs/express#5205
```

Latest proposal:
```
expressjs/discussions#160
```

Probably it will be ready in march 🤞🏽  (no, it's not)
```
expressjs/express#5111 (comment)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests