RFC: Node.js® 4.x support

[flavorjones]

RFC: Node.js® 4.x support

This issue is intended to describe the current challenges around supporting Node.js 4.x in the nodejs-buildpack, and introduces several potential tracks of work, along with the Buildpack PM's preferred solution.

The Challenge

Node.js 4.x depends on openssl 1.0.2

OpenSSL is not infrequently updated in response to security vulnerabilities (CVEs), and as such the Buildpacks team has been dynamically linking the Node binaries against the rootfs openssl library. This allows applications to be patched with a rootfs update (i.e., applications don't need to be restaged).

The cflinuxfs2 rootfs packages Debian's openssl v1.0.1, which is the version depended upon by Node 0.12.x and earlier.

As of Node 4.x, though, the dependency is on openssl 1.0.2. We've opened a Github Issue reporting this as a bug, but have not made much progress in convincing the Node core team to support openssl 1.0.1.

The Proposal

Full explanation and context are below, but in short the proposal is:

1. nodejs-buildpack would ship binaries for node 4.x that statically link the vendored openssl 1.0.2; with the downside that openssl patches will require app restaging.
2. Based on usage and feedback, we may decide later to maintain openssl 1.0.2 on the cflinuxfs2 rootfs; but would defer this decision for now.

---

What We've Tried So Far

Compile against openssl 1.0.2 and dynamically link against 1.0.1

We attempted to take advantage of the ABI backwards-compatibility between 1.0.2 and 1.0.1 by compiling against the former, then dynamically loading the latter.

Unsurprisingly, this didn't work, as Node 4.x is actually using new methods corresponding to new features in openssl 1.0.2.

Install openssl 1.0.2 into cflinuxfs2

We attempted to install 1.0.2 from upstream (backported) Ubuntu Wily on the same system as 1.0.1, but unfortunately the .deb packages clobber each other, as they have the same package name: openssl.

Remaining Possible Solutions

Introduce a wily-based stack

wily ships with openssl 1.0.2, and so we could introduce a new stack based on this Ubuntu distribution.

This is an extraordinarily heavyweight solution, which I'm extremely hesitant to even suggest, given an absence of compelling reasons to introduce this level of complexity to the product.

Let's pretend I didn't even bring this up.

Maintain our own openssl 1.0.2 for dynamic linking

We could build our own CF-specific openssl 1.0.2 package to install alongside the rootfs's openssl 1.0.1.

Pros:

• This isn't totally crazy,
• and would allow us to continue to patch applications via rootfs updates (not restaging)

Cons:

• This is a little bit crazy.
• It would add complexity to our build system,
• and would take a significant upfront investment to make sure it was done right and automated properly,
• to add support for an interpreter that we're not yet sure anyone actually wants to use.

Statically link the Node-vendored openssl 1.0.2

The final option is to follow the Node team's advice, and just take the Node-vendored openssl 1.0.2 as it is, and statically link it into the interpreter binary.

This solution requires the Node team make prompt releases after openssl vulnerabilities. The 0.12.x (and previous) release cycles were not great, but the io.js team running the release cycle for 3.x and 4.x appear to really have their act together, making nearly-weekly feature and bugfix releases.

Pros:

• Very little work for the Buildpacks team,
• allowing us to ship something and validate the need for Node 4.x support.

Cons:

• Restaging would be required to patch apps for future openssl CVEs.
• Requires relying on the Node team for prompt updates.

Related Issues

Worth noting that this bug filed against node 0.12.x is related, in that the stock openssl 1.0.1.f shipped with Ubuntu Trusty doesn't contain an upstream patch that is applied in the vendored openssl (1.0.1.k, it looks like) in the 0.12.x official build.

In Summary, or, How You Can Help

Given the fact that I've seen very little demand for Node 4.x support, I hereby propose to choose the most expedient solution, "Statically Link v1.0.2", in order to be able to quickly validate whether CF users actually even care about Node 4.

If we see real demand for production Node 4.x support, either in this ticket or after we implement static linking of v1.0.2, then I'm open to moving to the "Maintain Our Own Dynamically Linked Openssl v1.0.2" solution.

How You Can Help

Comment on this issue.

If you want to see Node 4.x support, let us know!

If you prefer one solution or the other, let us know!

If you have any other ideas on how to accomplish our goal of Node 4.x support, let us know!

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/105040706.

[jthomas]

👍 on Node.js v4 support.
It's a big release for the project and I definitely want to be able to use this on CF.

[libnux]

We want Node 4.x support on CF.

Some clients of our platform already asked whether node 4.x is supported.

Although it's possible to support node 4.x with docker image, I think this way is a little bit heavy compared with using buildpack.

[charyorde]

+1 for Node.js v4 support

Need the node-gyp rebuild optimization feature.

[flavorjones]

Thanks to everyone who's commented so far. It's great to see some people interested.

But to be completely clear: I want to know what you think about the proposal, and the fact that we'll be moving away from auto-updating apps (via rootfs updates and dynamic linking) to forcing devs to own app updates (via restaging) for opennsl CVEs.

[flavorjones]

@charyorde Can you please open a new issue with a full description of the behavior you want from node-gyp? We've attempted to engage the nodejs community on this topic before, and there was a decided lack of participation. Would love to have help.

[youngm]

+1 to statically linking. Seems like the simplest and most straight forward solution. Lets not waste time on a more complex one until we need to.

Should only be an issue until a new stack is released based on Ubuntu LTS 16.04, right?

[charyorde]

@flavorjones it's the same #31 which I believe is resolved in Node v4. Just need to be tested.

I'll favour "Static linking" considering that Node v4 release is a joint effort of the Node and io.js team. Release updates can only be better going forward.

[jthomas]

+1 on static linking. Seems like the best tradeoff of concerns.

[adrai]

+1 we're going to productively deploy over 70 apps with node 4.2.x in the next weeks.

[dotchev]

How do you guarantee the stability of the apps when you update the rootfs underneath without their knowledge?

[flavorjones]

@dotchev - Thanks for asking this question.

When we update the rootfs, that simply means that we've done a apt-get upgrade on the existing installed packages. So, if your app, or the interpreter used by your app, is dynamically linking against a shared library, things should Just Work™. Make sense?

The specific example here is that Debian/Canonical will make sure to backport (or simply merge) any patches that address CVEs into the packaged version of OpenSSL 1.0.1f. As long as the binary interface hasn't changed, this is generally considered a safe operation.

Updating the rootfs wouldn't be safe if the binary interfaces were allowed to change (e.g., upgrade libc from 2.19 to 2.20; or if we removed libraries; or if we changed the architecture of the rootfs (e.g., 64-bit to 32-bit); or if we changed the version of the libraries (e.g., update the rootfs from 14.04-based to 15.04-based).

Make sense?

[yurikoex]

👍 my two teams want to move our 20+ pcf apps to 4.x as soon as possible. Static linking sounds good.

[PMByrne]

+1 from me and my team on static linking as well. The NodeJS foundation is doing insanely well on getting out timely point releases. We have plans for a major rewrite of our entire framework from Scala to Node and need a reliable CF buildpack with NodeJS 4.2.x LTS as soon as you guys can.

Right now we are scraping by with a custom buildpack but an officially supported buildpack would be much appreciated.

[flavorjones]

It sounds like we have consensus on moving forward with the proposed plan.

We'll prioritize this work, and I'll update this Issue when we have something ready for people to beta-test.

[flavorjones]

Hello all,

We recently released nodejs-buildpack v1.5.1 which introduces support for Node.js 4.2.2.

Node.js 4.2.2 is the latest (as of 2015-11-06) version on Node's LTS branch, which we intend to support for the lifetime of the 4.x branch.

We welcome feedback on this version of Node, which you can provide by:

• opening a new Github Issue letting us know about any issues you experience using it
• commenting on this Github Issue if you've tried it and it's working as expected for you
• emailing cf-dev with any comments at all

Thank you!

-mike

[charyorde]

Wow. Thank you

Testing away...

[voelzmo]

@flavorjones Thanks for that release! So which of the above proposals did you end up taking? Is openssl now statically linked, so everybody has to restage their app when a CVE is patched by the node team?

[flavorjones]

@voelzmo Correct, as proposed, we're statically linkling openssl 1.0.2 into the node binary.

At some point in the future, we may re-examine how we're handling this and attempt to dynamically link against a library in the rootfs; but I want to get more feedback from people to justify that decision before committing to it.

[voelzmo]

@flavorjones Thanks for the clarification. In the OP there was still more than 1 proposal, so I wanted to make sure I got that correctly.