crypto: add crypto.sign() and crypto.verify() #26611

mscdex · 2019-03-12T14:19:36Z

These methods are to allow signing and verifying using Ed25519 and Ed448 keys, which do not support streaming of input data.

Fixes: #26320

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

nodejs-github-bot · 2019-03-12T14:19:43Z

@mscdex build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/2879/pipeline

mscdex · 2019-03-12T14:21:54Z

CI: https://ci.nodejs.org/job/node-test-pull-request/21456/

/cc @nodejs/crypto

tniessen

Maybe we should consider other names? I think having such "one shot" APIs might be useful, also for hashing, but the names sound a bit awkward to me. Do we need to reserve the names sign, verify, hash etc. for something else?

tniessen · 2019-03-12T14:54:16Z

doc/api/crypto.md

@@ -2651,6 +2651,18 @@ added: v10.0.0
 Enables the FIPS compliant crypto provider in a FIPS-enabled Node.js build.
 Throws an error if FIPS mode is not available.

+### crypto.signOneShot(data, privateKey)


IIRC all other functions that consume keys take them as the first argument.

I ordered them according to the ordinary streaming flow (.update(data) followed by .sign(privateKey)), which made sense to me.

I understand that and it makes sense, but it deviates from all other functions.

tniessen · 2019-03-12T14:54:50Z

doc/api/crypto.md

+added: REPLACEME
+-->
+* `data` {Buffer | TypedArray | DataView}
+* `privateKey` {Buffer | KeyObject}


This deviates from other functions that consume keys, where they can also be strings and certain objects.

Supporting string encoding names complicates things and makes separating required vs. optional function arguments nearly impossible as far as I can tell.

Besides, the only reason we have the string encoding parameters is for backwards compatibility and internally node immediately converts it to a Buffer using Buffer.from(), something userland could easily do themselves these days...

I'm not talking about the data, only about privateKey. See e.g. privateDecrypt:

If privateKey is not a KeyObject, this function behaves as if privateKey had been passed to crypto.createPrivateKey().

I designed that and I still think it is a good idea while maintaining compatibility with older API designs.

To me it should be all or nothing, it doesn't make sense to have an encoding parameter for just one parameter when we're discussing backwards compatibility (.update() also supports an encoding parameter).

Again, I am not talking about the data parameter and none of the existing APIs accept an encoding option or argument for parsing the key, since PEM is ASCII-only.

As far as I can tell, your code already accepts strings and objects as the privateKey, the documentation does not match that behavior, that's all I am trying to explain.

Doc updated.

tniessen · 2019-03-12T14:55:11Z

doc/api/crypto.md

+added: REPLACEME
+-->
+* `data` {Buffer | TypedArray | DataView}
+* `key` {Buffer | KeyObject}


Same as above.

tniessen · 2019-03-12T14:56:14Z

doc/api/crypto.md

+
+Calculates and returns the signature for `data` using the given private key.
+
+This method only supports Ed25519 and Ed448 keys.


Is there a reason for this restriction? I think having these "one shot" implementations could solve problems such as #25857 where JS object creation time is critical.

Currently the reason is to keep things simple and the fact that EdDSA does not utilize a hash algorithm argument with OpenSSL's API.

The lack of a digest arg and signing algorithm makes this very EdDSA specific, if its going to be that specific, might as well just call it what it is: crypto.eddsaSign(). I'm not opposed to that, its a bit odd, but then, EdDSA made some odd algorithm design choices so it doesn't fit so well into generice asym sign/verify APIs. Using the alg-specific name would allow us to do a full one-shot API later.

My skim of the RFCs gave me the impression that there was PureEdDSA and EdDSA, but this API doesn't allow a choice between them. Not a blocker, but I'm curious, does OpenSSL not support them, or did I misunderstand?

I think that even if the other key types are left unsupported for now and only EdDSA is implemented, if there is a one-shot API, it should be equivalent to the multi-shot API:

createSign(digest) .update(input, inputEncoding) .sign(privateKey+padding+saltLength, outputEncoding)

Its unfortunate IMO that we are squeezing the signature algorithm (technically, the padding method, but most APIs consider the padding to be part of the signature algorithm) into the privateKey, but that's how it is.

crypto.signOneShot(digest, data, dataEncoding, privateKey, outputEncoding) would be one way (with private Key being all the possibilities of Sign.sign()), but using an options object might be nicer, particularly since all 5 args can be string, so its not easy to just not provide them and have the API provide default values.

The one-shot version would be expected to have a null for the digest with EdDSA.

Same general comment on the verify function below.

The lack of a digest arg and signing algorithm makes this very EdDSA specific, if its going to be that specific, might as well just call it what it is: crypto.eddsaSign()

There was interest shown earlier about possibly allowing an algorithm in the future somehow, so I made the name more generic. I honestly don't care so much about the name as long as it's fairly accurate and concise and doesn't look like it came from Java.

My skim of the RFCs gave me the impression that there was PureEdDSA and EdDSA, but this API doesn't allow a choice between them. Not a blocker, but I'm curious, does OpenSSL not support them, or did I misunderstand?

As noted in the referenced issue, OpenSSL implements PureEdDSA which does not support streaming data.

but using an options object might be nicer, particularly since all 5 args can be string, so its not easy to just not provide them and have the API provide default values.

Possibly, but the other reason for using an object is that having the encoding parameters be optional like they are in the streaming API would be very hard to detect all of the various possibly-string arguments.

As noted in other comments, I did not bother implementing the encoding parameters because they are part of a legacy API and using Buffer.from() and buffer.toString() are trivial these days and it greatly simplifies the implementation.

OK, so that addresses most of my comments, but the plan for this is unclear:

There was interest shown earlier about possibly allowing an algorithm in the future somehow, so I made the name more generic. I honestly don't care so much about the name as long as it's fairly accurate and concise and doesn't look like it came from Java.

I don't know what Java's API conventions are, so this doesn't illuminate the kinds of names you like, for me at least.

I still don't love xxxOneShot() as a naming convention, but I can live with it.

I don't think the API should have a generic name if it doesn't have enough args to one-shot a generic sign algorithm. digest should be added as the first arg, IMO, if the name stays generic.

tniessen · 2019-03-12T14:58:03Z

test/parallel/test-crypto-sign-verify.js

+    assert.strictEqual(sig.length, pair.sigLen);
+
+    assert.strictEqual(crypto.verifyOneShot(data, pair.private, sig), true);
+    assert.strictEqual(crypto.verifyOneShot(data, pair.public, sig), true);


There should also be tests for verification failures (e.g. using the wrong key, the wrong message, the wrong signature).

mscdex · 2019-03-12T15:28:15Z

Maybe we should consider other names?

I had to choose something.

Do we need to reserve the names sign, verify, hash etc. for something else?

I think having a static method that's named the same as an instance method could be confusing.

jasnell · 2019-03-13T14:32:26Z

doc/api/crypto.md

@@ -2651,6 +2651,21 @@ added: v10.0.0
 Enables the FIPS compliant crypto provider in a FIPS-enabled Node.js build.
 Throws an error if FIPS mode is not available.

+### crypto.signOneShot(data, key)


Why not just crypto.sign() and crypto.verify()

Previous comments: #26611 (review) and #26611 (comment)

mscdex:

I think having a static method that's named the same as an instance method could be confusing.

I think its the opposite of confusing, it makes it clear it does the same thing, but with no saved state (because there is no object to hold state), its clearly one-shot.

jasnell · 2019-03-13T14:35:14Z

src/node_crypto.cc

+    case EVP_PKEY_ED448:
+      break;
+    default:
+      return env->ThrowError("Unsupported key type");


Is it possible to do this check at the JavaScript layer? Either way, it would be great to have an ERR_* code for this.

Probably not reliably and not without crossing the JS<->C++ boundary an extra time. Is there a way to throw errors with codes set from C++?

EDIT: I see that there is. I could add a new error code I suppose, although nothing else in this source file utilizes error codes.

Added, but I noticed that the name for error objects produced from C++ are not formatted the same as those generated from JS. JS includes the code in brackets after the error type. C++ does not.

I would like an error code, if its not too much trouble. I have a PR almost done that adds a lot more error codes, but it didn't work everywhere, and I had to put it aside while doing other work. Every new throw of a string is something I'll have to convert to setting .code sometime in the future.

mscdex · 2019-03-14T20:43:10Z

Marking this for not landing on v10.x/v11.x unless it's decided at some point that we don't support OpenSSL v1.1.0 or older for those branches.

sam-github · 2019-03-15T21:39:54Z

doc/api/crypto.md

@@ -2651,6 +2651,21 @@ added: v10.0.0
 Enables the FIPS compliant crypto provider in a FIPS-enabled Node.js build.
 Throws an error if FIPS mode is not available.

+### crypto.signOneShot(data, key)


mscdex:

I think having a static method that's named the same as an instance method could be confusing.

I think its the opposite of confusing, it makes it clear it does the same thing, but with no saved state (because there is no object to hold state), its clearly one-shot.

sam-github · 2019-03-15T21:57:16Z

doc/api/crypto.md

+
+Calculates and returns the signature for `data` using the given private key.
+
+This method only supports Ed25519 and Ed448 keys.


The lack of a digest arg and signing algorithm makes this very EdDSA specific, if its going to be that specific, might as well just call it what it is: crypto.eddsaSign(). I'm not opposed to that, its a bit odd, but then, EdDSA made some odd algorithm design choices so it doesn't fit so well into generice asym sign/verify APIs. Using the alg-specific name would allow us to do a full one-shot API later.

My skim of the RFCs gave me the impression that there was PureEdDSA and EdDSA, but this API doesn't allow a choice between them. Not a blocker, but I'm curious, does OpenSSL not support them, or did I misunderstand?

I think that even if the other key types are left unsupported for now and only EdDSA is implemented, if there is a one-shot API, it should be equivalent to the multi-shot API:

createSign(digest) .update(input, inputEncoding) .sign(privateKey+padding+saltLength, outputEncoding)

Its unfortunate IMO that we are squeezing the signature algorithm (technically, the padding method, but most APIs consider the padding to be part of the signature algorithm) into the privateKey, but that's how it is.

crypto.signOneShot(digest, data, dataEncoding, privateKey, outputEncoding) would be one way (with private Key being all the possibilities of Sign.sign()), but using an options object might be nicer, particularly since all 5 args can be string, so its not easy to just not provide them and have the API provide default values.

The one-shot version would be expected to have a null for the digest with EdDSA.

Same general comment on the verify function below.

mscdex · 2019-03-18T07:40:54Z

/cc @nodejs/collaborators

mscdex · 2019-03-25T14:52:53Z

I've added a commit now that adds support for all key types.

Thoughts @nodejs/collaborators ?

New CI: ~~https://ci.nodejs.org/job/node-test-pull-request/21871/~~ https://ci.nodejs.org/job/node-test-pull-request/21875/

nodejs-github-bot · 2019-03-25T14:53:54Z

CI: https://ci.nodejs.org/job/node-test-pull-request/21871/

jasnell · 2019-03-25T15:16:16Z

src/node_crypto.cc

+    return CheckThrow(env, SignBase::Error::kSignPrivateKey);
+  }
+
+  // XXX(mscdex): unnecessary?


Suggested change

// XXX(mscdex): unnecessary?

// TODO(mscdex): unnecessary?

I went ahead and just removed the line of code.

jasnell

Since don't love the names but code LGTM otherwise

nodejs-github-bot · 2019-03-25T15:45:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/21875/

jasnell · 2019-03-26T15:04:12Z

Thank you @mscdex ... I know those names weren't your preference but I definitely appreciate you being willing to accommodate.

mscdex · 2019-03-26T16:02:50Z

@sam-github With the recent changes do your concerns still stand?

sam-github

LGTM, sorry it took a few days to get to re-reviewing.

sam-github · 2019-03-26T16:20:33Z

doc/api/crypto.md

+  * `crypto.constants.RSA_PKCS1_PADDING` (default)
+  * `crypto.constants.RSA_PKCS1_PSS_PADDING`
+
+  Note that `RSA_PKCS1_PSS_PADDING` will use MGF1 with the same hash function


rather than repeating the docs from the streaming API, you could link to them. Or not. This isn't even a nit, just something to consider. I tend to try not to repeat docs, and emphasizing the relationship between the multi- and single-shot APIs might be worthwhile.

tniessen

LGTM. My only concern is that we are reinforcing the mixture of arguments and options objects, especially in crypto.verify where the options are not the last parameter, but I am not even sure whether that is actually a problem. It will be if create***Key ever accepts options that clash with options other crypto APIs accept.

sam-github · 2019-03-26T17:51:19Z

I share the concern above, but it applies equally to the existing APIs. When we come up with a solution, it should be applied uniformly to both APIs. In the meantime, this API is internally consistent.

nodejs-github-bot · 2019-03-28T17:48:38Z

CI: https://ci.nodejs.org/job/node-test-pull-request/21990/

nodejs-github-bot · 2019-03-28T23:02:34Z

CI: https://ci.nodejs.org/job/node-test-pull-request/21999/

These methods are added primarily to allow signing and verifying using Ed25519 and Ed448 keys, which do not support streaming of input data. However, any key type can be used with these new APIs, to allow better performance when only signing/verifying a single chunk. Fixes: nodejs#26320 PR-URL: nodejs#26611 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Tobias Nießen <[email protected]>

These methods are added primarily to allow signing and verifying using Ed25519 and Ed448 keys, which do not support streaming of input data. However, any key type can be used with these new APIs, to allow better performance when only signing/verifying a single chunk. Fixes: #26320 PR-URL: #26611 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Rod Vagg <[email protected]> Reviewed-By: Sam Roberts <[email protected]> Reviewed-By: Tobias Nießen <[email protected]>

mscdex added crypto Issues and PRs related to the crypto subsystem. lts-watch-v10.x labels Mar 12, 2019

nodejs-github-bot added the c++ Issues and PRs that require attention from people who are familiar with C++. label Mar 12, 2019

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from e1ee2d6 to 43be5b7 Compare March 12, 2019 14:30

tniessen reviewed Mar 12, 2019

View reviewed changes

panva mentioned this pull request Mar 12, 2019

OKP Support (EdDSA) panva/jose#12

Closed

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 43be5b7 to 000ccf0 Compare March 12, 2019 16:55

panva mentioned this pull request Mar 13, 2019

crypto: X25519 and X448 and ECDH #26626

Closed

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 000ccf0 to e3af7fb Compare March 13, 2019 09:52

jasnell reviewed Mar 13, 2019

View reviewed changes

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from e3af7fb to 61ca0f5 Compare March 14, 2019 20:24

mscdex added dont-land-on-v10.x and removed lts-watch-v10.x labels Mar 14, 2019

sam-github suggested changes Mar 15, 2019

View reviewed changes

sam-github mentioned this pull request Mar 18, 2019

crypto hash, hmac and cipher ops could be 2x-3x faster for "one-shot" small buffers #26748

Closed

panva mentioned this pull request Mar 19, 2019

crypto: add support for x25119 and x448 key pair generation #26774

Closed

4 tasks

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 61ca0f5 to 4179a02 Compare March 25, 2019 14:51

jasnell reviewed Mar 25, 2019

View reviewed changes

jasnell approved these changes Mar 25, 2019

View reviewed changes

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 4179a02 to 4152ec1 Compare March 25, 2019 15:44

sam-github approved these changes Mar 26, 2019

View reviewed changes

tniessen approved these changes Mar 26, 2019

View reviewed changes

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 7c60eb3 to 1d25465 Compare March 28, 2019 17:47

mscdex added the semver-minor PRs that contain new features and should be released in the next minor version. label Mar 28, 2019

mscdex force-pushed the crypto-sign-verify-eddsa-support branch 4 times, most recently from d76b1e3 to 109c17f Compare March 28, 2019 23:01

mscdex force-pushed the crypto-sign-verify-eddsa-support branch from 109c17f to 7d0e50d Compare March 29, 2019 02:00

mscdex merged commit 7d0e50d into nodejs:master Mar 29, 2019

mscdex deleted the crypto-sign-verify-eddsa-support branch March 29, 2019 11:39

Flarna mentioned this pull request Apr 24, 2019

feat(node): v12 DefinitelyTyped/DefinitelyTyped#34952

Merged

9 tasks

tniessen mentioned this pull request Mar 6, 2022

crypto: add crypto.hash #42233

Closed


		Calculates and returns the signature for `data` using the given private key.

		This method only supports Ed25519 and Ed448 keys.

crypto: add crypto.sign() and crypto.verify() #26611

crypto: add crypto.sign() and crypto.verify() #26611

Conversation

mscdex commented Mar 12, 2019

Checklist

nodejs-github-bot commented Mar 12, 2019

mscdex commented Mar 12, 2019 • edited Loading

tniessen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex Mar 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex commented Mar 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex Mar 13, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex commented Mar 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mscdex commented Mar 18, 2019

mscdex commented Mar 25, 2019 • edited Loading

nodejs-github-bot commented Mar 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Mar 25, 2019

jasnell commented Mar 26, 2019

mscdex commented Mar 26, 2019

sam-github left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tniessen left a comment

Choose a reason for hiding this comment

sam-github commented Mar 26, 2019

nodejs-github-bot commented Mar 28, 2019

nodejs-github-bot commented Mar 28, 2019

mscdex commented Mar 12, 2019 •

edited

Loading

mscdex Mar 16, 2019 •

edited

Loading

mscdex Mar 13, 2019 •

edited

Loading

mscdex commented Mar 14, 2019 •

edited

Loading

mscdex commented Mar 25, 2019 •

edited

Loading