Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

jakemas · 2024-10-25T16:48:35Z

Issues:

Resolves #CryptoAlg-2725

Description of changes:

This is a sizable PR, I've worked extremely hard to keep the size down in a single PR, but with ML-DSA touching X.509, PKEY, ASN1, there are a lot of changes that need to happen simultaneously to preserve library functionality, so I will document this PR well.

1. PKEY structure changes

This PR adds ML-DSA-44 and ML-DSA-87 to AWS-LC. Before this PR AWS-LC only supported ML-DSA-65, as such, we were utilizing the void *ptr of evp_pkey_st, rather than a distinct structure for ML-DSA.

This PR introduces the new structure NISTDSA_KEY * nistdsa_key; inside evp_pkey_st to support ML-DSA and any additional FIPS digital signature algorithms provided by NIST, should AWS-LC wish to include them in the library.

  union {
    void *ptr;
    RSA *rsa;
    DSA *dsa;
    DH *dh;
    EC_KEY *ec;
    KEM_KEY *kem_key;
    NISTDSA_KEY * nistdsa_key;
  } pkey;

While the structure DSA already exists it does not provide support for FIPS-like signature APIs (see more at NIST api conventions. Rather than create a PKEY struct of the form MLDSA_KEY that can only support ML-DSA, this is an opportunity to build support for future signature algorithms that have similar. calling structures, de-randomized testing modes, API converntions, etc. by making a more generic structure type -- much like we did for KEM_KEY. Under the design in this PR, adding SLH-DSA to the PKEY struct would be very simple, as it conforms to a NISTDSA_KEY in design. So too will be true of any additional signature algorithms produced by NISTs call for additional signature algorithms.

As, such, NISTDSA_KEY utilizes a new structure to hold public/secret keys, as well as a NISTDSA struct which defines signature algorithm specific information:

struct nistdsa_st {
  const NISTDSA *nistdsa;
  uint8_t *public_key;
  uint8_t *secret_key;
};

typedef struct {
  int nid;
  const uint8_t *oid;
  uint8_t oid_len;
  const char *comment;
  size_t public_key_len;
  size_t secret_key_len;
  size_t signature_len;
  size_t keygen_seed_len;
  size_t sign_seed_len;
  const NISTDSA_METHOD *method;
  const EVP_PKEY_ASN1_METHOD *asn1_method;
} NISTDSA;

This allows us to use a single PKEY structure for all NIST FIPS signature algorithms, much like the existing PKEY struct KEM_KEY *kem_key for KEMS. As a consequence, we are now able to define signature methods such as:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->sign = ml_dsa_65_sign;
  out->verify = ml_dsa_65_verify;
}

DEFINE_LOCAL_DATA(NISTDSA, sig_ml_dsa_65) {
  out->nid = NID_MLDSA65;
  out->oid = kOIDMLDSA65;
  out->oid_len = sizeof(kOIDMLDSA65);
  out->comment = "MLDSA65 ";
  out->public_key_len = MLDSA65_PUBLIC_KEY_BYTES;
  out->secret_key_len = MLDSA65_PRIVATE_KEY_BYTES;
  out->signature_len = MLDSA65_SIGNATURE_BYTES;
  out->keygen_seed_len = MLDSA65_KEYGEN_SEED_BYTES;
  out->sign_seed_len = MLDSA65_SIGNATURE_SEED_BYTES;
  out->method = sig_ml_dsa_65_method();
  out->asn1_method = &nistdsa_asn1_meth;
}

much like the existing kem.c file.

These structures will be incredibly useful for subsequent PRs, in which we will be adding de-randomized APIs to support FIPS validation and KATs from seeds. In effect, we will be extending the method to include:

DEFINE_LOCAL_DATA(NISTDSA_METHOD, sig_ml_dsa_65_method) {
  out->keygen = ml_dsa_65_keypair;
  out->keygen_internal = ml_dsa_65_keypair_internal;
  out->sign = ml_dsa_65_sign;
  out->sign_internal = ml_dsa_65_sign_internal;
  out->verify = ml_dsa_65_verify;
}

The directory dilithium/p_dilithium3.c is used to house generic PKEY operations, as well as NISTDSA specific EVP functions such as EVP_PKEY_nistdsa_set_params, EVP_PKEY_nistdsa_set_params, which work in a similar way to p_kem.c functions EVP_PKEY_CTX_kem_set_params and EVP_PKEY_kem_set_params.

The directory dilithium/p_dilithium3_asn1.c is used to house generic asn.1 operations for NIST FIPS DSA algorithms.

Rather than rename/relocate these files in this PR (which will convolute the CR), I will move these files in a subsequent PR.

2. Internal Design

We introduce new structures: NISTDSA_METHOD, NISTDSA, NISTDSA_KEY which are very similar to how both EC Crypto, and the KEM API is implemented. The figure above shows the newly proposed structures and their integration into the existing EVP structures.

NISTDSA_METHOD is a table of function pointers with 5 functions defined for a NISTDSA: key generation, key generation internal (for testing/FIPS), sign, sign internal (for testing/FIPS), and verify. Every NISTDSA implementation has to implement this API, for example sig_ml_dsa_44_method implements the three functions for ML-DSA-44 (analogous to EC_METHOD and for example, EC_GFp_nistz256_method).
NISTDSA is a structure that holds basic information about the DSA: the id, size of parameters, and the pointer to the implementation of the corresponding NISTDSA_METHOD.
NISTDSA_KEY structure is a helper structure that holds pointers to public and secret key and the pointer to the corresponding NISTDSA.
NISTDSA_PKEY_CTX is a helper structure used to store DSA parameters in an EVP_PKEY_CTX object (the same as EC_PKEY_CTX). Since NISTDSA has everything we need, that’s what we store in NISTDSA_PKEY_CTX.

3. OID changes

We now have real OIDs available from https://csrc.nist.gov/projects/computer-security-objects-register/algorithm-registration. These have been added to the obj files and automatically populated by go run objects.go. We include the following NIDs for ML-DSA:

//2.16.840.1.101.3.4.3.17
static const uint8_t kOIDMLDSA44[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x11};
//2.16.840.1.101.3.4.3.18
static const uint8_t kOIDMLDSA65[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x12};
//2.16.840.1.101.3.4.3.19
static const uint8_t kOIDMLDSA87[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x13};

and a "top-level" NID EVP_PKEY_NISTDSA (similar to EVP_PKEY_KEM to subset all NIST FIPS DSA algorithms. Much like for the KEM API, we include functions SIG_find_dsa_by_nid(int nid) to return actual algorithm specific methods.

4. Testing structure changes

Prior to this PR all testing for ML-DSA-65 was done as a series of g-tests. As we now have the ability to get algorithm/security level specific parameters from the NISTDSA struct, I have overhauled the testing suite to be parameterized over the currently supported signature algorithms:

static const struct ML_DSA parameterSet[] = {
  {"MLDSA44", NID_MLDSA44, 1312, 2560, 2420,  "dilithium/kat/mldsa44.txt", mldsa44kPublicKey, mldsa44kPublicKeySPKI, 1333},
  {"MLDSA65", NID_MLDSA65, 1952, 4032, 3309,  "dilithium/kat/mldsa65.txt", mldsa65kPublicKey, mldsa65kPublicKeySPKI, 1973},
  {"MLDSA87", NID_MLDSA87, 2592, 4896, 4627,  "dilithium/kat/mldsa87.txt", mldsa87kPublicKey, mldsa87kPublicKeySPKI, 2613},
};

For each of the above parameter sets we run:

TEST_P(MLDSAParameterTest, KAT)
TEST_P(MLDSAParameterTest, KeyGen)
TEST_P(MLDSAParameterTest, KeyCmp) 
TEST_P(MLDSAParameterTest, KeySize) 
TEST_P(MLDSAParameterTest, NewKeyFromBytes) 
TEST_P(MLDSAParameterTest, RawFunctions) 
TEST_P(MLDSAParameterTest, SIGOperations) 
TEST_P(MLDSAParameterTest, MarshalParse)

5. X.509 changes

Changes to X.509 code have been minimized to only the required changes to support this PR. This is already a big PR and I want to avoid expansion wherever possible. For X.509 the changes predominantly are regarding the change of NIDs from the old Dilithium NIDs to the new NIST DSA NIDs/OIDs.

As the OIDs changed, so too much the test certificates kDilithium3Cert, kDilithium3CertNull, and kDilithium3CertParam. These have been regenerated using the following:

Tool used https://github.com/google/der-ascii

1. First generate a valid test certificate in PEM encoding, say cert.pem
2. Convert PEM to DER: openssl x509 -in cert.pem -out cert.der -outform DER
3. Convert DER to ASCII: der2ascii -i cert.der >> cert.txt
4. Make edits to cert.txt as desired, say; certnew.txt
5. Convert ASCII to DER: ascii2der -i certnew.txt >> certnew.der
6. Convert DER to PEM: openssl x509 -in certnew.der -inform DER -out certnew.pem -outform PEM

6. Speed Tool

ML-DSA-44 and ML-DSA-87 have been added to the speed tool. To facilitate the new APIs used in this PR, the API version has been incremented by 1. The speed tool now produces benchmarks for all three parameter levels, example output:

Did 4870 MLDSA44 keygen operations in 1048820us (4643.3 ops/sec)
Did 1230 MLDSA44 signing operations in 1012232us (1215.1 ops/sec)
Did 4840 MLDSA44 verify operations in 1093069us (4427.9 ops/sec)
Did 2871 MLDSA65 keygen operations in 1051869us (2729.4 ops/sec)
Did 774 MLDSA65 signing operations in 1039468us (744.6 ops/sec)
Did 2893 MLDSA65 verify operations in 1089059us (2656.4 ops/sec)
Did 1600 MLDSA87 keygen operations in 1008766us (1586.1 ops/sec)
Did 654 MLDSA87 signing operations in 1179044us (554.7 ops/sec)
Did 1650 MLDSA87 verify operations in 1049198us (1572.6 ops/sec)

Callouts:

dilithium/p_dilithium3.call dilithium3 pkey specific functionality is now generic
dilithium/p_dilithium3_asn1.c all dilithium3 asn1 specific functionality is now generic
dilithium/p_dilithium_test.cc all dilithium3 testing specific functionality is now generic
dilithium/internal.h is a new file, added to maintain consistency with fipsmodule/kem/internal.h

Testing:

See above for description of changes to testing framework.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

codecov-commenter · 2024-10-25T17:37:17Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.75%. Comparing base (b77a698) to head (465d53f).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1949      +/-   ##
==========================================
+ Coverage   78.68%   78.75%   +0.06%     
==========================================
  Files         585      590       +5     
  Lines      100854   101428     +574     
  Branches    14298    14383      +85     
==========================================
+ Hits        79357    79876     +519     
- Misses      20863    20917      +54     
- Partials      634      635       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

dkostic · 2024-11-01T17:54:05Z

crypto/dilithium/internal.h

+  int (*sign)(uint8_t *sig, size_t *sig_len,
+              const uint8_t *message,
+              size_t message_len,
+              const uint8_t *ctx,


what's ctx?

ctx was added to all NIST PQC candidates in the final version of the FIPS standards. It is a context string (byte string 255 or fewer bytes) that is used as additional input when signing. See page 17 of FIPS 204.

dkostic · 2024-11-01T17:56:31Z

crypto/dilithium/internal.h

+extern "C" {
+#endif
+
+// NISTDSA_METHOD structure and helper functions.


I think we should change the name NISTDSA, maybe to NIST_PQDSA or something like that.

Happy to change the name -- originally wanted to use DSA but it was taken. I chose to add NIST because of the consistency between the FIPS 204, 205 standards, it seems as though any subsequent signature scheme standardizaed by NIST should follow these conventions. I didn't want to limit this to only signature schemes aimed towards PQ-secure use-cases.

dkostic · 2024-11-01T17:57:36Z

crypto/dilithium/internal.h

+  size_t keygen_seed_len;
+  size_t sign_seed_len;
+  const NISTDSA_METHOD *method;
+  const EVP_PKEY_ASN1_METHOD *asn1_method;


why do you need the asn1 method within this struct?

Good catch, not used, fixed in 465d53f

dkostic · 2024-11-01T17:58:11Z

crypto/dilithium/kat/mldsa44.txt

where are the KATs from?

These KATs are from the official upstream repository (https://github.com/pq-crystals/dilithium). They were generated the same way the ML-DSA-65 KATs were generated, from the reference.

dkostic · 2024-11-01T17:59:11Z

crypto/dilithium/p_dilithium3.c

-  if (key->priv == NULL) {
-    goto err;
-  }
+static const uint8_t kOIDMLDSA44[]  = {0x60, 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x03, 0x11};


please add reference for these ids

added in 465d53f

jakemas · 2024-11-04T21:11:19Z

Splitting this PR up further, first part adds the functionality to support multiple parameter sets of ML-DSA and other NIST PQC DSA (#1963). The second PR will add ML-DSA-44 and ML-DSA-87.

Added ML-DSA-44 and ML-DSA-87

819effc

jakemas mentioned this pull request Oct 25, 2024

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1948

Closed

EVP_PKEY_NISTDSA not defined when enable_dilithium=off

73316b0

fix NID in speed tool

63b8965

jakemas marked this pull request as ready for review October 25, 2024 20:05

jakemas requested a review from a team as a code owner October 25, 2024 20:05

jakemas added 4 commits October 25, 2024 14:12

moved EVP_PKEY helper functions to evp.h so speed can access them

de38d74

Merge branch 'main' into NISTDSA

2997995

moved nistdsa struc to evp_extra

feb1254

Merge branch 'NISTDSA' of github.com:jakemas/aws-lc into NISTDSA

38428aa

justsmth requested review from dkostic and nebeid October 29, 2024 12:54

jakemas added 3 commits October 29, 2024 08:55

added ml-dsa-44 and ml-dsa-87 to benchmark suite

3b5b6c5

removed include

1d15a90

increase API version for speed tool

cf5ae3f

dkostic reviewed Nov 1, 2024

View reviewed changes

removed unused asn1 method and added documentation

465d53f

jakemas closed this Nov 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

jakemas commented Oct 25, 2024 •

edited

Loading

codecov-commenter commented Oct 25, 2024 •

edited

Loading

dkostic Nov 1, 2024

jakemas Nov 4, 2024

dkostic Nov 1, 2024

jakemas Nov 4, 2024

dkostic Nov 1, 2024

jakemas Nov 4, 2024

dkostic Nov 1, 2024

jakemas Nov 4, 2024

dkostic Nov 1, 2024

jakemas Nov 4, 2024

jakemas commented Nov 4, 2024

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

Adding NISTDSA API to support ML-DSA-44 and ML-DSA-87 #1949

Conversation

jakemas commented Oct 25, 2024 • edited Loading

Issues:

Description of changes:

1. PKEY structure changes

2. Internal Design

3. OID changes

4. Testing structure changes

5. X.509 changes

6. Speed Tool

Callouts:

Testing:

codecov-commenter commented Oct 25, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakemas commented Nov 4, 2024

jakemas commented Oct 25, 2024 •

edited

Loading

codecov-commenter commented Oct 25, 2024 •

edited

Loading