RSA 8192 is supported only for verification, not for signing

[dns2utf8]

Hi

I can not load 8k RSA keys with rustls.
Since there is no size limit in the source code that I can see, is it possible that the buffer exceeds a stack frame somewhere?
If so how can I work around that? I am totally fine with allowing allocations to the heap if it solves the problem.

Cheers,
Stefan

[briansmith]

The maximum size of the public modulus for private key operations is 4096. The maximum key size for public key operations is 4096 bits. In theory we could raise it but it would bloat the stack size requirement for everybody if we did, so I'm inclined not to unless there's a really strong motivation. I recommend you switch to a 4096-bit or smaller key.

[dns2utf8]

Hm, this is not ideal. Is this documented yet?
Would it be possible to switch to a heap allocation with a feature flag?

[josephlr]

Is this documented yet?

https://docs.rs/ring/0.14.6/ring/signature/struct.RsaKeyPair.html "The public modulus must be no larger than 4096 bits."

Would it be possible to switch to a heap allocation with a feature flag?

The current Cargo feature that causes heap allocation to be used is use_heap. However, the eventual goal is to have RSA verification not require this feature (see #102).

Note modulus limits differ between signing and verifying, but the limit is actually higher for verification. The max size for public key operations (like verification) that use the stack is 8192 bits, while the max size for private key operations (like singing) that use the heap is 4096 bits. I don't know the reason for this disparity, I would have thought it would be the other way.

In theory we could raise it but it would bloat the stack size requirement for everybody if we did

I don't think this is true. Raising PRIVATE_KEY_PUBLIC_MODULUS_MAX_BITS to 8192 would only affect signing operations, which currently use the heap.

so I'm inclined not to unless there's a really strong motivation. I recommend you switch to a 4096-bit or smaller key.

Agreed, I don't know of any real use case for RSA keys greater than 4096 bits. We should make the key limits the same regardless, it doesn't make a whole lot of sense for ring to verity signatures it cannot create. So we should either:

• Raise PRIVATE_KEY_PUBLIC_MODULUS_MAX_BITS to 8192
  • Would increase flexibility
  • No real security win
• Lower PUBLIC_KEY_PUBLIC_MODULUS_MAX_LEN to 4096
  • It's a reasonable upper limit
  • Would save stack space
  • External API surface would have to change (example).

@briansmith given the important work done and in-progress for #219 and #102, we should decide exactly what RSA key lengths to support across the board.

[briansmith]

In theory we could raise it but it would bloat the stack size requirement for everybody if we did

I don't think this is true. Raising PRIVATE_KEY_PUBLIC_MODULUS_MAX_BITS to 8192 would only affect signing operations, which currently use the heap.

I mean, it would affect the stack size requirements for everybody doing signing with RSA.

We need to find a strong use case to justify adding RSA 8192 signing support, and probably a commercial sponsor for that effort. It would have a big effect on the fundamental implementation of RSA signing--not just the current implementation, but the much improved implementation that's coming at some unspecified point in time in the future.