Jose.JoseException: Unable to sign content

[binoypatel]

Description

Hi there,

After upgrading to .net 8.0, I am getting above error, I am using ES256 algorithm to sign the JWT using jose-jwt library

here is a full stack:

[17:12:14 ERR] HTTP POST /api/v1/auth/login responded 500 in 291.7296 ms
Jose.JoseException: Unable to sign content.
---> Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error 100000 - CSSM Exception: 100000 UNIX[Undefined error: 0])
at Interop.AppleCrypto.NativeCreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm) at Interop.AppleCrypto.CreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm)
at System.Security.Cryptography.ECDsaImplementation.ECDsaSecurityTransforms.SignHash(Byte[] hash)
at System.Security.Cryptography.ECDsa.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm)
at Jose.netstandard1_4.EcdsaUsingSha.Sign(ECDsa privateKey, Byte[] securedInput)
at Jose.netstandard1_4.EcdsaUsingSha.Sign(Byte[] securedInput, Object key)
--- End of inner exception stack trace ---
at Jose.netstandard1_4.EcdsaUsingSha.Sign(Byte[] securedInput, Object key)
at Jose.JWT.EncodeBytes(Byte[] payload, Object key, JwsAlgorithm algorithm, IDictionary2 extraHeaders, JwtSettings settings, JwtOptions options) at Jose.JWT.Encode(String payload, Object key, JwsAlgorithm algorithm, IDictionary2 extraHeaders, JwtSettings settings, JwtOptions options)
at Jose.JWT.Encode(Object payload, Object key, JwsAlgorithm algorithm, IDictionary`2 extraHeaders, JwtSettings settings, JwtOptions options)

I already checked #59703 but it didn't help. Kindly review and any help would be appreciated.

Thanks,
Binoy

Reproduction Steps

var payload = new Dictionary<string, object>
{
{ "iss", _jwtSettings.Issuer! },
{ "aud", _jwtSettings.Audience! },
{ "sub", userId },
{ ApplicationClaims.FullName, displayName! },
{ ApplicationClaims.ProductKind, productKind },
{ ApplicationClaims.Timezone, timezone },
{ "email", email },
{ ApplicationClaims.TenantId, tenantId },
{ "iat", DateTimeOffset.UtcNow.ToUnixTimeSeconds() },
{ "exp", DateTimeOffset.UtcNow.AddMinutes(_jwtSettings.Validity).ToUnixTimeSeconds() }
};
return Jose.JWT.Encode(payload, _privateKey, Jose.JwsAlgorithm.ES256);

Crashing when running JWT.Encode() methode

Expected behavior

It should sign the JWT and should not throw exception

Actual behavior

Throwing exception when calling JWT.Encode method:
Jose.JoseException: Unable to sign content.
---> Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error 100000 - CSSM Exception: 100000 UNIX[Undefined error: 0])

Regression?

No response

Known Workarounds

No response

Configuration

Runtime Environment:
OS Name: Mac OS X
OS Version: 14.1
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/8.0.100/

.NET workloads installed:
Workload version: 8.0.100-manifests.6c33ef20
There are no installed workloads to display.

Host:
Version: 8.0.0
Architecture: arm64
Commit: 5535e31

.NET SDKs installed:
8.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
x64 [/usr/local/share/dotnet/x64]
registered at [/etc/dotnet/install_location_x64]

Other information

No response

[ghost]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Hi there,

After upgrading to .net 8.0, I am getting above error, I am using ES256 algorithm to sign the JWT using jose-jwt library

here is a full stack:

[17:12:14 ERR] HTTP POST /api/v1/auth/login responded 500 in 291.7296 ms
Jose.JoseException: Unable to sign content.
---> Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error 100000 - CSSM Exception: 100000 UNIX[Undefined error: 0])
at Interop.AppleCrypto.NativeCreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm) at Interop.AppleCrypto.CreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm)
at System.Security.Cryptography.ECDsaImplementation.ECDsaSecurityTransforms.SignHash(Byte[] hash)
at System.Security.Cryptography.ECDsa.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm)
at Jose.netstandard1_4.EcdsaUsingSha.Sign(ECDsa privateKey, Byte[] securedInput)
at Jose.netstandard1_4.EcdsaUsingSha.Sign(Byte[] securedInput, Object key)
--- End of inner exception stack trace ---
at Jose.netstandard1_4.EcdsaUsingSha.Sign(Byte[] securedInput, Object key)
at Jose.JWT.EncodeBytes(Byte[] payload, Object key, JwsAlgorithm algorithm, IDictionary2 extraHeaders, JwtSettings settings, JwtOptions options) at Jose.JWT.Encode(String payload, Object key, JwsAlgorithm algorithm, IDictionary2 extraHeaders, JwtSettings settings, JwtOptions options)
at Jose.JWT.Encode(Object payload, Object key, JwsAlgorithm algorithm, IDictionary`2 extraHeaders, JwtSettings settings, JwtOptions options)

I already checked #59703 but it didn't help. Kindly review and any help would be appreciated.

Thanks,
Binoy

Reproduction Steps

var payload = new Dictionary<string, object>
{
{ "iss", _jwtSettings.Issuer! },
{ "aud", _jwtSettings.Audience! },
{ "sub", userId },
{ ApplicationClaims.FullName, displayName! },
{ ApplicationClaims.ProductKind, productKind },
{ ApplicationClaims.Timezone, timezone },
{ "email", email },
{ ApplicationClaims.TenantId, tenantId },
{ "iat", DateTimeOffset.UtcNow.ToUnixTimeSeconds() },
{ "exp", DateTimeOffset.UtcNow.AddMinutes(_jwtSettings.Validity).ToUnixTimeSeconds() }
};
return Jose.JWT.Encode(payload, _privateKey, Jose.JwsAlgorithm.ES256);

Crashing when running JWT.Encode() methode

Expected behavior

It should sign the JWT and should not throw exception

Actual behavior

Throwing exception when calling JWT.Encode method:
Jose.JoseException: Unable to sign content.
---> Interop+AppleCrypto+AppleCFErrorCryptographicException: The operation couldn’t be completed. (OSStatus error 100000 - CSSM Exception: 100000 UNIX[Undefined error: 0])

Regression?

No response

Known Workarounds

No response

Configuration

Runtime Environment:
OS Name: Mac OS X
OS Version: 14.1
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/8.0.100/

.NET workloads installed:
Workload version: 8.0.100-manifests.6c33ef20
There are no installed workloads to display.

Host:
Version: 8.0.0
Architecture: arm64
Commit: 5535e31

.NET SDKs installed:
8.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
x64 [/usr/local/share/dotnet/x64]
registered at [/etc/dotnet/install_location_x64]

Other information

No response

Author:	binoypatel
Assignees:	-
Labels:	area-System.Security
Milestone:	-

[vcsjones]

I attempted to reproduce the issue with the following code and was unable to.

using System;
using System.Collections.Generic;
using System.Security.Cryptography;

var payload = new Dictionary<string, object>
{
    { "iss", "kevin's console application" },
    { "aud", "production" },
    { "sub", "42" },
    { "iat", DateTimeOffset.UtcNow.ToUnixTimeSeconds() },
    { "exp", DateTimeOffset.UtcNow.AddMinutes(3600).ToUnixTimeSeconds() }
};

var privateKey = ECDsa.Create(ECCurve.NamedCurves.nistP256);
string encoded = Jose.JWT.Encode(payload, privateKey, Jose.JwsAlgorithm.ES256);
Console.WriteLine(encoded);

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="jose-jwt" Version="4.1.0" />
  </ItemGroup>
</Project>

Does that code crash for you as well?

A particular question comes to mind - how are you loading _privateKey that performs the signing? Is it coming from a certificate / PKCS12? Is it coming from a PEM on disk, etc.

Any additional information you can provide to help reproduce the issue would be greatly appreciated.

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[binoypatel]

Hi Kevin

Thanks for the updates. Here is how I am creating a private key:

_jwtSettings = jwtSettings;
var jweCertificateFile = Path.Combine(
AppDomain.CurrentDomain.BaseDirectory,
"jwe-cert.pfx"
);
using var certificate = new X509Certificate2(
jweCertificateFile,
"ruog^hKodBQozFWHL6"
);
_privateKey = certificate.GetECDsaPrivateKey()!;

Attaching the pfx file for reference. I am not sure but I am on macos 14.1.1 if that makes any difference.

Thanks,
Binoy
jwe-cert.pfx.zip

[vcsjones]

I can reproduce this now. It looks like we have a lifetime issue with the private key. Steps to reproduce:

using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

using ECDsa ca = ECDsa.Create(ECCurve.NamedCurves.nistP256);
CertificateRequest req = new("CN=potatos", ca, HashAlgorithmName.SHA256);
X509Certificate2 cert = req.CreateSelfSigned(DateTimeOffset.Now, DateTimeOffset.Now.AddDays(3));

X509Certificate2 loaded = new X509Certificate2(cert.Export(X509ContentType.Pkcs12, "carrots"), "carrots");
ECDsa signingKey = loaded.GetECDsaPrivateKey()!;

loaded.Dispose();
signingKey.SignHash(new byte[32]);

This throws for me. The ECDsa key's lifetime is not independent of the certificate.

@binoypatel you can work around this by not disposing of the X509Certificate2. In fact, you will need to keep it "alive" for as long as the _privateKey. Which is to say, do not put the certificate in a using, and assign it to a field like you did with _privateKey such as _certificate. You can dispose of the certificate whenever you dispose of the private key.

[binoypatel]

Works like a charm! Thank you so much Kevin, it works with the suggestion you made.

Kind regards,
Binoy

[vcsjones]

@binoypatel I am going to keep this open because this is not supposed to be happening. I think there is something that needs to be addressed for .NET 9 or even fixed for .NET 8. But I am glad you are unblocked for now.

[binoypatel]

Thanks Kevin, not an expert in this field but yes you are right using supposed to be working as in .net 7, and should be addressed in either.net 8 or .net 9.

[vcsjones]

I did a quick bisect and determined that this bug was introduced by 28f958d in PR #82205.

Using this unit test

[Fact]
public static void Repo94959()
{
    using ECDsa ca = ECDsa.Create(ECCurve.NamedCurves.nistP256);
    CertificateRequest req = new("CN=potatos", ca, HashAlgorithmName.SHA256);
    X509Certificate2 cert = req.CreateSelfSigned(DateTimeOffset.Now, DateTimeOffset.Now.AddDays(3));

    X509Certificate2 loaded = new X509Certificate2(cert.Export(X509ContentType.Pkcs12, "carrots"), "carrots");
    ECDsa signingKey = loaded.GetECDsaPrivateKey()!;

    loaded.Dispose();
    signingKey.SignHash(new byte[32]);
}

Before revert:

[xUnit.net 00:00:01.10]     System.Security.Cryptography.X509Certificates.Tests.CertTests.Repo94959 [FAIL]
  Failed System.Security.Cryptography.X509Certificates.Tests.CertTests.Repo94959 [196 ms]
  Error Message:
   Interop+AppleCrypto+AppleCFErrorCryptographicException : The operation couldn’t be completed. (OSStatus error -25294 - CSSM Exception: -2147413737 CSSMERR_DL_DATASTORE_DOESNOT_EXIST)
  Stack Trace:
     at Interop.AppleCrypto.NativeCreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan`1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm) in /Users/vcsjones/Projects/runtime/src/libraries/Common/src/Interop/OSX/System.Security.Cryptography.Native.Apple/Interop.SignVerify.cs:line 148
   at Interop.AppleCrypto.CreateSignature(SafeSecKeyRefHandle privateKey, ReadOnlySpan`1 dataHash, PAL_HashAlgorithm hashAlgorithm, PAL_SignatureAlgorithm signatureAlgorithm) in /Users/vcsjones/Projects/runtime/src/libraries/Common/src/Interop/OSX/System.Security.Cryptography.Native.Apple/Interop.SignVerify.cs:line 164
   at System.Security.Cryptography.ECDsaImplementation.ECDsaSecurityTransforms.SignHash(Byte[] hash) in /Users/vcsjones/Projects/runtime/src/libraries/Common/src/System/Security/Cryptography/ECDsaSecurityTransforms.cs:line 62
   at System.Security.Cryptography.X509Certificates.Tests.CertTests.Repo94959() in /Users/vcsjones/Projects/runtime/src/libraries/System.Security.Cryptography/tests/X509Certificates/CertTests.cs:line 39

Then do git revert -n 28f958de34be053bdebfca0b804cb6ac9232f4cc

After reverting the test passes.

/cc @filipnavara

[filipnavara]

The reference counting strikes again. I am not sure when I will be able to look into it. Having the test helps a lot though, thanks!

[vcsjones]

@filipnavara I will try to look at this in parallel with you. Just looping you in incase anything immediately came to mind.

[bartonjs]

I don't suppose Apple has given us a way to make SecIdentity(Ref) without a keychain, yet? :)

[filipnavara]

I don't suppose Apple has given us a way to make SecIdentity(Ref) without a keychain, yet? :)

Nope. There's a private API (SecIdentityCreate) to do that used by WebKit tests but nothing public.

PR #82205 fixed several reference counting mismatches which resulted both in extra decrements and extra increments, so this only worked by accident. All the AppleCertificatePal.Get[...]PrivateKey() methods create a SafeSecKeyRefHandle object for the private key through X509GetPrivateKeyFromIdentity method. This handle doesn't keep any reference to the _identityHandle or its keychain. macOS API will happily allow you to delete the keychain and have the SecKeyRef point to private key object that no longer behaves correctly. There are only two things that can be done about it:

• Pass the extra temporary keychain reference to SafeSecKeyRefHandle (or *SecurityTransforms) implementation. This makes the reference tracking even bigger mess than it already is, but it has relatively small overhead.
• Make a copy the private key through SecKeyCopyExternalRepresentation and SecKeyCreateWithData. This would need to be a special code path for temporary keychains since you are generally not guaranteed that the export is possible.

I would prefer the former.

[vcsjones]

Pass the extra temporary keychain reference to SafeSecKeyRefHandle (or *SecurityTransforms) implementation. This makes the reference tracking even bigger mess than it already is, but it has relatively small overhead.

This is more or less what we were doing before with the SecCertificateRef before that PR, right?

I would prefer the former.

It might be messy, but I feel that it would generally be more well understood and easier to reason about it. It would not require special casing on the key's exportability, either.

[filipnavara]

This is more or less what we were doing before with the SecCertificateRef before that PR, right?

SafeSecCertificateHandle still does that but the *SecurityTransforms returned from Get[...]PrivateKey() never had that reference. It only worked because some references were never released or released late by garbage collection.