CI failure in Rsa.Tests.RSAKeyExchangeFormatterTests.VerifyDecryptKeyExchangePkcs1

[danmoseley]

https://ci.dot.net/job/dotnet_corefx/job/master/job/portablelinux_debug_prtest/4738/testReport/junit/System.Security.Cryptography.Rsa.Tests/RSAKeyExchangeFormatterTests/VerifyDecryptKeyExchangePkcs1/

Stacktrace

MESSAGE:
Assert.Throws() Failure\nExpected: typeof(System.Security.Cryptography.CryptographicException)\nActual: (No exception was thrown)
+++++++++++++++++++
STACK TRACE:
at System.Security.Cryptography.Rsa.Tests.RSAKeyExchangeFormatterTests.VerifyDecryptKeyExchange(AsymmetricKeyExchangeFormatter formatter, AsymmetricKeyExchangeDeformatter deformatter) in /mnt/resource/j/workspace/dotnet_corefx/master/portablelinux_debug_prtest/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/RSAKeyExchangeFormatter.cs:line 80 at System.Security.Cryptography.Rsa.Tests.RSAKeyExchangeFormatterTests.VerifyDecryptKeyExchangePkcs1() in /mnt/resource/j/workspace/dotnet_corefx/master/portablelinux_debug_prtest/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/RSAKeyExchangeFormatter.cs:line 35

[safern]

@bartonjs

[safern]

Failed again in:
https://ci.dot.net/job/dotnet_corefx/job/master/job/windows_nt_debug_prtest/7717/testReport/junit/System.Security.Cryptography.Rsa.Tests/RSAKeyExchangeFormatterTests/VerifyDecryptKeyExchangePkcs1/

[bartonjs]

I marked this as 2.0 because we should know why this started failing. I can't think of any relevant changes we'd've done, and the test looks fairly static.

[danmoseley]

@bartonjs do you have time to look today Jeremy?

@karelz

[safern]

I can grab and investigate this one.

[safern]

Trying to get a repro, wasn't successful after 400 runs on Windows, will try now in portable linux.

[bartonjs]

I've been poking at this throughout the day, and haven't seen failures, either. I'm at commit 4ab9f71 (the last one before the test was disabled), and currently have the test running in a tight loop under COMPlus_GCStress=3. No sign of failure.

It's weird that it happened, and super weird that it happened twice...

I wish I could easily find out what commit those runs had been on.

[safern]

And the logs for the failures are deleted :(

[bartonjs]

I'm leaning toward "no bug", and that we should just turn this test back on in master... maybe once we're forked.

[bartonjs]

With the test modified to run 1,000,000 iterations, and print a '.' every 1000, running under GCStress=3, I'm getting nowhere:

$ COMPlus_GCStress=3 ./dotnet xunit.console.netcore.exe System.Security.Cryptography.OpenSsl.Tests.dll -method System.Security.Cryptography.Rsa.Tests.RSAKeyExchangeFormatterTests.VerifyDecryptKeyExchangePkcs1
xUnit.net console test runner (64-bit .NET Core)
Copyright (C) 2014 Outercurve Foundation.

Discovering: System.Security.Cryptography.OpenSsl.Tests
Discovered:  System.Security.Cryptography.OpenSsl.Tests
Starting:    System.Security.Cryptography.OpenSsl.Tests
..................................

That's more than 33k runs under GCStress (and still going). So... solar flares? Illuminati conspiracy?

[karelz]

Can we add more logging / hard crash upon error and let it repro again in CI?

[bartonjs]

Better reproducibility/diagnosibility seems fair. I'll do that right now (unless there's something different you think, @safern?)

[safern]

I think logging is a good alternative as I ran the tests for 2000 in windows and 2000 iterations in Ubuntu with -portable parameter and haven't seen any failure yet. Thanks for putting the PR @bartonjs