CI test failure: CryptographicException : Error occurred during a cryptographic operation.

[jkotas]

    System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_AllocatingSpan.RsaCryptRoundtrip_OaepSHA3_256 [FAIL]
      System.Security.Cryptography.CryptographicException : Error occurred during a cryptographic operation.
      Stack Trace:
        /_/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.EvpPkey.Rsa.cs(42,0): at Interop.Crypto.RsaGenerateKey(Int32 keySize)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSAOpenSsl.cs(726,0): at System.Security.Cryptography.RSAOpenSsl.GenerateKey()
        /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs(323,0): at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs(329,0): at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
        /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs(347,0): at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Lazy.cs(427,0): at System.Lazy`1.CreateValue()
        /_/src/libraries/Common/src/System/Security/Cryptography/RSAOpenSsl.cs(714,0): at System.Security.Cryptography.RSAOpenSsl.GetKey()
        /_/src/libraries/Common/src/System/Security/Cryptography/RSAOpenSsl.cs(237,0): at System.Security.Cryptography.RSAOpenSsl.TryEncrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSAWrapper.cs(81,0): at System.Security.Cryptography.RSAWrapper.TryEncrypt(ReadOnlySpan`1 data, Span`1 destination, RSAEncryptionPadding padding, Int32& bytesWritten)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(172,0): at System.Security.Cryptography.RSA.<Encrypt>g__TryWithEncrypt|15_0(RSA rsa, ReadOnlySpan`1 input, Byte[] destination, RSAEncryptionPadding padding, Int32& bytesWritten)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(1426,0): at System.Security.Cryptography.RSA.TryWithKeyBuffer[TState](ReadOnlySpan`1 input, TState state, TryFunc`1 callback, Boolean tryKeySizeFirst)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(175,0): at System.Security.Cryptography.RSA.Encrypt(ReadOnlySpan`1 data, RSAEncryptionPadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.netcoreapp.cs(42,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_AllocatingSpan.Encrypt(RSA rsa, Byte[] data, RSAEncryptionPadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(350,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip(RSAEncryptionPadding paddingMode, Boolean expectSuccess)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(322,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.RsaCryptRoundtrip_OaepSHA3_256()

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=708999
Build error leg or test failing: System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_AllocatingSpan.RsaCryptRoundtrip_OaepSHA3_256
Pull request: #103413

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": [ "Error occurred during a cryptographic operation.", "RsaGenerateKey" ],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=708999
Error message validated: [Error occurred during a cryptographic operation. RsaGenerateKey]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 6/16/2024 5:12:38 AM UTC

Report

Build	Definition	Test	Pull Request
791140	dotnet/runtime	System.Security.Cryptography.Xml.Tests.RSAKeyValueTest.GetXml_SameRsa	#107083

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[vcsjones]

RSA key generation is failing, but nothing is in the OpenSSL error queue.

This one is a bit difficult to diagnose since we are not getting a meaningful error back from OpenSSL, so we throw the default exception.

I suppose we could make the shim more specific about which of the OpenSSL functions is failing by returning different values.