Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib #53504

Closed
effyteva opened this issue May 31, 2021 · 12 comments

Comments

@effyteva
Copy link

effyteva commented May 31, 2021

Hi,

We've started receiving some very odd exceptions on our production environment today, which took our production application down for a short while.
We're using the latest ASP.NET 5.0 docker image (Debian). Our software is updated many times a week, and don't believe this has anything to do with the latest ASP.NET core release.

"Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib. This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names. Resource name: Argument_InvalidResourceCultureName"

We're not sure what caused this or how to reproduce this issue, but it looks as if some incoming JSON managed to take down the entire application.

The relevant stack trace:

Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib.  This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names.  Resource name: Argument_InvalidResourceCultureName
   at System.Environment.FailFast(System.String)
   at System.SR.InternalGetResourceString(System.String)
   at System.SR.GetResourceString(System.String, System.String)
   at System.Globalization.CultureInfo.VerifyCultureName(System.String, Boolean)
   at System.Resources.ResourceManager.GetResourceFileName(System.Globalization.CultureInfo)
   at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo, System.Collections.Generic.Dictionary`2<System.String,System.Resources.ResourceSet>, Boolean, Boolean)
   at System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean)
   at System.Resources.ResourceManager.GetString(System.String, System.Globalization.CultureInfo)
   at System.SR.InternalGetResourceString(System.String)
   at System.SR.GetResourceString(System.String, System.String)
   at System.Globalization.CultureInfo.VerifyCultureName(System.String, Boolean)
   at System.Resources.ResourceManager.GetResourceFileName(System.Globalization.CultureInfo)
   at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo, System.Collections.Generic.Dictionary`2<System.String,System.Resources.ResourceSet>, Boolean, Boolean)
   at System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean)
   at System.Resources.ResourceManager.GetString(System.String, System.Globalization.CultureInfo)
   at System.SR.InternalGetResourceString(System.String)
   at System.SR.GetResourceString(System.String, System.String)
   at System.Convert.FromBase64CharPtr(Char*, Int32)
   at System.Convert.FromBase64CharArray(Char[], Int32, Int32)
   at Newtonsoft.Json.JsonTextReader.ParseReadString(Char, Newtonsoft.Json.ReadType)
   at Newtonsoft.Json.JsonTextReader.ReadAsBytes()
   at Newtonsoft.Json.JsonReader.ReadForType(Newtonsoft.Json.Serialization.JsonContract, Boolean)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(System.Object, Newtonsoft.Json.JsonReader, Newtonsoft.Json.Serialization.JsonObjectContract, Newtonsoft.Json.Serialization.JsonProperty, System.String)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(Newtonsoft.Json.JsonReader, System.Type, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty, System.Object)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(Newtonsoft.Json.JsonReader, System.Type, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty, System.Object)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(Newtonsoft.Json.JsonReader, System.Type, Boolean)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(Newtonsoft.Json.JsonReader, System.Type)
   at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+<ReadRequestBodyAsync>d__12.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+<ReadRequestBodyAsync>d__12, Microsoft.AspNetCore.Mvc.NewtonsoftJson, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<ReadRequestBodyAsync>d__12 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+<ReadRequestBodyAsync>d__12, Microsoft.AspNetCore.Mvc.NewtonsoftJson, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<ReadRequestBodyAsync>d__12 ByRef)
   at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter.ReadRequestBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext, System.Text.Encoding)
   at Microsoft.AspNetCore.Mvc.Formatters.TextInputFormatter.ReadRequestBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext)
   at Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+<BindModelAsync>d__11.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+<BindModelAsync>d__11, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__11 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+<BindModelAsync>d__11, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__11 ByRef)
   at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(Microsoft.AspNetCore.Mvc.ModelBinding.ModelBindingContext)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__8 ByRef)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1[[Microsoft.AspNetCore.Mvc.ModelBinding.ModelBindingResult, Microsoft.AspNetCore.Mvc.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__8 ByRef)
   at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(Microsoft.AspNetCore.Mvc.ActionContext, Microsoft.AspNetCore.Mvc.ModelBinding.IModelBinder, Microsoft.AspNetCore.Mvc.ModelBinding.IValueProvider, Microsoft.AspNetCore.Mvc.Abstractions.ParameterDescriptor, Microsoft.AspNetCore.Mvc.ModelBinding.ModelMetadata, System.Object, System.Object)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef)
   at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0.<CreateBinderDelegate>g__Bind|0(Microsoft.AspNetCore.Mvc.ControllerContext, System.Object, System.Collections.Generic.Dictionary`2<System.String,System.Object>)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.BindArgumentsAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync()
   at Microsoft.AspNetCore.Mvc.Routing.ActionEndpointFactory+<>c__DisplayClass7_0.<CreateRequestDelegate>b__0(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+<Invoke>d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.ReDoc, Version=6.1.4.0, Culture=neutral, PublicKeyToken=52663dc7937b1e15]](<Invoke>d__5 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.ReDoc, Version=6.1.4.0, Culture=neutral, PublicKeyToken=52663dc7937b1e15]](<Invoke>d__5 ByRef)
   at Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](<Invoke>d__5 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](<Invoke>d__5 ByRef)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](<Invoke>d__5 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](<Invoke>d__5 ByRef)
   at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4, Swashbuckle.AspNetCore.Swagger, Version=6.1.4.0, Culture=neutral, PublicKeyToken=62657d7474907593]](<Invoke>d__4 ByRef)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4, Swashbuckle.AspNetCore.Swagger, Version=6.1.4.0, Culture=neutral, PublicKeyToken=62657d7474907593]](<Invoke>d__4 ByRef)
   at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext, Swashbuckle.AspNetCore.Swagger.ISwaggerProvider)
   at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions+<>c__DisplayClass5_1.<UseMiddleware>b__2(Microsoft.AspNetCore.Http.HttpContext)
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label May 31, 2021
@ghost
Copy link

ghost commented Jun 1, 2021

Tagging subscribers to this area: @tannergooding
See info in area-owners.md if you want to be subscribed.

Issue Details

Hi,

We've started receiving some very odd exceptions on our production environment today, which took our production application down for a short while.
We're using the latest ASP.NET 5.0 docker image (Debian). Our software is updated many times a week, and don't believe this has anything to do with the latest ASP.NET core release.

"Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib. This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names. Resource name: Argument_InvalidResourceCultureName"

We're not sure what caused this or how to reproduce this issue, but it looks as if some incoming JSON managed to take down the entire application.

The relevant stack trace:
Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib. This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names. Resource name: Argument_InvalidResourceCultureName at System.Environment.FailFast(System.String) at System.SR.InternalGetResourceString(System.String) at System.SR.GetResourceString(System.String, System.String) at System.Globalization.CultureInfo.VerifyCultureName(System.String, Boolean) at System.Resources.ResourceManager.GetResourceFileName(System.Globalization.CultureInfo) at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo, System.Collections.Generic.Dictionary2<System.String,System.Resources.ResourceSet>, Boolean, Boolean)
at System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean)
at System.Resources.ResourceManager.GetString(System.String, System.Globalization.CultureInfo)
at System.SR.InternalGetResourceString(System.String)
at System.SR.GetResourceString(System.String, System.String)
at System.Globalization.CultureInfo.VerifyCultureName(System.String, Boolean)
at System.Resources.ResourceManager.GetResourceFileName(System.Globalization.CultureInfo)
at System.Resources.ManifestBasedResourceGroveler.GrovelForResourceSet(System.Globalization.CultureInfo, System.Collections.Generic.Dictionary2<System.String,System.Resources.ResourceSet>, Boolean, Boolean) at System.Resources.ResourceManager.InternalGetResourceSet(System.Globalization.CultureInfo, Boolean, Boolean) at System.Resources.ResourceManager.GetString(System.String, System.Globalization.CultureInfo) at System.SR.InternalGetResourceString(System.String) at System.SR.GetResourceString(System.String, System.String) at System.Convert.FromBase64CharPtr(Char*, Int32) at System.Convert.FromBase64CharArray(Char[], Int32, Int32) at Newtonsoft.Json.JsonTextReader.ParseReadString(Char, Newtonsoft.Json.ReadType) at Newtonsoft.Json.JsonTextReader.ReadAsBytes() at Newtonsoft.Json.JsonReader.ReadForType(Newtonsoft.Json.Serialization.JsonContract, Boolean) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(System.Object, Newtonsoft.Json.JsonReader, Newtonsoft.Json.Serialization.JsonObjectContract, Newtonsoft.Json.Serialization.JsonProperty, System.String) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(Newtonsoft.Json.JsonReader, System.Type, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty, System.Object) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(Newtonsoft.Json.JsonReader, System.Type, Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json.Serialization.JsonProperty, Newtonsoft.Json.Serialization.JsonContainerContract, Newtonsoft.Json.Serialization.JsonProperty, System.Object) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(Newtonsoft.Json.JsonReader, System.Type, Boolean) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(Newtonsoft.Json.JsonReader, System.Type) at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+<ReadRequestBodyAsync>d__12.MoveNext() at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+<ReadRequestBodyAsync>d__12, Microsoft.AspNetCore.Mvc.NewtonsoftJson, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<ReadRequestBodyAsync>d__12 ByRef) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Start[[Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter+d__12, Microsoft.AspNetCore.Mvc.NewtonsoftJson, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](d__12 ByRef)
at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonInputFormatter.ReadRequestBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext, System.Text.Encoding)
at Microsoft.AspNetCore.Mvc.Formatters.TextInputFormatter.ReadRequestBodyAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext)
at Microsoft.AspNetCore.Mvc.Formatters.InputFormatter.ReadAsync(Microsoft.AspNetCore.Mvc.Formatters.InputFormatterContext)
at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+d__11.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+d__11, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](d__11 ByRef)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder+d__11, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](d__11 ByRef)
at Microsoft.AspNetCore.Mvc.ModelBinding.Binders.BodyModelBinder.BindModelAsync(Microsoft.AspNetCore.Mvc.ModelBinding.ModelBindingContext)
at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+d__8.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+d__8, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](d__8 ByRef)
at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder1[[Microsoft.AspNetCore.Mvc.ModelBinding.ModelBindingResult, Microsoft.AspNetCore.Mvc.Abstractions, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]].Start[[Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder+<BindModelAsync>d__8, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<BindModelAsync>d__8 ByRef) at Microsoft.AspNetCore.Mvc.ModelBinding.ParameterBinder.BindModelAsync(Microsoft.AspNetCore.Mvc.ActionContext, Microsoft.AspNetCore.Mvc.ModelBinding.IModelBinder, Microsoft.AspNetCore.Mvc.ModelBinding.IValueProvider, Microsoft.AspNetCore.Mvc.Abstractions.ParameterDescriptor, Microsoft.AspNetCore.Mvc.ModelBinding.ModelMetadata, System.Object, System.Object) at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d.MoveNext() at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0+<<CreateBinderDelegate>g__Bind|0>d, Microsoft.AspNetCore.Mvc.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60]](<<CreateBinderDelegate>g__Bind|0>d ByRef) at Microsoft.AspNetCore.Mvc.Controllers.ControllerBinderDelegateProvider+<>c__DisplayClass0_0.<CreateBinderDelegate>g__Bind|0(Microsoft.AspNetCore.Mvc.ControllerContext, System.Object, System.Collections.Generic.Dictionary2<System.String,System.Object>)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.BindArgumentsAsync()
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter()
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State ByRef, Scope ByRef, System.Object ByRef, Boolean ByRef)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync()
at Microsoft.AspNetCore.Mvc.Routing.ActionEndpointFactory+<>c__DisplayClass7_0.b__0(Microsoft.AspNetCore.Http.HttpContext)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+d__5.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+d__5, Swashbuckle.AspNetCore.ReDoc, Version=6.1.4.0, Culture=neutral, PublicKeyToken=52663dc7937b1e15]](d__5 ByRef)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware+d__5, Swashbuckle.AspNetCore.ReDoc, Version=6.1.4.0, Culture=neutral, PublicKeyToken=52663dc7937b1e15]](d__5 ByRef)
at Swashbuckle.AspNetCore.ReDoc.ReDocMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](d__5 ByRef)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](d__5 ByRef)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](d__5 ByRef)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+d__5, Swashbuckle.AspNetCore.SwaggerUI, Version=6.1.4.0, Culture=neutral, PublicKeyToken=4232c99127b3c254]](d__5 ByRef)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Microsoft.AspNetCore.HttpOverrides.ForwardedHeadersMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+d__4.MoveNext()
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+d__4, Swashbuckle.AspNetCore.Swagger, Version=6.1.4.0, Culture=neutral, PublicKeyToken=62657d7474907593]](d__4 ByRef)
at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+d__4, Swashbuckle.AspNetCore.Swagger, Version=6.1.4.0, Culture=neutral, PublicKeyToken=62657d7474907593]](d__4 ByRef)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext, Swashbuckle.AspNetCore.Swagger.ISwaggerProvider)
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions+<>c__DisplayClass5_1.b__2(Microsoft.AspNetCore.Http.HttpContext)
`

Author: effyteva
Assignees: -
Labels:

area-System.Runtime, untriaged

Milestone: -

@tannergooding
Copy link
Member

CC. @tarekgh, it looks like this is throwing some other exception and then that this line itself fails because cultureName was determined to be invalid, but we reuse it in the argument exception which leads to infinite recusion: https://source.dot.net/#System.Private.CoreLib/CultureInfo.cs,353

@tarekgh
Copy link
Member

tarekgh commented Jun 1, 2021

@tannergooding this failure usually indicate something else seriously failed and the real problem is not the resource lookup. Here is some example of that grpc/grpc-dotnet#1193. I don't think such issue will be actionable till we get a debugger dump.

@tannergooding
Copy link
Member

@tannergooding this failure usually indicate something else seriously failed and the real problem is not the resource lookup.

Right, but we are also masking the actual exception by throwing this one. I feel like we need to at least fix this so its not a guaranteed failure when we get to this state and to also preferably provide better information about what the original resource failure was.

That will help make these issues easier to diagnose in the future, since we'll get the actual failure shown.

@effyteva
Copy link
Author

effyteva commented Jun 1, 2021

Thanks guys, I strongly agree with @tannergooding,
The fact the cultureName was somehow determined to be invalid shouldn't have to crash the process. The exception should probably fallback to English exception instead of crashing.
I believe this was due to some hacking attempt we suffered from, we're not sure yet how an invalid Culture was set, but this seems like a potential major security threat, doesn't it?

@tarekgh
Copy link
Member

tarekgh commented Jun 1, 2021

We already know from the stack that System.Convert.FromBase64CharPtr(Char*, Int32) is the part throwing the exception https://source.dot.net/#System.Private.CoreLib/Convert.cs,2778. This exception will try to use CultureInfo.CurrentUICulture to get the resources which is eventually throw from https://source.dot.net/#System.Private.CoreLib/CultureInfo.cs,353. I don't know even why the resources trying to validate the culture name while the name is coming from the CurrentUICulture. I guess maybe doing that as we'll try to open the satellite assembly with that name.

@effyteva is there a way in that container you can dump the value of CultureInfo.CurrentUICulture.Name very early before executing the rest of the stack?

The exception should probably fallback to English exception instead of crashing.

And how you'll get the English exception in the first place? it is still in the resources. we know where the exception is thrown already.

@effyteva
Copy link
Author

effyteva commented Jun 1, 2021

@tarekgh
Sadly I couldn't reproduce the issue, and (thankfully) it stopped after hitting our prod environment for more than an hour.
Regarding the exception message, probably even hard-coded string would be better than attempting to access an invalid Resource.
Perhaps adding a minor check before throwing the exception, to check if cultureName = CultureInfo.CurrentUICulture.Name? And only in that case - throw the hard-coded text message?

@effyteva
Copy link
Author

effyteva commented Jun 1, 2021

BTW, How do you think an invalid CultureName managed to be set into the CultureInfo.CurrentUICulture?

@tarekgh
Copy link
Member

tarekgh commented Jun 1, 2021

BTW, how do you think an invalid CultureName managed to be set into the CultureInfo.CurrentUICulture?

We depend on the ICU library to read the default Culture.

uloc_getBaseName(defaultLocale, localeNameBuffer, ULOC_FULLNAME_CAPACITY, &status);
Could be the ICU API returning unexpected value which we just trusted without doing more validation on it. We may consider adding this validation there too.

Is there a way from your side to add more logging so if you hit this issue again, we can have more info?

@effyteva
Copy link
Author

effyteva commented Jun 1, 2021

I think so,
I should be able to easily call a cloned version of the VerifyCultureName method, and check if the CurrentUICulture is invalid on any request, then log it to figure out what was the invalid CultureName.

@tarekgh tarekgh added intermittent and removed untriaged New issue has not been triaged by the area owner labels Jun 1, 2021
@tarekgh tarekgh added this to the Future milestone Jun 1, 2021
@ghost ghost added the no-recent-activity label Jun 15, 2021
@ghost
Copy link

ghost commented Jun 15, 2021

This issue has been automatically marked no recent activity because it has been marked as needs author feedback but has not had any activity for 14 days. It will be closed if no further activity occurs within 7 more days. Any new comment (by anyone, not necessarily the author) will remove no recent activity

@ghost ghost closed this as completed Jun 23, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jul 23, 2021
@ghost ghost removed the no-recent-activity label Oct 5, 2021
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants