Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.Net.Quic is leaking UnobservedTaskExceptions #80111

Closed
MihaZupan opened this issue Jan 3, 2023 · 8 comments · Fixed by #104894
Closed

System.Net.Quic is leaking UnobservedTaskExceptions #80111

MihaZupan opened this issue Jan 3, 2023 · 8 comments · Fixed by #104894
Assignees
Labels
area-System.Net.Quic bug in-pr There is an active PR which will close this issue when it is merged
Milestone

Comments

@MihaZupan
Copy link
Member

I have a console app with HttpClient that continuously sends requests to a localhost Kestrel instance.
If I randomly shut down and restart the server while HttpClient is making requests, I can see sporadic UnobservedTaskExceptions coming from System.Net.Quic.

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Operation aborted.)
 ---> System.Net.Quic.QuicException: Operation aborted.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 157
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 221
   at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct& data) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 634
   at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 671
   at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 695
--- End of stack trace from previous location ---
   at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 235
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 306
   at System.Net.Http.Http3Connection.<>c__DisplayClass45_0.<<ProcessServerControlStreamAsync>g__ReadFrameEnvelopeAsync|0>d.MoveNext() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 679
--- End of stack trace from previous location ---
   at System.Net.Http.Http3Connection.ProcessServerControlStreamAsync(QuicStream stream, ArrayBuffer buffer) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 624
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream stream) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 520
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream stream) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 575
   --- End of inner exception stack trace ---
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Operation aborted.)
 ---> System.Net.Quic.QuicException: Operation aborted.
   at System.Net.Quic.QuicConnection.HandleEventShutdownComplete() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 492
   at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 571
   at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 598
--- End of stack trace from previous location ---
   at System.Net.Quic.QuicConnection.AcceptInboundStreamAsync(CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 402
   at System.Net.Http.Http3Connection.AcceptStreamsAsync() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 438
   --- End of inner exception stack trace ---
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The server is currently unreachable.)
 ---> System.Net.Quic.QuicException: The server is currently unreachable.
   at System.Net.Quic.QuicConnection.HandleEventShutdownInitiatedByTransport(_SHUTDOWN_INITIATED_BY_TRANSPORT_e__Struct& data) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 470
   at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 569
   at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 598
--- End of stack trace from previous location ---
   at System.Net.Quic.ValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ValueTaskSource.cs:line 176
   at System.Net.Quic.QuicConnection.FinishConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 312
   at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 68
   at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 73
   at System.Net.Http.ConnectHelper.ConnectQuicAsync(HttpRequestMessage request, DnsEndPoint endPoint, TimeSpan idleTimeout, SslClientAuthenticationOptions clientAuthenticationOptions, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\ConnectHelper.cs:line 120
   --- End of inner exception stack trace ---
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jan 3, 2023
@ghost
Copy link

ghost commented Jan 3, 2023

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

I have a console app with HttpClient that continuously sends requests to a localhost Kestrel instance.
If I randomly shut down and restart the server while HttpClient is making requests, I can see sporadic UnobservedTaskExceptions coming from System.Net.Quic.

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Operation aborted.)
 ---> System.Net.Quic.QuicException: Operation aborted.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception exception, Boolean final) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 157
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception exception, Boolean final) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 221
   at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct& data) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 634
   at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT& streamEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 671
   at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_STREAM_EVENT* streamEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 695
--- End of stack trace from previous location ---
   at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ResettableValueTaskSource.cs:line 235
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicStream.cs:line 306
   at System.Net.Http.Http3Connection.<>c__DisplayClass45_0.<<ProcessServerControlStreamAsync>g__ReadFrameEnvelopeAsync|0>d.MoveNext() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 679
--- End of stack trace from previous location ---
   at System.Net.Http.Http3Connection.ProcessServerControlStreamAsync(QuicStream stream, ArrayBuffer buffer) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 624
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream stream) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 520
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream stream) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 575
   --- End of inner exception stack trace ---
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Operation aborted.)
 ---> System.Net.Quic.QuicException: Operation aborted.
   at System.Net.Quic.QuicConnection.HandleEventShutdownComplete() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 492
   at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 571
   at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 598
--- End of stack trace from previous location ---
   at System.Net.Quic.QuicConnection.AcceptInboundStreamAsync(CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 402
   at System.Net.Http.Http3Connection.AcceptStreamsAsync() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 438
   --- End of inner exception stack trace ---
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The server is currently unreachable.)
 ---> System.Net.Quic.QuicException: The server is currently unreachable.
   at System.Net.Quic.QuicConnection.HandleEventShutdownInitiatedByTransport(_SHUTDOWN_INITIATED_BY_TRANSPORT_e__Struct& data) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 470
   at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 569
   at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 598
--- End of stack trace from previous location ---
   at System.Net.Quic.ValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\Internal\ValueTaskSource.cs:line 176
   at System.Net.Quic.QuicConnection.FinishConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 312
   at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 68
   at System.Net.Quic.QuicConnection.ConnectAsync(QuicClientConnectionOptions options, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 73
   at System.Net.Http.ConnectHelper.ConnectQuicAsync(HttpRequestMessage request, DnsEndPoint endPoint, TimeSpan idleTimeout, SslClientAuthenticationOptions clientAuthenticationOptions, CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\ConnectHelper.cs:line 120
   --- End of inner exception stack trace ---
Author: MihaZupan
Assignees: -
Labels:

bug, area-System.Net.Quic

Milestone: -

@wfurt wfurt removed the untriaged New issue has not been triaged by the area owner label Jan 3, 2023
@wfurt wfurt added this to the 8.0.0 milestone Jan 3, 2023
@JustArchi
Copy link
Contributor

JustArchi commented Jan 12, 2023

I can confirm this with entirely different project.

2023-01-12 02:50:08|ArchiSteamFarm-2292|FATAL|ASF|OnUnobservedTaskException() System.Net.Quic.QuicException: Reading has been aborted on the stream.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception, Boolean)
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception, Boolean )
   at System.Net.Quic.QuicStream.Abort(QuicAbortDirection, Int64)
   at System.Net.Http.Http3RequestStream.AbortStream()
   at System.Net.Http.Http3RequestStream.Dispose()
   at System.Net.Http.Http3RequestStream.Http3ReadStream.Dispose(Boolean)
   at System.IO.Stream.Close()
   at System.IO.Compression.BrotliStream.Dispose(Boolean)
   at System.IO.Stream.Close()
   at System.IO.StreamReader.Dispose(Boolean)
   at System.IO.StreamReader.Close()
   at Newtonsoft.Json.JsonTextReader.Close()
   at Newtonsoft.Json.JsonReader.Dispose(Boolean disposing)
   at Newtonsoft.Json.JsonReader.System.IAsyncDisposable.DisposeAsync()
   at System.Runtime.CompilerServices.ConfiguredAsyncDisposable.DisposeAsync()
   at ArchiSteamFarm.Web.WebBrowser.UrlGetToJsonObject[T](Uri request, IReadOnlyCollection`1 headers, Uri referer, ERequestOptions requestOptions, Byte maxTries, Int32 rateLimitingDelay)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at ArchiSteamFarm.Web.WebBrowser.UrlGetToStream(Uri request, IReadOnlyCollection`1 headers, Uri referer, ERequestOptions requestOptions, Byte maxTries, Int32 rateLimitingDelay)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at ArchiSteamFarm.Web.WebBrowser.InternalGet(Uri request, IReadOnlyCollection`1 headers, Uri referer, ERequestOptions requestOptions, HttpCompletionOption httpCompletionOption)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at ArchiSteamFarm.Web.WebBrowser.InternalRequest[T](Uri request, HttpMethod httpMethod, IReadOnlyCollection`1 headers, T data, Uri referer, ERequestOptions requestOptions, HttpCompletionOption httpCompletionOption, Byte maxRedirections)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage, HttpCompletionOption, CancellationTokenSource, Boolean, CancellationTokenSource, CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage, Boolean, CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage, Boolean, Boolean, CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.HttpConnectionPool.TrySendUsingHttp3Async(HttpRequestMessage, CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.Http3Connection.SendAsync(HttpRequestMessage, Int64, CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.Http3RequestStream.SendAsync(CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.SetResult()
   at System.Net.Http.Http3RequestStream.ReadResponseAsync(CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(Task`1, TResult)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Http.Http3RequestStream.ReadFrameEnvelopeAsync(CancellationToken)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox, Boolean)
   at System.Threading.Tasks.Task.RunContinuations(Object)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult)
   at System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder`1.SetResult(TResult)
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1, CancellationToken )
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread, ExecutionContext, ContextCallback, Object)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecuteFromThreadPool(Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
   at System.Threading.Thread.StartCallback()
--- End of stack trace from previous location ---
2023-01-12 03:05:01|ArchiSteamFarm-2292|FATAL|ASF|OnUnobservedTaskException() System.Net.Quic.QuicException: The connection timed out from inactivity.
   at System.Net.Quic.QuicConnection.HandleEventShutdownInitiatedByTransport(_SHUTDOWN_INITIATED_BY_TRANSPORT_e__Struct&)
   at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT&)
   at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE*, Void*, QUIC_CONNECTION_EVENT*)
--- End of stack trace from previous location ---
   at System.Net.Quic.QuicConnection.AcceptInboundStreamAsync(CancellationToken )
   at System.Net.Http.Http3Connection.AcceptStreamsAsync()
2023-01-12 03:05:01|ArchiSteamFarm-2292|FATAL|ASF|OnUnobservedTaskException() System.Net.Quic.QuicException: The connection timed out from inactivity.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception, Boolean)
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception, Boolean )
   at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct&)
   at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT&)
   at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE*, Void*, QUIC_STREAM_EVENT*)
--- End of stack trace from previous location ---
   at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16)
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1, CancellationToken )
   at System.IO.Stream.<CopyToAsync>g__Core|27_0(Stream source, Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)
2023-01-12 03:05:01|ArchiSteamFarm-2292|FATAL|ASF|OnUnobservedTaskException() System.Net.Quic.QuicException: The connection timed out from inactivity.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception, Boolean)
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception, Boolean )
   at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct&)
   at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT&)
   at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE*, Void*, QUIC_STREAM_EVENT*)
--- End of stack trace from previous location ---
   at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16)
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1, CancellationToken )
   at System.IO.Stream.<CopyToAsync>g__Core|27_0(Stream source, Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)
2023-01-12 03:05:01|ArchiSteamFarm-2292|FATAL|ASF|OnUnobservedTaskException() System.Net.Quic.QuicException: The connection timed out from inactivity.
   at System.Net.Quic.ResettableValueTaskSource.TryComplete(Exception, Boolean)
   at System.Net.Quic.ResettableValueTaskSource.TrySetException(Exception, Boolean )
   at System.Net.Quic.QuicStream.HandleEventShutdownComplete(_SHUTDOWN_COMPLETE_e__Struct&)
   at System.Net.Quic.QuicStream.HandleStreamEvent(QUIC_STREAM_EVENT&)
   at System.Net.Quic.QuicStream.NativeCallback(QUIC_HANDLE*, Void*, QUIC_STREAM_EVENT*)
--- End of stack trace from previous location ---
   at System.Net.Quic.ResettableValueTaskSource.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16)
   at System.Net.Quic.QuicStream.ReadAsync(Memory`1, CancellationToken )
   at System.Net.Http.Http3Connection.<>c__DisplayClass45_0.<<ProcessServerControlStreamAsync>g__ReadFrameEnvelopeAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.Net.Http.Http3Connection.ProcessServerControlStreamAsync(QuicStream, ArrayBuffer)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)
   at System.Net.Http.Http3Connection.ProcessServerStreamAsync(QuicStream)

The application seems to work fine however.

JustArchi added a commit to JustArchiNET/ArchiSteamFarm that referenced this issue Jan 12, 2023
JustArchi added a commit to JustArchiNET/ArchiSteamFarm that referenced this issue Jan 13, 2023
* Ignore QUIC exceptions

See dotnet/runtime#80111

* Update madness

* No i jaki jest twój problem ja się pytam
@ManickaP
Copy link
Member

Triage: we should fix this in 8.0.

@alexrp
Copy link
Contributor

alexrp commented Aug 26, 2023

Just wanted to confirm given #80111 (comment), was this moved to 9.0.0 intentionally?

@ManickaP
Copy link
Member

Yes, it was moved. Unfortunately, we didn't have a chance to look into it. And as this doesn't affect application behavior, it got lower priority. It's annoying, but not detrimental.

@liveans
Copy link
Member

liveans commented Mar 19, 2024

Looks like this is already fixed by some changes, can't repro it on main.
@MihaZupan can you repro this on main?

@MihaZupan
Copy link
Member Author

I can consistently repro on 8, but wasn't able to on 9.0 anymore.
Might have been fixed recently, closing. Feel free to reopen if anyone's still seeing it.

@MihaZupan
Copy link
Member Author

MihaZupan commented Jul 3, 2024

Doing a simple HTTP test run

.dotnet/dotnet build .\src\libraries\System.Net.Http\tests\FunctionalTests\ /t:Test

while watching for unobserved exceptions

[Fact]
public async Task WatchForUnobserved()
{
    bool seenUnobservedExceptions = false;

    EventHandler<UnobservedTaskExceptionEventArgs> eventHandler = (_, e) =>
    {
        _output.WriteLine(e.Exception.ToString());
        seenUnobservedExceptions = true;
    };

    TaskScheduler.UnobservedTaskException += eventHandler;
    await Task.Delay(10_000);
    TaskScheduler.UnobservedTaskException -= eventHandler;

    Assert.False(seenUnobservedExceptions);
}

I'm still seeing 100s of such unobserved exceptions (mainly from Quic):

System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'System.Net.Quic.QuicConnection'.)
---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'System.Net.Quic.QuicConnection'.
 at System.Net.Quic.QuicConnection.HandleEventShutdownComplete() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 652
 at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 738
 at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 771
--- End of stack trace from previous location ---
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Operation aborted.)
---> System.Net.Quic.QuicException: Operation aborted.
 at System.Net.Quic.QuicConnection.HandleEventShutdownComplete() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 652
 at System.Net.Quic.QuicConnection.HandleConnectionEvent(QUIC_CONNECTION_EVENT& connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 738
 at System.Net.Quic.QuicConnection.NativeCallback(QUIC_HANDLE* connection, Void* context, QUIC_CONNECTION_EVENT* connectionEvent) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 771
--- End of stack trace from previous location ---
 at System.Net.Quic.QuicConnection.AcceptInboundStreamAsync(CancellationToken cancellationToken) in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Quic\src\System\Net\Quic\QuicConnection.cs:line 565
 at System.Net.Http.Http3Connection.AcceptStreamsAsync() in C:\MihaZupan\runtime-debug\src\libraries\System.Net.Http\src\System\Net\Http\SocketsHttpHandler\Http3Connection.cs:line 523
 --- End of inner exception stack trace ---

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Quic bug in-pr There is an active PR which will close this issue when it is merged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants
@alexrp @JustArchi @karelz @ManickaP @wfurt @MihaZupan @liveans and others