-
Notifications
You must be signed in to change notification settings - Fork 528
Transport agnostic kestrel refactoring #1551
Conversation
_log.ConnectionError(_connectionId, error); | ||
} | ||
_completed = true; | ||
_pipe.Complete(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't call Complete in the lock. This can dispatch.
KestrelHttpServer.sln
Outdated
@@ -61,6 +61,10 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "TestResources", "TestResour | |||
test\shared\TestResources\testCert.pfx = test\shared\TestResources\testCert.pfx | |||
EndProjectSection | |||
EndProject | |||
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.AspNetCore.Server.Kestrel.Libuv", "src\Microsoft.AspNetCore.Server.Kestrel.Libuv\Microsoft.AspNetCore.Server.Kestrel.Libuv.csproj", "{A76B8C8C-0DC5-4DD3-9B1F-02E51A0915F4}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As long as we're moving all of the code into different folders, would this be a good time to make project folder names shorter? Now that we're off PJ we can shorten folder names. Improves browsing code on GitHub and easier to use on commandline.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
} | ||
} | ||
} | ||
//[Theory] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Theory(Skip = "reason")
...unless this was intentional to avoid compiler errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is to avoid compilation errors. FrameFactory no longer exists. This is what I'm referencing in the PR description where I mention:
- Rewrite RequestHeaderLimitsTests "functional" tests to be unit tests or not access Frame
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to be clear, you're talking about only rewriting the ones that access Frame
, right? We should still have functional tests for things like timeouts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I only disabled two tests that both accessed server.Frame.RequestHeaders.Count
.
@@ -0,0 +1,23 @@ | |||
<Project Sdk="Microsoft.NET.Sdk"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we've agreed to name transport packages as Microsoft.AspNetCore.Server.Kestrel.Transport.*
, let's do this now do avoid further loss of history on renames.
|
||
<PropertyGroup> | ||
<Description>Libuv transport for the ASP.NET Core Kestrel cross-platform web server.</Description> | ||
<TargetFrameworks>net451;netstandard1.3</TargetFrameworks> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
net46
samples/SampleApp/SampleApp.csproj
Outdated
@@ -12,6 +12,8 @@ | |||
</ItemGroup> | |||
|
|||
<ItemGroup> | |||
<!-- REVIEW: Having the Https project depend on the non-Core project is breaking since it no longer transitively pulls in UseKestrel() :( --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remember to remove this comment when this is addressed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that we'll have to leave this as an intentional breaking change. I just wanted to make people aware.
public Frame Frame { get; set; } | ||
public SocketOutputProducer OutputProducer { get; set; } | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: extra blank line
using Microsoft.Extensions.Internal; | ||
|
||
namespace Microsoft.AspNetCore.Server.Kestrel.Internal.Http | ||
{ | ||
public class SocketOutput : ISocketOutput | ||
public class SocketOutputProducer : ISocketOutput |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we're no longer interacting directly with a socket, rename this to OutputProducer
. Rename the interface too.
@@ -30,7 +30,7 @@ public class KestrelServerOptions | |||
|
|||
/// <summary> | |||
/// Enables the Listen options callback to resolve and use services registered by the application during startup. | |||
/// Typically initialized by <see cref="Hosting.WebHostBuilderKestrelExtensions.UseKestrel(Hosting.IWebHostBuilder, Action{KestrelServerOptions})"/>. | |||
/// Typically initialized by UseKestrel()"/>. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we put this class in the non-core package?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This all becomes transport specific hmmm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You wan't to put KestrelServerOptions in the meta package or Core couldn't read those options. The UseKestrel() extension method needs to be in the meta package so it can reference both Core and configure the default libuv transport.
@davidfowl Do you think we should have a WebHostBuilder extension method for using Kestrel without wiring the default libuv transport? What should it be called?
|
||
namespace Microsoft.AspNetCore.Server.Kestrel.Internal | ||
{ | ||
public class ConnectionLifetime : IConnectionContext |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this class should be named Connection
, or better HttpConnection
to avoid ambiguity with transport connection classes.
The reason I think this one shouldn't have Lifetime
in its name is that it does more than lifetime stuff e.g. exposes I/O stuff. OTOH the other class exposes just End
, which is clearly related to the connection's lifetime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FrameConnection?
using Microsoft.Extensions.Logging; | ||
using Microsoft.Extensions.Options; | ||
|
||
namespace Microsoft.AspNetCore.Server.Kestrel | ||
{ | ||
public class KestrelServer : IServer | ||
{ | ||
private Stack<IDisposable> _disposables; | ||
private readonly IApplicationLifetime _applicationLifetime; | ||
//private Stack<IDisposable> _disposables; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
?
{ | ||
_serviceContext = serviceContext; | ||
_application = application; | ||
_pipeFactory = new PipeFactory(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be passed into OnConnection. The transport may want a pool per thread (in the case of libuv), or may want to affinitize the memory to a specific connection.
_disposables.Push(engine.CreateServer(ipv4ListenOptions)); | ||
var transport = _transportFactory.Create(ipv4ListenOptions, connectionHandler); | ||
_transports.Add(transport); | ||
transport.BindAsync().Wait(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need to .Wait?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For backwards compatibility we do. BindAsync() can throw AddressInUse exceptions for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use await
instead?
{ | ||
public interface IConnectionHandler | ||
{ | ||
IConnectionContext OnConnection(IConnectionInformation connectionInfo, IScheduler inputWriterScheduler, IScheduler outputReaderScheduler); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add the PipeFactory here
catch (Exception e) | ||
{ | ||
Log.LogError(0, e, "Connection.StartFrame"); | ||
throw; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is interesting, do we catch this elsewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I copied this from old code, but Listener.OnConnection()
should catch this if it isn't caught here.
// TODO: Remove these (Use Pipes instead?) | ||
Task StopAsync(); | ||
void Abort(Exception ex); | ||
void SetBadRequestState(RequestRejectionReason reason); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change this to Timeout()
and keep the RequestRejectionReason
in Frame.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I told you about this Thursday, and you told me to keep it lol. Obvs should change though.
ConnectionId = _connectionContext.ConnectionId; | ||
|
||
Log.ConnectionStart(ConnectionId); | ||
KestrelEventSource.Log.ConnectionStart(this); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should move this to ConnectionHandler.OnConnection
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. I had
- Re-enable KestrelEventSource
in the original description, because i had it commented out for a while until I could fix it. I might file an issue to do this after merged if not trivial.
|
||
public IApplicationLifetime AppLifetime { get; set; } | ||
|
||
public IKestrelTrace Log { get; set; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// TODO: Remove
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll keep something like this, but it won't be (I)KestrelTrace.
@@ -92,8 +89,6 @@ internal KestrelThread(KestrelEngine engine, int maxLoops) | |||
|
|||
public MemoryPool Memory { get; } | |||
|
|||
public PipeFactory PipelineFactory { get; } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should stay here
private readonly WriteReqPool _writeReqPool; | ||
private readonly IPipeReader _pipe; | ||
|
||
// https://github.com/dotnet/corefxlab/issues/1334 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This belongs in SocketOutputProducer
{ | ||
Log.LogError(0, ex, $"Uncaught exception from the {nameof(IConnectionAdapter.OnConnectionAsync)} method of an {nameof(IConnectionAdapter)}."); | ||
_frameStartedTcs.SetResult(null); | ||
CloseRawPipes(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When this happens does the SocketOutputConsumer
end as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I doesn't. This isn't the consuming class. The closing of the pipes was different for adapters and raw pipes prior to this PR. I just kept existing behavior that we used to have in Connection.OnSocketClosed(). We can clean up in another PR.
Log.LogError(0, ex, "Abort"); | ||
} | ||
// Potentially calling user code. ThreadPool will catch and log any errors. | ||
ServiceContext.ThreadPool.Run(() => RequestAbortedSource.Cancel()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thread pool isn't pluggable anymore right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dispatching user code or not is still selectable using InternalKestrelServerOptions
@@ -190,15 +193,18 @@ public Frame(IHttpApplication<TContext> application, ConnectionContext context) | |||
// SetBadRequestState logs the error. | |||
SetBadRequestState(ex); | |||
} | |||
catch (IOException ex) when (ex.InnerException is UvException) | |||
catch (ConnectionResetException ex) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this alot 👍
if (_completed) | ||
{ | ||
// TODO: Get actual notification when the consumer stopped from Pipes, | ||
// so we know if the socket is fully closed and why (for logging exceptions); | ||
_log.ConnectionDisconnectedWrite(_connectionId, buffer.Count, ex: null); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
File a bug for this
@@ -61,9 +61,11 @@ public static void Main(string[] args) | |||
|
|||
// The following section should be used to demo sockets | |||
//options.ListenUnixSocket("/tmp/kestrel-test.sock"); | |||
|
|||
}) | |||
.UseLibuv(options => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's weird to have this on the same level as kestrel. This needs to move to options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That just means that hosting services don't get injected into the ITransportFactory for free. For example, LibuvTransportFactory
depends on IOptions<LibuvTransportOptions>,
IApplicationLifetimeand
ILoggerFactory`.
Do you want to use ActivatorUtilities to activate a registered ITransportFactory type on KestrelServerOptions? That seems really gross.
_disposables.Push(engine.CreateServer(ipv6ListenOptions)); | ||
var transport = _transportFactory.Create(ipv6ListenOptions, connectionHandler); | ||
_transports.Add(transport); | ||
transport.BindAsync().Wait(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.Wait()?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😢
} | ||
catch (Exception e) | ||
{ | ||
Log.LogError(0, e, "Connection.StartFrame"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nameof
, especially now that things are being moved around and renamed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The names are often wrong and could muck up the PR.
In the current code I can find 26 LogError calls that could use nameof, but don't. I would change them all together
API check LOL 😄 |
- All existing code stays in the Core package for now
🎉 |
👏 |
I ran all the BenchmarkDotNet benchmarks, and the numbers for those stayed very stable before and after the change. This makes sense considering those don't test the integration between the transport and parsing code. I ran the end-to-end techempower benchmark on 2 F4 linux VMs, and the results were comparable: Before
After
|
👍 🎉 |
What I plan to do as soon as I'm physically capable so this can be quickly merged to unblock people
Possible issues/work items
Quickish fixes
Bigger work items
Notes:
#828