Skip to content
This repository has been archived by the owner on Dec 18, 2018. It is now read-only.

Transport agnostic kestrel refactoring #1551

Merged
merged 20 commits into from
Mar 29, 2017
Merged

Transport agnostic kestrel refactoring #1551

merged 20 commits into from
Mar 29, 2017

Conversation

halter73
Copy link
Member

What I plan to do as soon as I'm physically capable so this can be quickly merged to unblock people

  • Get unit tests compiling and passing again
  • Get performance project compiling and running again
  • Run perf tests

Possible issues/work items

Quickish fixes

  • Fix API check (exceptions.json)
  • Rewrite RequestHeaderLimitsTests "functional" tests to be unit tests or not access Frame
  • Fix namespaces/folders/class names (rename Microsoft.AspNetCore.Server.Kestrel.Libuv to Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv)
  • Create Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions project
  • Add doc comments for new public classes and interfaces
  • Don't use KestrelTrace class from Libuv project
  • Re-enable KestrelEventSource

Bigger work items

  • Move shutdown logic to Core and make it tranport agnostic
  • Move timeout logic to Core and make it tranport agnostic
  • Remove the IConnectionInformation.ConnectionControl (helped by above items)
  • Clean up IConnectionContext methods
  • Share KestrelThreads (LibuvThreads) between endpoints (will be easier when shutdown logic is moved)
  • Split transport-specific tests and general tests into distinct test projects
  • Write managed socket transport to truly test the abstraction

Notes:

  • The Https project no longer depends on the meta project which contains UseKestrel(). This is a breaking change.

#828

_log.ConnectionError(_connectionId, error);
}
_completed = true;
_pipe.Complete();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't call Complete in the lock. This can dispatch.

@@ -61,6 +61,10 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "TestResources", "TestResour
test\shared\TestResources\testCert.pfx = test\shared\TestResources\testCert.pfx
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.AspNetCore.Server.Kestrel.Libuv", "src\Microsoft.AspNetCore.Server.Kestrel.Libuv\Microsoft.AspNetCore.Server.Kestrel.Libuv.csproj", "{A76B8C8C-0DC5-4DD3-9B1F-02E51A0915F4}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as we're moving all of the code into different folders, would this be a good time to make project folder names shorter? Now that we're off PJ we can shorten folder names. Improves browsing code on GitHub and easier to use on commandline.

Copy link
Member

@davidfowl davidfowl Mar 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
}
}
//[Theory]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theory(Skip = "reason")...unless this was intentional to avoid compiler errors.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to avoid compilation errors. FrameFactory no longer exists. This is what I'm referencing in the PR description where I mention:

  • Rewrite RequestHeaderLimitsTests "functional" tests to be unit tests or not access Frame

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be clear, you're talking about only rewriting the ones that access Frame, right? We should still have functional tests for things like timeouts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I only disabled two tests that both accessed server.Frame.RequestHeaders.Count.

@natemcmaster natemcmaster mentioned this pull request Mar 24, 2017
@@ -0,0 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we've agreed to name transport packages as Microsoft.AspNetCore.Server.Kestrel.Transport.*, let's do this now do avoid further loss of history on renames.


<PropertyGroup>
<Description>Libuv transport for the ASP.NET Core Kestrel cross-platform web server.</Description>
<TargetFrameworks>net451;netstandard1.3</TargetFrameworks>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

net46

@@ -12,6 +12,8 @@
</ItemGroup>

<ItemGroup>
<!-- REVIEW: Having the Https project depend on the non-Core project is breaking since it no longer transitively pulls in UseKestrel() :( -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to remove this comment when this is addressed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we'll have to leave this as an intentional breaking change. I just wanted to make people aware.

public Frame Frame { get; set; }
public SocketOutputProducer OutputProducer { get; set; }


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: extra blank line

using Microsoft.Extensions.Internal;

namespace Microsoft.AspNetCore.Server.Kestrel.Internal.Http
{
public class SocketOutput : ISocketOutput
public class SocketOutputProducer : ISocketOutput
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're no longer interacting directly with a socket, rename this to OutputProducer. Rename the interface too.

@@ -30,7 +30,7 @@ public class KestrelServerOptions

/// <summary>
/// Enables the Listen options callback to resolve and use services registered by the application during startup.
/// Typically initialized by <see cref="Hosting.WebHostBuilderKestrelExtensions.UseKestrel(Hosting.IWebHostBuilder, Action{KestrelServerOptions})"/>.
/// Typically initialized by UseKestrel()"/>.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put this class in the non-core package?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all becomes transport specific hmmm.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You wan't to put KestrelServerOptions in the meta package or Core couldn't read those options. The UseKestrel() extension method needs to be in the meta package so it can reference both Core and configure the default libuv transport.

@davidfowl Do you think we should have a WebHostBuilder extension method for using Kestrel without wiring the default libuv transport? What should it be called?


namespace Microsoft.AspNetCore.Server.Kestrel.Internal
{
public class ConnectionLifetime : IConnectionContext
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this class should be named Connection, or better HttpConnection to avoid ambiguity with transport connection classes.

The reason I think this one shouldn't have Lifetime in its name is that it does more than lifetime stuff e.g. exposes I/O stuff. OTOH the other class exposes just End, which is clearly related to the connection's lifetime.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FrameConnection?

using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace Microsoft.AspNetCore.Server.Kestrel
{
public class KestrelServer : IServer
{
private Stack<IDisposable> _disposables;
private readonly IApplicationLifetime _applicationLifetime;
//private Stack<IDisposable> _disposables;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

{
_serviceContext = serviceContext;
_application = application;
_pipeFactory = new PipeFactory();
Copy link
Member

@davidfowl davidfowl Mar 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be passed into OnConnection. The transport may want a pool per thread (in the case of libuv), or may want to affinitize the memory to a specific connection.

_disposables.Push(engine.CreateServer(ipv4ListenOptions));
var transport = _transportFactory.Create(ipv4ListenOptions, connectionHandler);
_transports.Add(transport);
transport.BindAsync().Wait();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to .Wait?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For backwards compatibility we do. BindAsync() can throw AddressInUse exceptions for example.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use await instead?

{
public interface IConnectionHandler
{
IConnectionContext OnConnection(IConnectionInformation connectionInfo, IScheduler inputWriterScheduler, IScheduler outputReaderScheduler);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the PipeFactory here

catch (Exception e)
{
Log.LogError(0, e, "Connection.StartFrame");
throw;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is interesting, do we catch this elsewhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from old code, but Listener.OnConnection() should catch this if it isn't caught here.

// TODO: Remove these (Use Pipes instead?)
Task StopAsync();
void Abort(Exception ex);
void SetBadRequestState(RequestRejectionReason reason);
Copy link
Member

@davidfowl davidfowl Mar 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to Timeout() and keep the RequestRejectionReason in Frame.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I told you about this Thursday, and you told me to keep it lol. Obvs should change though.

ConnectionId = _connectionContext.ConnectionId;

Log.ConnectionStart(ConnectionId);
KestrelEventSource.Log.ConnectionStart(this);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move this to ConnectionHandler.OnConnection

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I had

  • Re-enable KestrelEventSource

in the original description, because i had it commented out for a while until I could fix it. I might file an issue to do this after merged if not trivial.


public IApplicationLifetime AppLifetime { get; set; }

public IKestrelTrace Log { get; set; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// TODO: Remove

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll keep something like this, but it won't be (I)KestrelTrace.

@@ -92,8 +89,6 @@ internal KestrelThread(KestrelEngine engine, int maxLoops)

public MemoryPool Memory { get; }

public PipeFactory PipelineFactory { get; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should stay here

private readonly WriteReqPool _writeReqPool;
private readonly IPipeReader _pipe;

// https://github.com/dotnet/corefxlab/issues/1334
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This belongs in SocketOutputProducer

{
Log.LogError(0, ex, $"Uncaught exception from the {nameof(IConnectionAdapter.OnConnectionAsync)} method of an {nameof(IConnectionAdapter)}.");
_frameStartedTcs.SetResult(null);
CloseRawPipes();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this happens does the SocketOutputConsumer end as well?

Copy link
Member Author

@halter73 halter73 Mar 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doesn't. This isn't the consuming class. The closing of the pipes was different for adapters and raw pipes prior to this PR. I just kept existing behavior that we used to have in Connection.OnSocketClosed(). We can clean up in another PR.

Log.LogError(0, ex, "Abort");
}
// Potentially calling user code. ThreadPool will catch and log any errors.
ServiceContext.ThreadPool.Run(() => RequestAbortedSource.Cancel());
Copy link
Member

@davidfowl davidfowl Mar 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread pool isn't pluggable anymore right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dispatching user code or not is still selectable using InternalKestrelServerOptions

@@ -190,15 +193,18 @@ public Frame(IHttpApplication<TContext> application, ConnectionContext context)
// SetBadRequestState logs the error.
SetBadRequestState(ex);
}
catch (IOException ex) when (ex.InnerException is UvException)
catch (ConnectionResetException ex)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this alot 👍

if (_completed)
{
// TODO: Get actual notification when the consumer stopped from Pipes,
// so we know if the socket is fully closed and why (for logging exceptions);
_log.ConnectionDisconnectedWrite(_connectionId, buffer.Count, ex: null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File a bug for this

@@ -61,9 +61,11 @@ public static void Main(string[] args)

// The following section should be used to demo sockets
//options.ListenUnixSocket("/tmp/kestrel-test.sock");

})
.UseLibuv(options =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird to have this on the same level as kestrel. This needs to move to options.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That just means that hosting services don't get injected into the ITransportFactory for free. For example, LibuvTransportFactory depends on IOptions<LibuvTransportOptions>, IApplicationLifetimeandILoggerFactory`.

Do you want to use ActivatorUtilities to activate a registered ITransportFactory type on KestrelServerOptions? That seems really gross.

_disposables.Push(engine.CreateServer(ipv6ListenOptions));
var transport = _transportFactory.Create(ipv6ListenOptions, connectionHandler);
_transports.Add(transport);
transport.BindAsync().Wait();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.Wait()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😢

}
catch (Exception e)
{
Log.LogError(0, e, "Connection.StartFrame");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nameof, especially now that things are being moved around and renamed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The names are often wrong and could muck up the PR.

In the current code I can find 26 LogError calls that could use nameof, but don't. I would change them all together

@davidfowl
Copy link
Member

API check LOL 😄

- All existing code stays in the Core package for now
@halter73 halter73 changed the title [WIP] Transport agnostic kestrel refactoring Transport agnostic kestrel refactoring Mar 29, 2017
@cesarblum
Copy link
Contributor

🎉

@natemcmaster
Copy link
Contributor

👏

@halter73
Copy link
Member Author

I ran all the BenchmarkDotNet benchmarks, and the numbers for those stayed very stable before and after the change. This makes sense considering those don't test the integration between the transport and parsing code.

I ran the end-to-end techempower benchmark on 2 F4 linux VMs, and the results were comparable:

Before

shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.14ms   12.15ms 229.67ms   98.34%
    Req/Sec    16.58k     2.61k   38.33k    85.11%
  Latency Distribution
     50%    4.42ms
     75%    7.07ms
     90%   10.37ms
     99%   24.65ms
  7662062 requests in 14.91s, 0.94GB read
  Socket errors: connect 0, read 0, write 0, timeout 669
Requests/sec: 514007.22
Transfer/sec:     64.71MB
shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.37ms    4.04ms  54.98ms   74.30%
    Req/Sec    16.58k     2.58k   33.12k    81.80%
  Latency Distribution
     50%    4.42ms
     75%    7.35ms
     90%   10.70ms
     99%   18.41ms
  7859493 requests in 14.91s, 0.97GB read
  Socket errors: connect 0, read 0, write 0, timeout 761
Requests/sec: 527223.63
Transfer/sec:     66.37MB
shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.38ms    3.90ms  58.41ms   71.52%
    Req/Sec    16.42k     2.19k   46.99k    83.27%
  Latency Distribution
     50%    4.46ms
     75%    7.48ms
     90%   10.75ms
     99%   16.91ms
  7795328 requests in 14.91s, 0.96GB read
  Socket errors: connect 0, read 0, write 0, timeout 765
Requests/sec: 522778.47
Transfer/sec:     65.81MB

After

shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.25ms   14.37ms 258.17ms   98.51%
    Req/Sec    16.79k    22.40k    1.10M    99.96%
  Latency Distribution
     50%    4.32ms
     75%    6.77ms
     90%    9.99ms
     99%   31.72ms
  7759079 requests in 14.89s, 0.95GB read
  Socket errors: connect 0, read 0, write 0, timeout 767
Requests/sec: 521248.89
Transfer/sec:     65.62MB
shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.98ms    3.51ms  69.46ms   73.95%
    Req/Sec    16.70k     2.22k   38.88k    81.03%
  Latency Distribution
     50%    4.34ms
     75%    6.70ms
     90%    9.29ms
     99%   16.65ms
  7924464 requests in 14.91s, 0.97GB read
  Socket errors: connect 0, read 0, write 0, timeout 766
Requests/sec: 531397.56
Transfer/sec:     66.89MB
shalter@f4-linux:~$ wrk -H 'Host: localhost' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 256 --timeout 8 -t 32 http ://10.1.2.6:5000/plaintext -s ~/pipeline.lua -- 16
Running 15s test @ http://10.1.2.6:5000/plaintext
  32 threads and 256 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.34ms    3.89ms  69.39ms   72.32%
    Req/Sec    16.60k     2.50k   43.53k    79.97%
  Latency Distribution
     50%    4.44ms
     75%    7.34ms
     90%   10.64ms
     99%   17.58ms
  7875381 requests in 14.91s, 0.97GB read
  Socket errors: connect 0, read 0, write 0, timeout 767
Requests/sec: 528140.19
Transfer/sec:     66.48MB

@tmds
Copy link
Contributor

tmds commented Mar 30, 2017

👍 🎉

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants