clustering of reverse proxy

[grahamehorner]

I would like to cluster the reverse-proxy so to avoid outages should a service die; I have create 5 instances of the reverse-proxy all instances read configuration from shared filesystem. How can I make sure that ;-

1. load balancing works across the reverse-proxy cluster and selects the best node for a service?
2. have the reverse-proxy report the health of their cluster and alert if/when a reverse-proxy node is unhealthy/failed.

[Tratcher]

I would like to cluster the reverse-proxy so to avoid outages should a service die;

To do this you'd have to have another proxy/load balancer in front of all of your proxies. This would normally be a level 4 (TCP) proxy with only limited balancing capabilities. Anything smarter becomes a bottleneck and itself a single point of failure.

• load balancing works across the reverse-proxy cluster and selects the best node for a service?

Trying to collect the up-to-the-second load state for each destination instance across all proxies is an extremely complex and expensive task and is a non-goal for YARP. However, there are many algorithms that perform acceptably even without that perfect knowledge.

PowerOfTwoChoices (YARP's default) selects two random nodes and then sends requests to the least busy of the two. This doesn't directly account for traffic from other proxy instances, but it can indirectly infer it. If a destination becomes overloaded then requests sent to it from any given proxy will start to backlog and that instance will stop sending additional requests. This doesn't result in a perfect distribution, but it does perform decently on average at 1/1000th the cost.

As of preview8 (soon to be released), load balancing is fully extensible so if you need something more complex I suggest starting here:
https://github.com/microsoft/reverse-proxy/blob/master/docs/docfx/articles/load-balancing.md

• have the reverse-proxy report the health of their cluster and alert if/when a reverse-proxy node is unhealthy/failed.

#306 tracks some of the reporting mechanisms you'd need for this, such as exposing the active health check results for destinations. Since YARP itself plugs into ASP.NET Core you can use Health Checks in the proxy instances to respond to health probes.

You can also monitor and report the performance metrics of each instance using these tools that are in development: #571

[karelz]

Duplicate of #125