Shadow deployment for Istio

[amoldavsky]

Among the various Istio examples there no example of a Shadow deployment and Istio. There seem to be no annotation support to do a shadow deployment with Istio like there is with Ambassador.

[ryandawsonuk]

Notes on this...

ISTIO

Two Deployments with different labels. One VirtualService pointing to one k8s Service (which matches both Deployments).

Shadow by adding a 'mirror' entry to the VirtualService.

AMBASSADOR

Two Deployments and two Services.

Two Mappings with same 'prefix'. Shadow by adding 'shadow: true'.

SELDON

Currently ambassador shadowing is done by creating a second SeldonDeployment and annotating it as a shadow of the first.

For istio we can't use two SeldonDeployments as then it won't be clear which owns the single VirtualService or Service.

Instead we could have two predictors in a SeldonDeployment and mark one as shadowing the other. A field could be added to the CRD to indicate a shadow. Much as we have a 'traffic' with a numeric value to indicate a canary traffic-split, we could have 'shadow' attribute ('true' or 'false').

This would mean that the ambassador and istio shadowing would be set differently by the user. Possibly we provide a way for the istio style with predictors to also work for ambassador. (But not the other way around because the single VirtualService has to belong to a single SeldonDeployment.)

If there were canary or A/B test traffic splitting in place then there would be multiple predictors with their own weights. In the VirtualService these predictors are different destinations under the same route. There would be a single 'mirror' for the whole route, not per-destination. So there would be a single shadow for the whole SeldonDeployment, even if an A/B test or canary is active. When weights are used we check that the weights are present and add up to 100 - we may need to exempt the shadow predictor from that check.

[ryandawsonuk]

The istio example referenced above matches to the mirror by label and the mirror shares the same k8s Service as the main version. This doesn't seem to be strictly necessary. The mirror can have its own Service and the traffic can be sent to that host. This is what we do for traffic splitting. Don't see an explicit example doing mirroring this way but the approach is working in the referenced PR.