[kong] Ingress configuration doesn't provide enough flexibility to configure an ALB with SSL redirect

[agaffney]

When using the kong helm chart's functionality to create an Ingress resource for the kong proxy, it's not possible to use it with the ALB Ingress controller to configure a SSL redirect at the ALB.

That configuration requires adding an additional path entry to the generated ingress rule like:

      - backend:
          serviceName: ssl-redirect
          servicePort: use-annotation
        path: /*

The serviceName and servicePort values are arbitrary and have no relation to Kong itself. They refer to an annotation on the Ingress resource like:

    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig":
      { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_302"}}'

The helm chart only allows specifying the path pattern and optionally a list of hosts to create rules for (each rule with only a single path).

I know this is a bit of a niche case, but it would be nice to have the flexibility to configure this with the helm chart rather than needing to create a separate Ingress resource outside of helm.

[hbagdi]

I'd say that this is at least a little bit ugly, AWS. -_-

cc @rainest

[rainest]

Why do providers' native controller implementations always do weird things?

I wouldn't really want to add this in--that rule can be interpreted as a normal rule by other controllers (ours included) and will create unusable configuration. The alternative is providing a means to configure arbitrary rules, and I'd aimed to make the ingress configuration simpler, not more complicated.

@hbagdi do you know if there are plans to standardize a redirect convention for these rules in future versions of the Ingress spec? https://kubernetes-sigs.github.io/service-apis/concepts/#httproute is presumably where that'd be covered, but hasn't been written yet.

[hbagdi]

I wouldn't really want to add this in--that rule can be interpreted as a normal rule by other controllers (ours included) and will create unusable configuration. The alternative is providing a means to configure arbitrary rules, and I'd aimed to make the ingress configuration simpler, not more complicated.

I agree. How can we solve @agaffney's problem stilll? @agaffney, do you have any suggestions on a dirty hack or a way to handle this that doesn't degrade the experience for other users?

@hbagdi do you know if there are plans to standardize a redirect convention for these rules in future versions of the Ingress spec? https://kubernetes-sigs.github.io/service-apis/concepts/#httproute is presumably where that'd be covered, but hasn't been written yet.

I couldn't find one so I created one: kubernetes-sigs/gateway-api#200

[agaffney]

I think all that's really needed is a ingress.customPaths var (or whatever you want to call it) in the chart that can be naively iterated over and rendered by the template. Having separate vars for pre and post path entries (before and after the kong-specific path entry) would make it flexible enough for most use cases.

[hbagdi]

ping @rainest

[rainest]

I think all that's really needed is a ingress.customPaths var (or whatever you want to call it) in the chart that can be naively iterated over and rendered by the template.

I don't disagree that this is possible, but it is a significant jump in template and configuration complexity from the existing system without any other obvious uses. Ingress templates furthermore aren't consolidated yet, so it'd have to be maintained in each separately.

Given the lack of a standard to target, I recommend one of the following workarounds:

• Maintain a fork that hard-codes the new path into the proxy Ingress template.
• Modify the Ingress resource after it is created. Helm 3 should preserve changes on subsequent upgrades.
• If it's okay to handle this outside the Ingress, set https_redirect_status_code on your route(s) with the redirect code of your choice: https://docs.konghq.com/2.0.x/admin-api/#add-route

[hbagdi]

It would also help to work with AWS team to see how they plan to fix this feature in the long-term.
#eks channel in the Kubernetes slack might be a good place to start this conversation.

[agaffney]

Given the lack of a standard to target, I recommend one of the following workarounds:

• Maintain a fork that hard-codes the new path into the proxy Ingress template.
• Modify the Ingress resource after it is created. Helm 3 should preserve changes on subsequent upgrades.
• If it's okay to handle this outside the Ingress, set https_redirect_status_code on your route(s) with the redirect code of your choice: https://docs.konghq.com/2.0.x/admin-api/#add-route

I'd rather not have to fork the chart to get this functionality, especially since it's something that other people may want and isn't really that weird. Modifying the resource after the fact isn't really an option, since the entire deployment process is automated from Terraform, and there's no reasonably way to make these changes after the fact. We are currently handling the HTTPS redirect in Kong, but the goal of moving it to the ALB is to reduce traffic actually making it into our cluster.

[rainest]

We work to be flexible with the types of environments we support, but this doesn't easily fit into our current user experience goals. One alternative route here is to use our proxy along with our controller, which would allow you to configure the Kong redirect from the Ingress directly. Would that work for you? If not, is there functionality in the AWS controller that ours can't provide, or features in the ALB proxy that we can't provide that require placing it in front? We can explore options for addressing those instead.

[agaffney]

The point of using an ALB in front of Kong is to use an ACM SSL cert (to keep the kong config mostly env-neutral). Using an ELB/ALB is also really the only way to expose Kong outside of the k8s cluster.

The main goal of moving the HTTPS redirect functionality to the ALB was to remove the usage of a custom plugin that does the redirect. I know that Kong now natively supports this functionality, but moving to it is more effort than we really care to put in.

If this is something that you just don't want to support in the helm chart for good reasons, that's fine, and it's not a blocker for us.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.