GKE Autopilot support

[curtbushko]

Changes proposed in this PR:

• This adds a first pass of support for running on GKE Autopilot.
• We won't know how things fully work until we are certified with 1.2.2 (next patch release)
• I tried to do this without any helm chart values changes but it would have been considerably more messy than it is now.

How I've tested this PR:

• went through an initial acceptance pass with the Autopilot team
• installed consul-k8s using the following helm values:

global:
  name: consul
  datacenter: dc1
  logLevel: "debug"
  tls:
    enabled: false
ingress:
  enabled: false
server:
  replicas: 1
  resources:
    requests:
      memory: "200Mi"
      cpu: "500m"
    limits:
      memory: "200Mi"
      cpu: "500m"
connectInject:
  enabled: true
  default: true
  replicas: 1
  transparentProxy:
    defaultEnabled: true
  apiGateway:
    manageExternalCRDs: false
    manageCustomCRDs: true
dns:
  enabled: false
ui:
  enabled: true
  service:
    enabled: true
controller:
  enabled: true

How I expect reviewers to test this PR:

Checklist:

• Tests added
• CHANGELOG entry added

[nathancoleman]

I think it would be better to avoid having "custom" refer to both our own CRDs -- which are "custom" for Consul -- and some of the CRDs defined by the Kubernetes SIG that have not graduated to "standard" yet but most likely will in the future.

These are the groups as I see them

• "Standard" CRDs defined by Gateway API here (Gateway, HTTPRoute, etc.)
• "Experimental" CRDs defined by Gateway API here (TCPRoute in this case)
• "Consul" CRDs defined by us (GatewayClassConfig, MeshService)

Here are a couple of slightly different config structures that I think make more sense given that grouping. Let me know what you think.

manageExternalCRDs: false # Let GKE manage the "standard" CRDs
manageNonStandardCRDs: true # Install TCPRoute, GatewayClassConfig + MeshService

manageExternalCRDs: false # Let GKE manage the "standard" CRDs
# This is a higher bar for the config since this set *must* be installed right now; however, it
# avoids grouping things together and acknowledges that TCPRoute, for example, might one
# day be considered "standard". Our upgrade instructions would need to include
#   - "remove TCPRoute if..."
#   - "add SomeNewConsulCRD if..."
manageIndividualCRDs:
- TCPRoute
- GatewayClassConfig
- MeshService

[nathancoleman]

If I'm understanding correctly, the combinations that matter are

# manageCustomCRDs doesn't matter here and could be true or false, all CRDs will be installed
manageExternalCRDs: true
# manageCustomCRDs: false

# This only installs TCPRoute, GatewayClassConfig and MeshService
manageExternalCRDs: false
manageCustomCRDs: true

# No CRDs at all will be installed
manageExternalCRDs: false
manageCustomCRDs: false 

[curtbushko]

That looks correct.

[nathancoleman]

I'm fairly certain @Jeff-Apple will have thoughts on the UX here

[curtbushko]

I think it would be better to avoid having "custom" refer to both our own CRDs -- which are "custom" for Consul -- and some of the CRDs defined by the Kubernetes SIG that have not graduated to "standard" yet but most likely will in the future.

These are the groups as I see them

* "Standard" CRDs defined by Gateway API [here](https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd/standard) (`Gateway`, `HTTPRoute`, etc.)

* "Extended" CRDs defined by Gateway API [here](https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd/experimental) (`TCPRoute` in this case)

* "Consul" CRDs defined by us (`GatewayClassConfig`, `MeshService`)

Here are a couple of slightly different config structures that I think make more sense given that grouping. Let me know what you think.

manageExternalCRDs: false # Let GKE manage the "standard" CRDs
manageNonStandardCRDs: true # Install TCPRoute, GatewayClassConfig + MeshService

manageExternalCRDs: false # Let GKE manage the "standard" CRDs
# This is a higher bar for the config since this set *must* be installed right now; however, it
# avoids grouping things together and acknowledges that TCPRoute, for example, might one
# day be considered "standard". Our upgrade instructions would include "remove TCPRoute if..."
manageIndividualCRDs:
- TCPRoute
- GatewayClassConfig
- MeshService

I am fine with manageNonStandardCRD, but not the list. I do not want to get into that game for something that you will probably change in the future...

[david-yu]

I'm ok with manageNonStandardCRD as well seems to be appropriate.