Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable terminating gateways to use ACL Auth Method #1102

Merged
merged 14 commits into from
Mar 23, 2022
Merged

Enable terminating gateways to use ACL Auth Method #1102

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7dc6e25
Enable terminating gateway policy to be generated via Auth Method
jmurret Mar 17, 2022
b6f79b5
Filtering out failing portion of test for terminating gateway work
jmurret Mar 17, 2022
70a394d
PR feedback.Fixing tests. Changing naming conventions for policy and…
jmurret Mar 17, 2022
b4ba641
Update control-plane/subcommand/server-acl-init/command.go
jmurret Mar 17, 2022
9bf59b8
Update control-plane/subcommand/server-acl-init/command.go
jmurret Mar 17, 2022
500a244
Fixing enterprise tests
jmurret Mar 17, 2022
1e4c83e
Changing terminating gateway to pass acl-init a -component-name flag …
jmurret Mar 17, 2022
ca14c3c
fixing acceptance test to recognize that long lived tokens will not e…
jmurret Mar 18, 2022
162ec69
Correcting serviceAccount used on deployment
jmurret Mar 18, 2022
cfd1275
Making all nameshavea-ingress-gateway
jmurret Mar 19, 2022
875dd81
Update charts/consul/templates/terminating-gateways-deployment.yaml
jmurret Mar 23, 2022
8b764b0
Update control-plane/subcommand/server-acl-init/command.go
jmurret Mar 23, 2022
e9416cd
Update control-plane/subcommand/server-acl-init/command.go
jmurret Mar 23, 2022
03a268e
Update control-plane/subcommand/server-acl-init/command.go
jmurret Mar 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions acceptance/tests/terminating-gateway/terminating_gateway_namespaces_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,11 @@ func TestTerminatingGatewaySingleNamespace(t *testing.T) {
// Register the external service.
registerExternalService(t, consulClient, testNamespace)

// If ACLs are enabled we need to update the token of the terminating gateway
// If ACLs are enabled we need to update the role of the terminating gateway
// with service:write permissions to the static-server service
// so that it can can request Connect certificates for it.
if c.secure {
updateTerminatingGatewayToken(t, consulClient, fmt.Sprintf(staticServerPolicyRulesNamespace, testNamespace))
updateTerminatingGatewayRole(t, consulClient, fmt.Sprintf(staticServerPolicyRulesNamespace, testNamespace))
}

// Create the config entry for the terminating gateway.
Expand Down Expand Up @@ -205,11 +205,11 @@ func TestTerminatingGatewayNamespaceMirroring(t *testing.T) {
// Register the external service
registerExternalService(t, consulClient, testNamespace)

// If ACLs are enabled we need to update the token of the terminating gateway
// If ACLs are enabled we need to update the role of the terminating gateway
// with service:write permissions to the static-server service
// so that it can can request Connect certificates for it.
if c.secure {
updateTerminatingGatewayToken(t, consulClient, fmt.Sprintf(staticServerPolicyRulesNamespace, testNamespace))
updateTerminatingGatewayRole(t, consulClient, fmt.Sprintf(staticServerPolicyRulesNamespace, testNamespace))
}

// Create the config entry for the terminating gateway
Expand Down
30 changes: 15 additions & 15 deletions acceptance/tests/terminating-gateway/terminating_gateway_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,11 @@ func TestTerminatingGateway(t *testing.T) {
// Register the external service
registerExternalService(t, consulClient, "")

// If ACLs are enabled we need to update the token of the terminating gateway
// If ACLs are enabled we need to update the role of the terminating gateway
// with service:write permissions to the static-server service
// so that it can can request Connect certificates for it.
if c.secure {
updateTerminatingGatewayToken(t, consulClient, staticServerPolicyRules)
updateTerminatingGatewayRole(t, consulClient, staticServerPolicyRules)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: we should make sure to include it terminating gw docs (it looks like right now we don't have any docs about updating token policy but I think we definitely should add it https://www.consul.io/docs/k8s/connect/terminating-gateways). We'd also need to call it out as a breaking change for anyone using terminating gateway today because if they have updated their token policy for the service, it will not apply anymore because we'll be using a token from consul login.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 When Luke asked me about what I was working on last week, he mentioned this part of that doc would need to change to reflect updating role rather than the token. I used what he described to do the acceptance test change and plan on updating this doc now based on the acceptance test change.

}

// Create the config entry for the terminating gateway.
Expand Down Expand Up @@ -133,32 +133,32 @@ func registerExternalService(t *testing.T, consulClient *api.Client, namespace s
require.NoError(t, err)
}

func updateTerminatingGatewayToken(t *testing.T, consulClient *api.Client, rules string) {
func updateTerminatingGatewayRole(t *testing.T, consulClient *api.Client, rules string) {
t.Helper()

// Create a write policy for the static-server.
logger.Log(t, "creating a write policy for the static-server")
_, _, err := consulClient.ACL().PolicyCreate(&api.ACLPolicy{
Name: "static-server-write-policy",
Rules: rules,
}, nil)
require.NoError(t, err)

// Get the terminating gateway token.
tokens, _, err := consulClient.ACL().TokenList(nil)
logger.Log(t, "getting the terminating gateway role")
roles, _, err := consulClient.ACL().RoleList(nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use the RoleReadByName method to simplify this a bit as I think we can construct the role name (not a blocker though!)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will follow up with this. I originally tried it, but I don't think I was passing in the correct name.

require.NoError(t, err)
var termGwTokenID string
for _, token := range tokens {
if strings.Contains(token.Description, "terminating-gateway-terminating-gateway-token") {
termGwTokenID = token.AccessorID
terminatingGatewayRoleID := ""
for _, role := range roles {
if strings.Contains(role.Name, "terminating-gateway") {
terminatingGatewayRoleID = role.ID
break
}
}
termGwToken, _, err := consulClient.ACL().TokenRead(termGwTokenID, nil)
require.NoError(t, err)

// Add policy to the token and update it
termGwToken.Policies = append(termGwToken.Policies, &api.ACLTokenPolicyLink{Name: "static-server-write-policy"})
_, _, err = consulClient.ACL().TokenUpdate(termGwToken, nil)
logger.Log(t, "update role with policy")
termGwRole, _, err := consulClient.ACL().RoleRead(terminatingGatewayRoleID, nil)
require.NoError(t, err)
termGwRole.Policies = append(termGwRole.Policies, &api.ACLTokenPolicyLink{Name: "static-server-write-policy"})
_, _, err = consulClient.ACL().RoleUpdate(termGwRole, nil)
require.NoError(t, err)
}

Expand Down
8 changes: 7 additions & 1 deletion charts/consul/templates/mesh-gateway-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,13 @@ spec:
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-ec", "/consul-bin/consul services deregister -id=\"{{ .Values.meshGateway.consulServiceName }}\"", "&&", "/bin/sh", "-ec", "/consul-bin/consul logout"]
command:
- "/bin/sh"
- "-ec"
- "/consul-bin/consul services deregister -id=\"{{ .Values.meshGateway.consulServiceName }}\""
{{- if .Values.global.acls.manageSystemACLs }}
- "/consul-bin/consul logout"
jmurret marked this conversation as resolved.
Show resolved Hide resolved
{{- end}}

# consul-sidecar ensures the mesh gateway is always registered with
# the local Consul agent, even if it loses the initial registration.
Expand Down
40 changes: 24 additions & 16 deletions charts/consul/templates/terminating-gateways-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,15 +24,15 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
labels:
app: {{ template "consul.name" $root }}
chart: {{ template "consul.chart" $root }}
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
spec:
replicas: {{ default $defaults.replicas .replicas }}
selector:
Expand All @@ -42,7 +42,7 @@ spec:
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
template:
metadata:
labels:
Expand All @@ -51,7 +51,7 @@ spec:
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
annotations:
{{- if (and $root.Values.global.secretsBackend.vault.enabled $root.Values.global.tls.enabled) }}
"vault.hashicorp.com/agent-init-first": "true"
Expand Down Expand Up @@ -90,7 +90,7 @@ spec:
{{ tpl (default $defaults.tolerations .tolerations) $root | nindent 8 | trim }}
{{- end }}
terminationGracePeriodSeconds: 10
serviceAccountName: {{ template "consul.fullname" $root }}-{{ .name }}
serviceAccountName: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
volumes:
- name: consul-bin
emptyDir: {}
Expand Down Expand Up @@ -154,8 +154,8 @@ spec:
{{- if (and $root.Values.global.tls.enabled $root.Values.global.tls.enableAutoEncrypt) }}
{{- include "consul.getAutoEncryptClientCA" $root | nindent 8 }}
{{- end }}
# service-init registers the terminating gateway service.
- name: service-init
# terminating-gateway-init registers the terminating gateway service with Consul.
- name: terminating-gateway-init
image: {{ $root.Values.global.imageK8S }}
env:
- name: HOST_IP
Expand Down Expand Up @@ -185,9 +185,14 @@ spec:
- |
{{- if $root.Values.global.acls.manageSystemACLs }}
consul-k8s-control-plane acl-init \
-secret-name="{{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway-acl-token" \
-k8s-namespace={{ $root.Release.Namespace }} \
-token-sink-file=/consul/service/acl-token
-component-name=terminating-gateway/{{ template "consul.fullname" $root }}-{{ .name }} \
-acl-auth-method={{ template "consul.fullname" $root }}-k8s-component-auth-method \
{{- if $root.Values.global.adminPartitions.enabled }}
-partition={{ $root.Values.global.adminPartitions.name }} \
{{- end }}
-token-sink-file=/consul/service/acl-token \
-log-level={{ default $root.Values.global.logLevel }} \
-log-json={{ $root.Values.global.logJSON }}
{{- end }}

cat > /consul/service/service.hcl << EOF
Expand Down Expand Up @@ -252,6 +257,9 @@ spec:
volumeMounts:
- name: consul-bin
mountPath: /consul-bin
- mountPath: /consul/service
jmurret marked this conversation as resolved.
Show resolved Hide resolved
name: consul-service
readOnly: true
{{- if $root.Values.global.tls.enabled }}
{{- if $root.Values.global.tls.enableAutoEncrypt }}
- name: consul-auto-encrypt-ca-cert
Expand Down Expand Up @@ -280,12 +288,9 @@ spec:
fieldRef:
fieldPath: metadata.name
{{- if $root.Values.global.acls.manageSystemACLs }}
- name: CONSUL_HTTP_TOKEN
valueFrom:
secretKeyRef:
name: "{{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway-acl-token"
key: "token"
{{- end}}
- name: CONSUL_HTTP_TOKEN_FILE
value: "/consul/service/acl-token"
{{- end }}
{{- if $root.Values.global.tls.enabled }}
- name: CONSUL_HTTP_ADDR
value: https://$(HOST_IP):8501
Expand Down Expand Up @@ -345,6 +350,9 @@ spec:
-partition={{ $root.Values.global.adminPartitions.name }} \
{{- end }}
-id="${POD_NAME}"
{{- if $root.Values.global.acls.manageSystemACLs }}
- "/consul-bin/consul logout"
{{- end}}

# consul-sidecar ensures the terminating gateway is always registered with
# the local Consul agent, even if it loses the initial registration.
Expand Down
4 changes: 2 additions & 2 deletions charts/consul/templates/terminating-gateways-podsecuritypolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
labels:
app: {{ template "consul.name" $root }}
chart: {{ template "consul.chart" $root }}
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
spec:
privileged: false
# Required to prevent escalations to root.
Expand Down
6 changes: 3 additions & 3 deletions charts/consul/templates/terminating-gateways-role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,22 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
labels:
app: {{ template "consul.name" $root }}
chart: {{ template "consul.chart" $root }}
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
{{- if (or $root.Values.global.acls.manageSystemACLs $root.Values.global.enablePodSecurityPolicies) }}
rules:
{{- if $root.Values.global.enablePodSecurityPolicies }}
- apiGroups: ["policy"]
resources: ["podsecuritypolicies"]
resourceNames:
- {{ template "consul.fullname" $root }}-{{ .name }}
- {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
verbs:
- use
{{- end }}
Expand Down
8 changes: 4 additions & 4 deletions charts/consul/templates/terminating-gateways-rolebinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,22 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
labels:
app: {{ template "consul.name" $root }}
chart: {{ template "consul.chart" $root }}
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
subjects:
- kind: ServiceAccount
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
---
{{- end }}
Expand Down
4 changes: 2 additions & 2 deletions charts/consul/templates/terminating-gateways-serviceaccount.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ template "consul.fullname" $root }}-{{ .name }}
name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
namespace: {{ $root.Release.Namespace }}
labels:
app: {{ template "consul.name" $root }}
chart: {{ template "consul.chart" $root }}
heritage: {{ $root.Release.Service }}
release: {{ $root.Release.Name }}
component: terminating-gateway
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}
terminating-gateway-name: {{ template "consul.fullname" $root }}-{{ .name }}-terminating-gateway
{{- if (or $defaults.serviceAccount.annotations $serviceAccount.annotations) }}
annotations:
{{- if $defaults.serviceAccount.annotations }}
Expand Down
4 changes: 2 additions & 2 deletions charts/consul/test/unit/mesh-gateway-deployment.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -587,15 +587,15 @@ key2: value2' \
#--------------------------------------------------------------------
# manageSystemACLs

@test "meshGateway/Deployment: consul logout preStop hook is added when ACLs are enabled" {
@test "meshGateway/Deployment: consul-logout preStop hook is added when ACLs are enabled" {
cd `chart_dir`
local actual=$(helm template \
-s templates/mesh-gateway-deployment.yaml \
--set 'meshGateway.enabled=true' \
--set 'connectInject.enabled=true' \
--set 'global.acls.manageSystemACLs=true' \
. | tee /dev/stderr |
yq '[.spec.template.spec.containers[0].lifecycle.preStop.exec.command[6]] | any(contains("logout"))' | tee /dev/stderr)
yq '[.spec.template.spec.containers[0].lifecycle.preStop.exec.command[3]] | any(contains("/consul-bin/consul logout"))' | tee /dev/stderr)
[ "${actual}" = "true" ]
}

Expand Down
Loading