Skip to content

Commit

Permalink
tproxy: transparent_proxy reference docs (#20241)
Browse files Browse the repository at this point in the history
Ref: #20175
  • Loading branch information
tgross committed Apr 4, 2024
1 parent bb062de commit d1f3a72
Show file tree
Hide file tree
Showing 5 changed files with 229 additions and 24 deletions.
2 changes: 1 addition & 1 deletion website/content/docs/job-specification/expose.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ check {
```

[network-to]: /nomad/docs/job-specification/network#to
[consul-expose-path-config]: /consul/docs/connect/registration/service-registration#expose-paths-configuration-reference
[consul-expose-path-config]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference
[expose-path]: /nomad/docs/job-specification/expose#path-1
[expose]: /nomad/docs/job-specification/service#expose
[path]: /nomad/docs/job-specification/expose#path-parameters 'Nomad Expose Path Parameters'
Expand Down
57 changes: 40 additions & 17 deletions website/content/docs/job-specification/proxy.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,8 @@ description: |-
/>

The `proxy` block allows configuring various options for the sidecar proxy
managed by Nomad for [Consul
Connect](/nomad/docs/integrations/consul-connect). It is valid only
within the context of a `sidecar_service` block.
managed by Nomad for [Consul Connect][]. It is valid only within the context of
a `sidecar_service` block.

```hcl
job "countdash" {
Expand Down Expand Up @@ -50,23 +49,29 @@ job "countdash" {

## `proxy` Parameters

- `local_service_address` `(string: "127.0.0.1")` - The address the local service binds to. Useful to
customize in clusters with mixed Connect and non-Connect services.
- `config` `(map: nil)` - Proxy configuration that is opaque to Nomad and passed
directly to Consul. See [Consul Connect documentation][envoy_dynamic_config]
for details. Keys and values support [runtime variable interpolation][].
- `expose` <code>([expose]: nil)</code> - Used to configure expose path
configuration for Envoy. See Consul's [Expose Paths Configuration
Reference][expose_path_ref] for more information.
- `local_service_address` `(string: "127.0.0.1")` - The address the local
service binds to. Useful to customize in clusters with mixed Connect and
non-Connect services.
- `local_service_port` `(int: <varies>)` - The port the local service binds to.
Usually the same as the parent service's port, it is useful to customize in clusters with mixed
Connect and non-Connect services.
- `upstreams` <code>([upstreams][]: nil)</code> - Used to configure details of each upstream service that
this sidecar proxy communicates with.
- `expose` <code>([expose]: nil)</code> - Used to configure expose path configuration for Envoy.
See Consul's [Expose Paths Configuration Reference](/consul/docs/connect/registration/service-registration#expose-paths-configuration-reference)
for more information.
- `config` `(map: nil)` - Proxy configuration that is opaque to Nomad and
passed directly to Consul. See [Consul Connect documentation](/consul/docs/connect/proxies/envoy#dynamic-configuration)
for details. Keys and values support [runtime variable interpolation][interpolation].
Usually the same as the parent service's port, it is useful to customize in
clusters with mixed Connect and non-Connect services.
- `transparent_proxy` <code>([transparent_proxy][]: nil)</code> - Used to enable
[transparent proxy][tproxy] mode, which allows the proxy to use Consul service
intentions to automatically configure upstreams, and configures iptables rules
to force traffic from the allocation to flow through the proxy.
- `upstreams` <code>([upstreams][]: nil)</code> - Used to configure details of
each upstream service that this sidecar proxy communicates with.

## `proxy` Examples

The following example is a proxy specification that includes upstreams configuration.
The following example is a proxy specification that includes upstreams
configuration.

```hcl
sidecar_service {
Expand All @@ -79,10 +84,28 @@ sidecar_service {
}
```

The following example is a proxy specification that includes transparent proxy
configuration. Note that with transparent proxy, you will not need to configure
an `upstreams` block.

```hcl
sidecar_service {
proxy {
transparent_proxy {
}
}
}
```

[Consul Connect]: /nomad/docs/integrations/consul-connect
[job]: /nomad/docs/job-specification/job 'Nomad job Job Specification'
[group]: /nomad/docs/job-specification/group 'Nomad group Job Specification'
[task]: /nomad/docs/job-specification/task 'Nomad task Job Specification'
[interpolation]: /nomad/docs/runtime/interpolation 'Nomad interpolation'
[runtime variable interpolation]: /nomad/docs/runtime/interpolation 'Nomad interpolation'
[sidecar_service]: /nomad/docs/job-specification/sidecar_service 'Nomad sidecar service Specification'
[upstreams]: /nomad/docs/job-specification/upstreams 'Nomad upstream config Specification'
[expose]: /nomad/docs/job-specification/expose 'Nomad proxy expose configuration'
[envoy_dynamic_config]: /consul/docs/connect/proxies/envoy#dynamic-configuration
[expose_path_ref]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference
[transparent_proxy]: /nomad/docs/job-specification/transparent_proxy
[tproxy]: /consul/docs/k8s/connect/transparent-proxy
174 changes: 174 additions & 0 deletions website/content/docs/job-specification/transparent_proxy.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
layout: docs
page_title: transparent_proxy Block - Job Specification
description: |-
The "transparent_proxy" block allows specifying options for configuring Envoy
in Consul Connect transparent proxy mode.
---

# `transparent_proxy` Block

<Placement
groups={[
'job',
'group',
'service',
'connect',
'sidecar_service',
'proxy',
'transparent_proxy',
]}
/>

The `transparent_proxy` block configures the Envoy sidecar proxy to act as a
Consul Connect [transparent proxy][tproxy]. This simplifies the configuration of
Consul Connect by eliminating the need to configure [`upstreams`][] blocks in
Nomad. Instead, the Envoy proxy will takes its configuration entirely from
Consul [service intentions][].

When transparent proxy is enabled traffic will automatically flow through the
Envoy proxy. If the local Consul agent is serving DNS, Nomad will also set up
the task's nameservers to use Consul. This lets your workload use the [virtual
IP][] DNS name from Consul, rather than configuring a `template` block that
queries services.

Using transparent proxy has some important restrictions:

* You can only have a single `connect` block in any task group that uses
transparent proxy.
* You cannot set a [`network.dns`][] block on the allocation (unless you set
[`no_dns`](#no_dns), see below).
* The node where the allocation is placed must be configured as described in
the Service Mesh integration documentation for [Transparent Proxy][].

## `transparent_proxy` Parameters

* `exclude_inbound_ports` `([]string: nil)` - A list of inbound ports to exclude
from the inbound traffic redirection. This allows traffic on these ports to
bypass the Envoy proxy. These ports can be specified as either [network port
labels][port_labels] or as numeric ports. Nomad will automatically add the
following to this list:
* The [`local_path_port`][] of any [`expose`][] block.
* The port of any service check with [`expose=true`][check_expose] set.
* The port of any `network.port` with a [`static`][] value.
* `exclude_outbound_cidrs` `([]string: nil)` - A list of CIDR subnets that
should be excluded from outbound traffic redirection. This allows traffic to
these subnets to bypass the Envoy proxy. Note this is independent of
`exclude_outbound_ports`; CIDR subnets listed here are excluded regardless of
the port.
* `exclude_outbound_ports` `([]int: nil)` - A list of port numbers that should
be excluded from outbound traffic redirection. This allows traffic to these
subnets to bypass the Envoy proxy. Note this is independent of
`exclude_outbound_cidrs`; ports listed here are excluded regardless of the
CIDR.
* `exclude_uids` `([]string: nil)` - A list of Unix user IDs (UIDs) that should
be excluded from outbound traffic redirection. When unset, only the Envoy
proxy's user will be allowed to bypass the iptables rule.
* `no_dns` `(bool: false)` - By default, Consul will be set as the nameserver
for the workload and IP tables rules will redirect DNS queries to Consul. If
you want only external DNS, set `no_dns=true`. You will need to add your own
CIDR and port exclusions for your DNS nameserver. You cannot set
[`network.dns`][] if `no_dns=false`.
* `outbound_port` `(int: 15001)` - The port that Envoy will bind on inside the
network namespace. The iptables rules created by `consul-cni` will force
traffic to flow to this port. You should only set this value if you have
specifically set the [`outbound_listener_port`][] in your Consul proxy
configuration. You can change the default value for a given node via [client
metadata](#client-metadata) (see below).
* `uid` `(string "101")` - The Unix user ID (UID) used by the Envoy proxy. You
should only set this value if you have a custom build of the Envoy container
image which uses a different UID. You can change the default value for a given
node via [client metadata](#client-metadata) (see below).

## Client Metadata

You can change the default [`outbound_port`](#outbound_port) and [`uid`](#uid)
for a given client node by updating the node metadata via the [`nomad node meta
apply`][] command. The attributes that can be updated are:

* `connect.transparent_proxy.default_uid`: Sets the default value of
[`uid`](#uid) for this node.
* `connect.transparent_proxy.default_outbound_port`: Sets the default value of
[`outbound_port`](#outbound_port) for this node.

For example, to set the default value for the `uid` field to 120:

```shell-session
$ nomad node meta apply connect.transparent_proxy.default_uid=120
$ nomad node meta read -json | jq -r '.Dynamic | ."connect.transparent_proxy.default_uid"'
120
```

You should not normally need to set these values unless you are using custom
Envoy images.

## Examples

### Minimal Example

The following example is a minimal transparent proxy specification. Note that
with transparent proxy, you will not need to configure an `upstreams` block.

```hcl
sidecar_service {
proxy {
transparent_proxy {
}
}
}
```

If you had a downstream task group `count-dashboard` that needed to connect to
an upstream task group `count-api` listening on port 9001, you could create a
Consul service intention with the following specification:

```hcl
Kind = "service-intentions"
Name = "count-api"
Sources = [
{
Name = "count-dashboard"
Action = "allow"
}
]
```

And then the downstream service `count-dashboard` could reach the `count-api`
service by making requests to `http://count-api.virtual.consul`.

### External DNS

The following example is a transparent proxy specification where external DNS is
used. To find the address of other allocations in this configuration, you will
need to use a [`template`][] block to query Consul.

```hcl
sidecar_service {
proxy {
transparent_proxy {
excluded_outbound_ports = [53]
excluded_outbound_cidrs = ["208.67.222.222/32", "208.67.220.220/32"]
no_dns = true
}
}
}
```

[tproxy]: /consul/docs/k8s/connect/transparent-proxy
[`upstreams`]: /nomad/docs/job-specification/upstreams
[service intentions]: /consul/docs/connect/config-entries/service-intentions
[virtual IP]: /consul/docs/services/discovery/dns-static-lookups#service-virtual-ip-lookups
[`consul-cni`]: https://releases.hashicorp.com/consul-cni
[cni_plugins]: /nomad/docs/networking/cni#cni-reference-plugins
[consul_dns_port]: /consul/docs/agent/config/config-files#dns_port
[`recursors`]: /consul/docs/agent/config/config-files#recursors
[port_labels]: /nomad/docs/job-specification/network#port-parameters
[`local_path_port`]: /nomad/docs/job-specification/expose#local_path_port
[`expose`]: /nomad/docs/job-specification/expose
[check_expose]: /nomad/docs/job-specification/service#expose
[`static`]: /nomad/docs/job-specification/network#static
[`outbound_listener_port`]: /consul/docs/connect/proxies/proxy-config-reference#outbound_listener_port
[`template`]: /nomad/docs/job-specification/template#consul-integration
[`nomad node meta apply`]: /nomad/docs/commands/node/meta/apply
[`network.dns`]: /nomad/docs/job-specification/network#dns-parameters
[Transparent Proxy]: /nomad/docs/integrations/consul/service-mesh#transparent-proxy
16 changes: 10 additions & 6 deletions website/content/docs/job-specification/upstreams.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ description: |-
/>

The `upstreams` block allows configuring various options for managing upstream
services that a [Consul
Connect](/nomad/docs/integrations/consul-connect) proxy routes to. It
is valid only within the context of a `proxy` block.
services that a [Consul Connect][] proxy routes to. It is valid only within the
context of a `proxy` block.

For Consul-specific details see the [Consul Connect
Guide](/consul/tutorials/get-started-vms/virtual-machine-gs-service-discovery).
For Consul-specific details see the [Consul Connect Guide][]. Note that using
`upstream` may not be necessary if you have configured the proxy with the
[`transparent_proxy`][] block.

```hcl
job "countdash" {
Expand Down Expand Up @@ -82,7 +82,7 @@ job "countdash" {
## `upstreams` Parameters

- `config` `(map: nil)` - Upstream configuration that is opaque to Nomad and passed
directly to Consul. See [Consul Connect documentation](/consul/docs/connect/registration/service-registration#upstream-configuration-reference)
directly to Consul. See [Consul Connect documentation][consul_expose_path_ref]
for details. Keys and values support [runtime variable interpolation][interpolation].
- `destination_name` `(string: <required>)` - Name of the upstream service.
- `destination_namespace` `(string: <required>)` - Name of the upstream Consul namespace.
Expand Down Expand Up @@ -135,6 +135,9 @@ and a local bind port.
}
```

[Consul Connect]: /nomad/docs/integrations/consul-connect
[Consul Connect Guide]: /consul/tutorials/get-started-vms/virtual-machine-gs-service-discovery
[`transparent_proxy`]: /nomad/docs/job-specification/transparent_proxy
[job]: /nomad/docs/job-specification/job 'Nomad job Job Specification'
[group]: /nomad/docs/job-specification/group 'Nomad group Job Specification'
[task]: /nomad/docs/job-specification/task 'Nomad task Job Specification'
Expand All @@ -144,3 +147,4 @@ and a local bind port.
[service_defaults_mode]: /consul/docs/connect/config-entries/service-defaults#meshgateway
[mesh_gateway_param]: /nomad/docs/job-specification/upstreams#mesh_gateway-parameters
[mesh_gateways]: /consul/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#mesh-gateways
[consul_expose_path_ref]: /consul/docs/connect/proxies/proxy-config-reference#expose-paths-configuration-reference
4 changes: 4 additions & 0 deletions website/data/docs-nav-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -1803,6 +1803,10 @@
"title": "template",
"path": "job-specification/template"
},
{
"title": "transparent_proxy",
"path": "job-specification/transparent_proxy"
},
{
"title": "update",
"path": "job-specification/update"
Expand Down

0 comments on commit d1f3a72

Please sign in to comment.