TEP-0033 Add tep for proxy support

[piyush-garg]

No description provided.

[tekton-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign piyush-garg
You can assign the PR to them by writing /assign @piyush-garg in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• teps/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[piyush-garg]

cc @vdemeester

[piyush-garg]

/kind tep

[bobcatfish]

It sounds like Tasks would need to be written to be aware of the proxy settings, is that right? e.g. if you wanted to use the git-clone Task from the catalog (https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2) the git-clone task would need to become aware of the env vars to use for the proxy, or the user would need to write their own Task that is aware of the env vars, is that right?

I'm wondering if there is a version where use can set something up using DNS such that requests are automatically routed to the proxy? i.e. some option such that Tasks do not need to be written to be proxy aware

(Might be addressed by some more "universal" non-tekton solution as discussed in Tekton WG today)

[vdemeester]

It sounds like Tasks would need to be written to be aware of the proxy settings, is that right? e.g. if you wanted to use the git-clone Task from the catalog (https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2) the git-clone task would need to become aware of the env vars to use for the proxy, or the user would need to write their own Task that is aware of the env vars, is that right?

The goal is not to have to write tasks to be aware of the proxy settings. Otherwise, all the task in the tekton catalog would need to be rewritten to have the env variables (and certs workspace possibility for https proxy) in case they are run behind a proxy. If all tektoncd/catalog tasks needs to have the exact same set of env var, param, workspace, this probably means we need to support it (by the controller or externally, or …).

This TEP is to make sure we don't have to make all the task proxy aware (like maven for example).

I'm wondering if there is a version where use can set something up using DNS such that requests are automatically routed to the proxy? i.e. some option such that Tasks do not need to be written to be proxy aware

(Might be addressed by some more "universal" non-tekton solution as discussed in Tekton WG today)

I don't see how DNS would play a role here. Usually, you are given a set of URL to use as HTTP_PROXY and HTTPS_PROXY (and NO_PROXY for skipping the proxy on some url/domain), and some certificates (for the https part). Most of the tooling support this (go does, …), it's more about "how" to configure it automatically on the cluster.

(Might be addressed by some more "universal" non-tekton solution as discussed in Tekton WG today)

Yes, that is something to look into. There is some feature in OpenShift for example to set a cluster wide proxy (or per-operator proxy) but I am not sure if it goes onto modifying the pods schedule by components… Something to dig into indeed.

[vdemeester]

Also, the idea here is to make the knowledge of Proxy in tekton as small as possible. Allow environment variable to be passed at runtime (using a new field or PodTemplate) is a way to enable this without having anything specific about proxy in the controller (also, having dynamic env var, helps other use cases). The rest is just "documentation on how to configure tekton behind a proxy".

[bobcatfish]

The goal is not to have to write tasks to be aware of the proxy settings.

Thanks for explaining @vdemeester , sounds like we agree on the goals!

I think it would help me a lot to see an example of what a Task would need to look like and/or maybe what the underlying pod would look like with the env vars provided. I was imagining that we'd be expecting Tasks to need to be aware of these env vars and use them explicitly, but it sounds like we are expecting most tooling to use these "universal" env vars by default?

I don't see how DNS would play a role here.

my experience with proxies is pretty much limited to some vague experiences with haproxy, so i was imagining something where you'd resolve DNS to localhost and have a proxy that automatically forwards requests to ... the actual proxy? maybe this is one more proxy than we'd actually need.

(might help me to see a diagram of the pieces in play as well, e.g. modeling pod -> proxy (running on some other pod?) -> ?)

[vdemeester]

I think it would help me a lot to see an example of what a Task would need to look like and/or maybe what the underlying pod would look like with the env vars provided.

So the Task should look differently at all, but the TaskRun and most importantly the Pod should look different yeah (it should have more env. variables and an additionnal mount point for certs if need be). We should add this yes.

I was imagining that we'd be expecting Tasks to need to be aware of these env vars and use them explicitly, but it sounds like we are expecting most tooling to use these "universal" env vars by default?

Well, we cannot go out of our scope in Kubernetes and Tekton. All we can do is make sure we set the correct environment variable (that should be supported by whatever runs into the containers), and those are pretty standards : HTTP_PROXY, HTTPS_PROXY and NO_PROXY. If the processe(s) running in the container(s) do not support those, that's not on Tekton (nor on Kubernetes), and there is nothing much we can do anyway.

(might help me to see a diagram of the pieces in play as well, e.g. modeling pod -> proxy (running on some other pod?) -> ?)

Yeah we can do that 😉 (we can't make any assumption on where the proxy runs, it can run in the cluster but most likely it doesn't — most of the case it runs in a very specific set of servers).

[piyush-garg]

It sounds like Tasks would need to be written to be aware of the proxy settings, is that right? e.g. if you wanted to use the git-clone Task from the catalog (https://github.com/tektoncd/catalog/tree/master/task/git-clone/0.2) the git-clone task would need to become aware of the env vars to use for the proxy, or the user would need to write their own Task that is aware of the env vars, is that right?

Based on this proposal, tasks don't need to be rewritten or modified. The current way to have proxy env is to modify task/step and this proposal is solving the exact problem to specify proxy envs as default which will be available for all workloads and pods further or can override the defaults at taskrun/pipelinerun level or if the defaults are not specified, then the proxy envs can be provided at taskrun/pipelinerun level. This will help users to not edit tasks and also can use different envs for different workloads if required without modifying the task.

I have mentioned the current way available in the Alternatives sections which is the way atm.

I'm wondering if there is a version where use can set something up using DNS such that requests are automatically routed to the proxy? i.e. some option such that Tasks do not need to be written to be proxy aware

(Might be addressed by some more "universal" non-tekton solution as discussed in Tekton WG today)

[piyush-garg]

The goal is not to have to write tasks to be aware of the proxy settings.

Thanks for explaining @vdemeester , sounds like we agree on the goals!

I think it would help me a lot to see an example of what a Task would need to look like and/or maybe what the underlying pod would look like with the env vars provided. I was imagining that we'd be expecting Tasks to need to be aware of these env vars and use them explicitly, but it sounds like we are expecting most tooling to use these "universal" env vars by default?

I don't see how DNS would play a role here.

my experience with proxies is pretty much limited to some vague experiences with haproxy, so i was imagining something where you'd resolve DNS to localhost and have a proxy that automatically forwards requests to ... the actual proxy? maybe this is one more proxy than we'd actually need.

(might help me to see a diagram of the pieces in play as well, e.g. modeling pod -> proxy (running on some other pod?) -> ?)

The idea is to have no change at the task level, but currently, we specify env in step like

    - name: prepare
      image: docker.io/alpine@sha256:203ee936961c0f491f72ce9d3c3c67d9440cdb1d61b9783cf340baa09308ffc1
      command: ["/bin/sh"]
      env:
      - name: "FOO"
        value: "bar"

This is to extract the env field at taskrun/pipelinerun level or also system-wide level through default pod template/env like

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: read-repo
spec:
  taskRef:
    name: read-task
  env:
  - name: "FOO"
    value: "bar"

So the end result would be like - controller when scheduling the pod, it should add all the env specified at default,taskrun/pipelinerun level to all containers.

apiVersion: v1
kind: Pod
metadata:
  name: read-repo-taskrun
spec:
  containers:
  - name: prepare
    image: docker.io/alpine@sha256:203ee936961c0f491f72ce9d3c3c67d9440cdb1d61b9783cf340baa09308ffc1
    env:
    - name: "FOO"
      value: "bar"

[afrittoli]

/test pull-community-teps-lint

[gabemontero]

@piyush-garg @vdemeester I think the proposal needs to be a bit more precise on with respect to certificates and HTTPS

I did not see anything in the review comments, but if I missed some prior discussion, apologies.

More specific comments embedded below.

[gabemontero]

So I see mention of cert here, but then I'm missing it elsewhere (at least my serach of cert just got one hit.

Still reading ....

[gabemontero]

But I don't see a "we won't support certs initially" disclaimer in the non goals

[gabemontero]

yeah I would expect another field here for the admin to define the certs needed to communicate with a https proxy

something like a key of say ca-bundle.crt and the value being the pem encoded cert

Now, at least for git, it has some env's it recognizes for using a cert, or skipping tls verify (see https://stackoverflow.com/questions/11621768/how-can-i-make-git-accept-a-self-signed-certificate and https://stackoverflow.com/questions/11621768/how-can-i-make-git-accept-a-self-signed-certificate/26785963#26785963 for example)

But that may not be the case for other binaries like the image building ones ... they typically like the certs in well known locations on the file system, with parameter overrides

[vdemeester]

@gabemontero right, I think we may need to "define" a contract, aka "This is the set of environment variable we set and volumes we provide for certificates", and well… if the task's binaries handle things differently, it's the task responsibility to wrap/adapt 🙃. wdyt ?

[gabemontero]

yep @vdemeester, aside from the env's, provide a volume for the cert the admin stores in the special configmap we monitor, and document how they can consume it in their task if they need it

[tekton-robot]

@piyush-garg: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

Update from @vdemeester in API working group: implemented on the operator side, need tektoncd/pipeline#1606

Pinging @vdemeester and @piyush-garg to decide next steps

[pritidesai]

@piyush-garg is updating the TEP, will need reviewers once this is updated.

[piyush-garg]

/hold

[vdemeester]

@piyush-garg let's close this one and re-create one for dynamic environment variable with one of the use case that it would tackle is a better more optimized proxy support.

/close

[tekton-robot]

@vdemeester: Closed this PR.

In response to this:

@piyush-garg let's close this one and re-create one for dynamic environment variable with one of the use case that it would tackle is a better more optimized proxy support.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.