[TEP-0117] Concurrency controls

This commit adds requirements and use cases for controlling concurrency of PipelineRuns.

teps/0117-concurrency-controls.md

@@ -0,0 +1,99 @@
+---
+status: proposed
+title: Concurrency Controls
+creation-date: '2022-08-19'
+last-updated: '2022-08-19'
+authors:
+- '@vdemeester'
+- '@williamlfish'
+- '@lbernick'
+---
+
+# TEP-0120: Concurrency Controls
+
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Use Cases](#use-cases)
+  - [Requirements](#requirements)
+  - [Future Work](#future-work)
+- [Proposal](#proposal)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Allow users to configure desired behavior for managing concurrent PipelineRuns.
+
+## Motivation
+
+Allow users to avoid wasting resources on redundant work, to prevent concurrent runs of
+non-idempotent operations, and to have more control over concurrent load on a cluster.
+
+### Use Cases
+
+Avoiding redundant work
+- A developer pushes a pull request and quickly notices and fixes a typo. They would like to have the first CI run automatically canceled and replaced by the second.
+
+Controlling non-idempotent operations
+- An organization uses a Pipeline for deployment, and wants to make sure that only the most recent changes are applied to their cluster.
+If a new deployment PipelineRun starts while a previous one is still running, they would like the previous one to be canceled.
+  - The organization might want only one deployment PipelineRun per cluster, per namespace, per environment (prod/staging),
+  per repo, or per user ([example](https://github.com/tektoncd/pipeline/issues/2828#issuecomment-646150534)).
+- An integration test communicates with a stateful external service (like a database), and a developer wants to ensure
+that integration testing TaskRuns within their CI PipelineRun don’t run concurrently with other integration testing TaskRuns
+(based on [this comment](https://github.com/tektoncd/pipeline/issues/2828#issuecomment-647747330)).
+- An organization would like to ensure Terraform operations run serially (based on [this comment](https://github.com/tektoncd/experimental/issues/699#issuecomment-951606279))
+  - This user would like to be able to cancel queued runs that are pending, rather than cancelling running runs. 
+
+Controlling load on a cluster
+- An organization has multiple teams working on a mobile application with a limited number of test devices.
+They want to limit the number of concurrent CI runs per team, to prevent one team from using all the available devices
+and crowding out CI runs from other teams.
+- A Pipeline performs multiple parallelizable things with different concurrency caps, as described in [this comment](https://github.com/tektoncd/pipeline/issues/2591#issuecomment-626778025).
+- Allow users to cap the number of matrixed TaskRuns (alpha) that can run at a given time.
+  - Currently, we have the feature flag “default-maximum-matrix-fan-out”, which restricts the total number of TaskRuns
+  that can be created from one Pipeline Task. However, we would like to support capping the number of matrixed TaskRuns
+  that can run concurrently, instead of statically capping the number of matrixed TaskRuns that can be created at all.
+
+### Requirements
+
+- Must be able to limit the maximum number of concurrent PipelineRuns for a given Pipeline, both within a namespace and within a cluster.
+- PipelineRuns are handled using a first-in, first-out strategy.
+- Must be possible to define multiple concurrency controls for a given Pipeline.
+  - For example, both limiting the number of CI runs per repo and only allowing one at a time per pull request.
+- Avoid opinionated, advanced concurrency controls, like “only one run per pull request”
+- Handle race conditions related to starting concurrent PipelineRuns.
+  - When two PipelineRuns start around the same time, they will both need to determine whether they can start based on what PipelineRuns
+  are already running. This design will need to prevent these PipelineRuns from both attempting to cancel the same existing PipelineRun,
+  or both starting when only one additional PipelineRun can be allowed to start.
+
+### Future Work
+
+- Managing TaskRun concurrency, whether run in isolation or as part of a PipelineRun.
+- Priority and preemption of queued PipelineRuns, including prioritizing based on compute resources.
+- Capping the amount of time a PipelineRun can be queued for, or providing a way to clear the queue.
+
+## Proposal
+
+TODO
+
+## References
+
+Feature requests and discussions
+- [Idea: Pipeline Mutexes](https://github.com/tektoncd/pipeline/issues/2828)
+- [Discussion: out of order execution in CD](https://github.com/tektoncd/community/issues/733)
+- [Concurrency limiter controller](https://github.com/tektoncd/experimental/issues/699)
+- [Provide a Pipeline concurrency limit](https://github.com/tektoncd/pipeline/issues/1305)
+- [Controlling max parallel jobs per Pipeline](https://github.com/tektoncd/pipeline/issues/2591)
+- [race conditions when having more than one pipeline of the same branch](https://github.com/opendevstack/ods-pipeline/issues/394)
+  - This is for OpenDevStack, which uses Tekton
+
+Design Proposals
+- [Run concurrency keys/mutexes](https://hackmd.io/GK_1_6DWTvSiVHBL6umqDA)
+- [TEP-0013: Add limit to Pipeline concurrency](https://github.com/tektoncd/community/pull/228)
+- [Managing PipelineRun concurrency](https://docs.google.com/document/d/1mORY-zKkTw0N-HJtIOnDthTK79bOsQvY_-Qz6j70SpI)
+- [Blog post: Using Lease Resources to Manage Concurrency in Tekton Builds](https://holly-k-cummins.medium.com/using-lease-resources-to-manage-concurrency-in-tekton-builds-344ba84df297)
+
+Similar features in other CI/CD systems
+- [Github Actions concurrency controls](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency)

teps/README.md

@@ -262,4 +262,5 @@ This is the complete list of Tekton teps:
 |[TEP-0112](0112-replace-volumes-with-workspaces.md) | Replace Volumes with Workspaces | proposed | 2022-07-20 |
 |[TEP-0114](0114-custom-tasks-beta.md) | Custom Tasks Beta | implementable | 2022-07-12 |
 |[TEP-0116](0116-referencing-finally-task-results-in-pipeline-results.md) | Referencing Finally Task Results in Pipeline Results | implemented | 2022-08-11 |
+|[TEP-0117](0117-concurrency-controls.md) | Concurrency Controls | proposed | 2022-08-19 |
 |[TEP-0118](0118-matrix-with-explicit-combinations-of-parameters.md) | Matrix with Explicit Combinations of Parameters | implementable | 2022-08-08 |