Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update dev guide and resources #660

Merged
merged 13 commits into from
Jul 21, 2022
39 changes: 35 additions & 4 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,24 @@
# Setup Development Environment

**Tools used in this guide**
* [kubectl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html)
* [aws](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions) - version 2 is recommended
* [eksctl](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)
* [jq](https://stedolan.github.io/jq/)
* [envsubst](https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html)
* [wget](https://www.gnu.org/software/wget/)
* [Go](https://go.dev/dl/) - version 1.17+

## 1. Clone the repo

```sh
git clone --branch v2 https://github.com/aws/aws-node-termination-handler.git nthv2
cd nthv2
git clone --branch v2 https://github.com/aws/aws-node-termination-handler.git "${GOPATH}/src/nthv2"
cd "${GOPATH}/src/nthv2"

# Display all targets and the descriptions.
make help

make test
```

## 2. Specify an EKS Cluster
Expand All @@ -26,7 +40,13 @@ envsubst <resources/eks-cluster.yaml.tmpl | eksctl create cluster --kubeconfig "
export KUBECONFIG="$PWD/kubeconfig"
```

If you do not want to use `envsubst` you can copy the template file and substitute the referenced values.
As an alternative to using `envsubst` you can copy the template file and substitute the referenced values.

### 2.2. Create an IAM OIDC provider

*Note:* If you created your cluster using `resource/eks-cluster.yaml.tmpl` in the previous step then your cluster will already have an IAM OIDC provider.

This [guide](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html) will walk you through determining whether you cluster already has an IAM OIDC provider, and how to create one if it does not already exist.

## 3. Create Infrastructure

Expand Down Expand Up @@ -59,6 +79,10 @@ Resources created:
* `WebhookRespository` - ECR Repository for images of the Kubernetes admission webhook.

```sh
# Note: The queue-infrastructure.yaml template generates the names of EventBridge rules
# from the ClusterName and QueueName parameters. To avoid exceeding name length limits
# the combined length of ClusterName and QueueName parameters should not exceed 51
# characters.
export QUEUE_NAME=<name>
export QUEUE_STACK_NAME="${INFRASTRUCTURE_STACK_NAME}-queue-${QUEUE_NAME}"

Expand Down Expand Up @@ -124,6 +148,13 @@ The `apply` target sets some Helm chart values for you based on environment vari
make HELM_OPTS='--set logging.level=debug' apply
```

### 6.2. (Optional) List all deployed resources

```sh
kubectl api-resources --verbs=list --namespaced -o name | \
xargs -n 1 kubectl get --show-kind --ignore-not-found --namespace "${CLUSTER_NAMESPACE}"
```

## 7. Define and deploy a Terminator to EKS cluster

```sh
Expand All @@ -135,7 +166,7 @@ envsubst <resources/terminator.yaml.tmpl >terminator-${TERMINATOR_NAME}.yaml
kubectl apply -f terminator-${TERMINATOR_NAME}.yaml
```

If you do not want to use `envsubst` you can copy the template file and substitute the referenced values.
As an alternative to using `envsubst` you can copy the template file and substitute the referenced values.

## 8. Remove deployed controller from EKS cluster

Expand Down
6 changes: 4 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ $(GINKGO):
GOBIN="$(BIN_DIR)" go install github.com/onsi/ginkgo/v2/ginkgo@v2.1.3

$(KO):
GOBIN="$(BIN_DIR)" go install github.com/google/ko@v0.9.3
@./scripts/download-ko.sh "$(BIN_DIR)"

$(SETUP_ENVTEST):
GOBIN="$(BIN_DIR)" go install sigs.k8s.io/controller-runtime/tools/setup-envtest@v0.0.0-20220217150738-f62a0f579d73
Expand Down Expand Up @@ -105,7 +105,9 @@ $(KODATA):

.PHONY: apply
apply: $(KO) $(KODATA) ## Deploy the controller into the current kubernetes cluster.
helm upgrade --install dev charts/aws-node-termination-handler-2 --namespace nthv2 --create-namespace \
helm upgrade --install dev charts/aws-node-termination-handler-2 \
--namespace ${CLUSTER_NAMESPACE} \
cjerad marked this conversation as resolved.
Show resolved Hide resolved
--create-namespace \
$(HELM_BASE_OPTS) \
$(HELM_OPTS) \
--set controller.image=$(shell $(KO) publish -B github.com/aws/aws-node-termination-handler/cmd/controller) \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ spec:
listKind: TerminatorList
plural: terminators
singular: terminator
categories:
- all
scope: Namespaced
versions:
- name: v1alpha1
Expand Down
2 changes: 1 addition & 1 deletion resources/eks-cluster.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ kind: ClusterConfig
metadata:
name: ${CLUSTER_NAME}
region: ${AWS_REGION}
version: "1.21"
version: "1.22"
tags:
karpenter.sh/discovery: ${CLUSTER_NAME}
managedNodeGroups:
Expand Down
10 changes: 5 additions & 5 deletions resources/queue-infrastructure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Resources:
AutoScalingTerminateRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_auto-scaling-terminate"
Name: !Sub "nth_${ClusterName}_${QueueName}_asgtrm"
Description: !Sub "Route instance-terminate lifecycle actions from EC2 Auto Scaling to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -53,7 +53,7 @@ Resources:
RebalanceRecommendationRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_rebalance-recommendation"
Name: !Sub "nth_${ClusterName}_${QueueName}_rebrec"
Description: !Sub "Route rebalance recommendations from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -70,7 +70,7 @@ Resources:
ScheduledChangeRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_scheduled-change"
Name: !Sub "nth_${ClusterName}_${QueueName}_schchg"
Description: !Sub "Route scheduled change health events from AWS Health to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -92,7 +92,7 @@ Resources:
SpotInterruptionRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_spot-interruption"
Name: !Sub "nth_${ClusterName}_${QueueName}_sptint"
Description: !Sub "Route spot interruption notices from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -109,7 +109,7 @@ Resources:
StateChangeRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_state-change"
Name: !Sub "nth_${ClusterName}_${QueueName}_sttchg"
Description: !Sub "Route state change notifications from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand Down
1 change: 1 addition & 0 deletions resources/terminator.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apiVersion: node.k8s.aws/v1alpha1
kind: Terminator
metadata:
name: ${TERMINATOR_NAME}
namespace: ${CLUSTER_NAMESPACE}

spec:
sqs:
Expand Down
57 changes: 57 additions & 0 deletions scripts/download-ko.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env bash

set -euo pipefail

usage=$(cat << EOM
usage: $(basename "$0") -h | DIR_PATH

Download the ko binary to DIR_PATH.

Arguments:
-h Print usage message then exit.

EOM
)

while getopts "h" opt; do
case $opt in
h ) echo "${usage}"
exit 0
;;
\? ) echo "${usage}" 1>&2
exit 1
;;
esac
done

dir_path="$1"

if [[ -z "${dir_path}" ]]; then
echo "error: missing directory path" 1>&2
echo 1>&2
echo "${usage}" 1>&2
exit 1
fi

if ! which wget >/dev/null ; then
echo "error: wget not installed" 1>&2
exit 1
fi

version="0.11.2"
os="$(go env GOHOSTOS)"
arch="$(go env GOHOSTARCH)"

if [[ "${arch}" == "amd64" ]]; then
arch="x86_64"
elif [[ "${arch}" == "arm" ]]; then
arch="arm64"
elif [[ "${arch}" == "386" ]]; then
arch="i386"
fi

echo "Downloading github.com/google/ko@v${version} ..."

cd "${dir_path}"
wget https://github.com/google/ko/releases/download/v${version}/ko_${version}_${os}_${arch}.tar.gz -O - | \
tar xzf - ko