From ea7a546a1226a5ab7b9314c5a324b033cc93d928 Mon Sep 17 00:00:00 2001 From: Sergey Smolnikov Date: Fri, 27 Sep 2024 14:05:09 +0200 Subject: [PATCH 1/2] Added info about private submodules --- .../docs/guides/git-submodules/index.md | 4 + .../update-submodule-in-pipeline-job.md | 110 ++++++++++++++++++ 2 files changed, 114 insertions(+) create mode 100644 public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md diff --git a/public-site/docs/guides/git-submodules/index.md b/public-site/docs/guides/git-submodules/index.md index 2497cd85..053ea1a4 100644 --- a/public-site/docs/guides/git-submodules/index.md +++ b/public-site/docs/guides/git-submodules/index.md @@ -10,6 +10,10 @@ title: Git submodules As of December 2022, Radix only supports a single SSH key for authenticating to remote git repositories. This means that a Radix repository's submodules must be either public or accessible using the same deploy key as the main repository. Because GitHub globally prohibits reuse of a single deploy key across multiple repositories, it's not possible to use submodules which point to private GitHub repositories. +:::info Note +There is an option to update submodule within the pipeline job. Please look [at the example](/guides/git-submodules/update-submodule-in-pipeline-job.md). +::: + ## Sample application with git submodule [This GitHub repository](https://github.com/equinor/radix-app-with-submodule-example) contains a Radix application that has a single component, `redis`. This component is built from the source code inside the `redis` directory, and this directory is in turn a git submodule which points to a git repository at https://github.com/equinor/radix-submodule-example. Take note that the remote URL of the submodule is HTTPS and not SSH; unauthenticated clone via SSH, even for public repositories, is not supported. diff --git a/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md b/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md new file mode 100644 index 00000000..5428ee99 --- /dev/null +++ b/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md @@ -0,0 +1,110 @@ +--- +title: Update Git submodules in a pipeline job +--- + +# Update Git submodules in a pipeline job + +[Submodules](/guides/git-submodules/index.md) are mapped to a Git repository within a folder. This is an example of a Radix app with a submodule, located in the external private or internal Git repository. + +## Radix application repository structure +```sh +/ +├── app +├── .gitmodules +├── Dockerfile +└── radixconfig.yaml +``` +* `app` - a "virtual" folder, referenced to a submodule +* `.gitmodules` - a file, describing the reference to a submodule: + + ``` + [submodule "app"] + path = app + url = git@github.com:organisation-name/submodule-repository-name.git + ``` +## Submodule repository structure +```sh +/ +├── server.js +``` +Repository content will be cloned to the Radix application cloned repository within the folder, specified in the `.gitmodules` property `path` (in this example in the folder `app`). +## Radix config file +radixconfig.yaml +```yaml +apiVersion: radix.equinor.com/v1 +kind: RadixApplication +metadata: + name: your-radix-app-name +spec: + build: + useBuildKit: true + secrets: + - SSH_KEY + environments: + - name: dev + build: + from: main + components: + - name: server + ports: + - name: http + port: 8080 + publicPort: http +``` +This Radix application has a build secret `SSH_KEY` and it uses build-kit to be built. +## Dockerfile +```Dockerfile +FROM docker.io/alpine AS builder + +RUN apk update && apk add git openssh-client +RUN mkdir -p /root/.ssh && ssh-keyscan github.com > /root/.ssh/known_hosts + +WORKDIR /app +COPY . . + +WORKDIR /root/.ssh +RUN --mount=type=secret,id=SSH_KEY,dst=id_rsa \ +cd /app && git submodule update --init --remote --merge --recursive --verbose + +FROM docker.io/node:alpine + +WORKDIR /app +COPY --from=builder /app . + +WORKDIR /app/app +USER 1001 +EXPOSE 8080 +CMD ["node", "server.js"] +``` +The build secret `SSH_KEY`, specified in the `radixconfig.yaml` should contain a private key (in PEM format), which will be mounted within the file `/root/.ssh/id_rsa` (which is used by default by Git CLI). +This Dockerfile has two stages - the first (with an alias `builder`) is to update submodules, the second with a runtime to run the code. +If an option `--remote` is not specified - submodule will be cloned with a version referenced in the current commit of the Radix application repository, not the latest version of the submodule repository. + +:::info Hint +The default file name with a private key can be changed with one of following options: +* `env GIT_SSH_COMMAND='ssh -i /path/to/your/private_key' submodule update --init --recursive` +* `git -c core.sshCommand="ssh -i /path/to/your/private_key" submodule update --init --recursive` +::: +## Prepare keys +* Generate private and public keys. The key format need to be PEM, do not set passphrase (hit the Enter on the request "Enter passphrase" and "Enter same passphrase again"): + ```shell + ssh-keygen -t rsa -b 4096 -m PEM -f private-key-file + ``` +* Get generated keys with commands `cat` (Linux, MacOS, Windows PowerShell), `type` (Windows Terminal) or with an editor: + * the private key will be copy-pasted to the SSH_KEY build secret on the next step + ```shell + cat private-key-file + ``` + * the public key need to be copy-pasted to a new deploy-key in the submodule's GitHub repository: `Repository/Settings/Deploy keys/Add deploy key`. `Allow write access` is not needed. + ```shell + cat private-key-file.pem` + ``` +## Register and deploy Radix application +* Register a Radix application in the Radix cluster +* Create a first pipeline job `build-deploy` +* This job will fail due to a build secret `SSH_KEY` is not populated. This can be fixed within the Radix console, "Configuration" page of the application, the section "Build secrets" - click on the secret name `SSH_KEY` and copy-paste the private key from the previous step +* Create a new `build-deploy` pipeline job - when it is completed. Navigate to the job step `Building server-dev component`, ensure that log does not have an error on the step: + ```shell + [1/2] STEP 7/7: RUN --mount=type=secret,id=SSH_KEY,dst=id_rsa cd /app && git submodule update --init --remote --merge --recursive --verbose + ``` +* The application should be up and running \ No newline at end of file From cd9188cdb58cdbbb35660f291969fd6ee55dac4f Mon Sep 17 00:00:00 2001 From: Sergey Smolnikov Date: Tue, 1 Oct 2024 10:55:28 +0200 Subject: [PATCH 2/2] Extended info --- .../guides/git-submodules/update-submodule-in-pipeline-job.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md b/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md index 5428ee99..d1766a22 100644 --- a/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md +++ b/public-site/docs/guides/git-submodules/update-submodule-in-pipeline-job.md @@ -95,14 +95,14 @@ The default file name with a private key can be changed with one of following op ```shell cat private-key-file ``` - * the public key need to be copy-pasted to a new deploy-key in the submodule's GitHub repository: `Repository/Settings/Deploy keys/Add deploy key`. `Allow write access` is not needed. + * the public key need to be copy-pasted (the whole text, ending with email) to a new deploy-key in the submodule's GitHub repository: `Repository/Settings/Deploy keys/Add deploy key`. `Allow write access` is not needed. ```shell cat private-key-file.pem` ``` ## Register and deploy Radix application * Register a Radix application in the Radix cluster * Create a first pipeline job `build-deploy` -* This job will fail due to a build secret `SSH_KEY` is not populated. This can be fixed within the Radix console, "Configuration" page of the application, the section "Build secrets" - click on the secret name `SSH_KEY` and copy-paste the private key from the previous step +* This job will fail due to a build secret `SSH_KEY` is not populated. This can be fixed within the Radix console, "Configuration" page of the application, the section "Build secrets" - click on the secret name `SSH_KEY` and copy-paste the private key from the previous step (the whole text, starting with `-----BEGIN RSA PRIVATE KEY-----` and ending with `-----END RSA PRIVATE KEY-----`). * Create a new `build-deploy` pipeline job - when it is completed. Navigate to the job step `Building server-dev component`, ensure that log does not have an error on the step: ```shell [1/2] STEP 7/7: RUN --mount=type=secret,id=SSH_KEY,dst=id_rsa cd /app && git submodule update --init --remote --merge --recursive --verbose