GitHub organization self-hosted runners in Azure Container Apps

This repository is a starter for hosting an organization's GitHub Actions runners in Azure Container Apps.
It contains Bicep code to provision the resources, a simple Dockerfile and GitHub Actions workflows to automate everything and test the self-hosted runners.
It was side-created with a series of blog posts in two parts: the first one sets a single runner and the second one adds auto-scaling.

Getting started

The best way to use this is to fork this repository, and set-up your fork to connect with GitHub and your Azure subscription.
You will need:

• A GitHub organization: this repo is not for runners associated to a personal account, so you need an organization. You can create on for free if you need.
• An Azure subscription

Fork this repo

Let's start by forking this repo. In the Owner dropdown, make sure to select your organization and not your personal account. You can leave the other settings as they are, and click on the Create fork button.

Create a GitHub App

Self-hosted runners interact with the GitHub REST API to register themselves and query queued jobs. The workflows in this repo also interact with the REST API to set variables.
The recommended authentication method against the GitHub REST API in the context of an organization is to use a GitHub App, let's create one for the runners and the workflows.

From your organization settings, click on Developer Settings, then GitHub Apps and New GitHub App. Give it a name and a homepage URL (any URL will work), and disable the Webhook feature.
The important settings are the permissions, set them as follow:

• In Repository permissions:
  • Set Actions to Read-only
  • Set Metadata to Read-only (it should be selected by default)
  • Set Variables to Read and write.
• In Organization permissions:
  • Set Administration to Read-only
  • Set Self-hosted runners to Read and write

Keep the other settings as default and click on Create GitHub App. On the next page you are prompted to generate a private key: do this and your browser will download a .pem file. Note also the id of your app as you will need it in a few seconds.

Then you need to install your GitHub App to grant the permissions defined above in your organization. You can choose to give it the access to all your repos or just some of them. At least include your fork otherwise it won't work.

Lastly, in the settings of your fork, go to Secrets and variables, and Actions. Create a secret named GH_APP_PRIVATE_KEY with your GitHub App private key (the content of the .pem file) and two variables named GH_APP_ID and GH_APP_INSTALLATION_ID with the GitHub App id and installation id as values

Tip

You can find the installation id from the settings of your organization or repo, in Third-party Access, and GitHub Apps. Click on Configure next to your app and you'll find the installation id in the URL.

Connect GitHub with Azure

To grant access to your Azure subscription to the GitHub Action runners, you need to create a service principal with the owner role to your subscription (or contributor and user access administrator roles).

Tip

This use of privileged role(s) is necessary to create a role assignment in the Bicep code. If you have an Entra P1 or P2 license your can also create a custom role for finer-grained control

To create your service principal, follow the instructions here, until you have added federated credentials and create the following variables:

• AZURE_TENANT_ID with your tenant id
• AZURE_CLIENT_ID with your application (client) id
• AZURE_SUBSCRIPTION_ID with your subscription id
• AZURE_LOCATION with the Azure region you want to create the resources in (not related to the GitHub-Azure connection but better set it while already setting variables)

Note

No client secret is required thanks to OpenID Connect and federated credentials

Now that everything is set-up, you can start to deploy some resources.

Deploy the prerequisites

The first workflow to run is Deploy prerequisites from the Actions tab in your fork. It will create the following resources in your Azure subscription:

• A resource group named rg-aca-gh-runners
• A Container Apps environment
• A Container registry
• A Log Analytics workspace
• A Key Vault containing the GitHub App private key
• A user-assigned Managed Identity with pull access to the registry and secret user access on the Key Vault

It will also build a container image from the Dockerfile here which is based on the work from this great repo and push it to your registry with the tag runners/github/linux:from-base.

Lastly it sets a few deployment outputs as variables so that the next workflow can re-use them.

Note

The workflow generates an access token using the GitHub App to create variables to store values for the next workflow.

Deploy the runners

Next workflow to run is Create and register self-hosted runners. This one uses Bicep to deploy a Container App (Job) in the Container App Environment, using the container image built and pushed by the previous workflow.

Note

The deployment is split in two parts as the Container App (Job) needs a container image already pushed to a Container Registry. A single workflow could have been used but would take too long to execute.

When you launch the workflow, you can choose between deploying a Container App or a Container App Job. The job is used by default as it's a better fit for this scenario.

Once the workflow has finished you should see the Container App Job (or the App) in your resource group. Checking the result depends on type of deployed app.

Using Container Apps

In the _Revisions_ panel of the Container App, you should see an active revision and in the _Log Stream_ panel, a message indicating the successful connection to GitHub:

Runner reusage is disabled
Obtaining the token of the runner
Ephemeral option is enabled
Configuring
--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
√ Runner successfully added
√ Runner connection is good
# Runner settings
√ Settings Saved.
√ Connected to GitHub
Current runner version: '2.311.0'
2023-11-22 15:48:14Z: Listening for Jobs

You should also see the runner in the settings of your fork (in Settings > Actions > Runners):
You can also see it in the settings of your organization.

Using Container Apps Jobs

Jobs need to be triggered to appear as a runner in GitHub. At first you can check that the Container App Job has been created in the Azure portal, and the Execution history is empty.

The new runner(s) will be in the Default runner group of your GitHub organization. If you have forked this repository to your own organization, it will be a public repository. In order to make the new runner(s) available to public repositories, you need to check the "Allow public repositories" checkbox in the settings of the Default runner group. You can find this setting under Your organization -> Settings -> Actions -> Runner groups -> Default.

Test the self-hosted runners

To test the runner, simply run the Test self-hosted runners workflow. This is a simple workflow that connects to Azure and run Azure CLI commands to output the account used and the list of resource groups in the subscription.
You can trigger the workflow several times to see how the runner scales in response to queued jobs.

Important

Notice the use of the runs-on: self-hosted property of the single job. It means that the job has to run on a self-hosted runner, whereas the previous workflow run on runners managed by GitHub (using the runs-on: ubuntu-latest property).

Once the workflow manually triggered, you can check that the jobs are picked up by the self-hosted runners from the GitHub Actions UI or from the Azure portal:

• For Container Apps, you can use the Log stream panel
• For Container Apps Jobs, you can use the Execution history panel or drill into the logs