hub-dashboard-control-room

Note

This project is a proof of concept.

This repository contains operational workflows for the hubDashboard App. The workflow will generate predtimechart data and build a static site for each repository that installed the app.

Broad Architecture

The hubDashboard App allows hub administrators to opt-in to an auto-generated dashboard. By default, when anyone installs the app on their dashboard repository, the dashboard will be built once on the initial build and then every day on a schedule defined in this repository. The role of the app in this workflow is twofold:

1. provide a list of repositories that have it installed.
2. provide credentials for our workflows to write to branches of the dashboard

The workflow looks like this for each repository (note the build processes are in parallel):

sequenceDiagram
     participant GitHub
     participant App
     GitHub-->>App: Request installation name
     App->>GitHub: Installed: "dashboard-repo"

     create participant dashboard-repo
     GitHub-->>dashboard-repo: fetch "dashboard-repo"
     Note over GitHub: Builds static site to site/
     GitHub-->>dashboard-repo: Read hub repository name from config
     dashboard-repo->>GitHub: Hub Repository: "hub-repo"
     create participant hub as hub-repository
     GitHub-->>hub: fetch "hub-repo"
     Note over GitHub: Builds Predtimechart data to data/
     GitHub-->>App: Generate Access Token
     App-->>GitHub: Access Toekn
     GitHub->>dashboard-repo: Push site/ to gh-pages
     GitHub->>dashboard-repo: Push data/ to ptc/data

Loading

Tools

There are two tools that are used in this workflow. The processes are described below

• hub-dash-site-builder (Docker container to build the site with quarto)
• hub-dashboard-predtimechart (Python app to build predtimechart data for the forecasts)

On-Demand Builds

One of the advantages of using an app is that it can provide on-demand builds without requiring maintainers running or knowing how to run GitHub workflows.

It's useful to understand that a GitHub App is kind of like a mailbox. It can receive messages (webhooks) and send messages (API requests). Just like a mailbox, you cannot just stick one in a random place an assume that you can start receiving messages---you have to give it an address (a webserver).

In our case, we have hubDashboard (GitHub app) which is running on glitch (webserver). The scripts for this app live in app/index.js.

When the app is installed on a repository, GitHub knows that it has to send webhook events related to installations, issue comments, and pushes to the webserver address (which at the moment is https://crystal-glimmer-path.glitch.me/probot). When the app receives webhooks that it recognises, then it will send a GitHub API request with a repository dispatch event which will trigger a build.

All the pieces together look like this:

flowchart TD
    webserver{{"webserver (glitch)"}}


    app{"App"}
    app-workflows["control-room"]

    hub["org1/hub"]

    subgraph org1/site
    main>main]
    gh-pages>gh-pages]
    ptc/data>"ptc/data"]
    end

    

    org1/site==>|"sends webhook events"|webserver
    app-->|grants permission to|app-workflows
    webserver==>|"triggers build (via GH API)"|app-workflows
    app-->|runs on|webserver
    

    site1(["org1.github.io/site"])
    
    
    org1/site~~~site1
    app-workflows~~~hub
    hub~~~org1/site

    main-->|links to| hub
    app-->|installed on|org1/site
    main-.->|markdown content|app-workflows
    hub-.->|data|app-workflows
    app-workflows==>|writes site to|gh-pages
    app-workflows==>|writes data to|ptc/data
    gh-pages==>|deploys to|site1
    site1-.->|fetches data from|ptc/data

Loading

General workflow steps

In the two sections below, I detail the general implementations.

Predtimechart data

The general steps to generate the data for predtimechart are:

1. install hub-dashboard-predtimechart via

   pip install --upgrade pip
   pip install git+https://github.com/hubverse-org/hub-dashboard-predtimechart

2. clone the dashboard repository and enter it (you only need the predtimechart-config.yml)
3. clone the hub repository into repo/
4. Generate the Predtimechart data

   mkdir -p out/targets/
   mkdir -p out/forecasts/
   ptc_generate_target_json_files \
     repo \
     predtimechart-config.yml \
     out/targets
   ptc_generate_json_files \
     repo \
     predtimechart-config.yml \
     out/predtimechart-options.json \
     out/forecasts

5. enter repo/ and checkout the ptc/data branch
6. copy the contents of ../data to your current folder
7. add, commit, and push

Static Site

The static site is generated via the hubverse-org/hub-dash-site-builder container and writes a folder called _site/ under the pages/ folder of the dashboard repository. You need to then copy the contents of _site/ into the gh-pages branch of the dashboard repository.

1. clone the dashboard repository
2. Run the container:

   $ docker run \
     --platform=linux/amd64 \
     --rm \
     -ti \
     -v "/path/to/dashboard/repo":"/site" \
     ghcr.io/hubverse-org/hub-dash-site-builder:main \
     bash render.sh 

3. clone the gh-pages branch of the dashboard repository into pages/
4. copy the files from the dashboard repo/site/pages/_site/ folder into pages/
5. push the pages/ folder up.