Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[No QA] Make GitHub Actions build asynchronously #3702

Merged
merged 1 commit into from
Jun 22, 2021

Conversation

roryabraham
Copy link
Contributor

Details

...Just something I worked on during a flight, mostly to get some much-needed practice with bash scripting. But still a nice time-saver!

This PR makes the GitHub Actions build script asynchronous, and it's considerably faster now.

Fixed Issues

n/a

Tests

Testing this was a pretty interesting task, and I mostly wanted to test two things:

  1. Does it produce the same output?
  2. Does it actually run faster?

To verify that the output is the same, I did the following:

  1. Starting from an up-to-date main branch, create a test branch: git checkout temp-test
  2. Make some changes to .github/libs/GithubUtils.js. This file is used by many GHA, so it's a good way to test the build script.
  3. Run npm run gh-actions-build, then commit the changes.
  4. git checkout main && git checkout Rory-AsyncGHActionsBuilds
  5. Make the same changes to .github/libs/GithubUtils.js.
  6. Run npm run gh-actions-build again
  7. Run git diff temp-test... There should be no diff.
  8. Cleanup by deleting your test branch: git checkout main && git branch -D temp-test

To verify that the script actually runs the build faster:

  1. Add the following function to your bash profile (or just copy/paste it into your shell):

    # This is a basic shell utility to benchmark a bash command
    # @param $1 – The command to run
    # @param $2 - The number of time to run the command for benchmarking purposes
    function benchmark(){
      if [ -z "$1" ]; then
        echo "[Benchmark Err] – No command input"
        exit 1
      fi
      
      if [ -z "$2" ]; then
        echo "[Benchmark Err] – No number input"
        exit 1
      fi
      
      if [ "$2" -le 0 ]; then
        echo "[Benchmark Err] – Invalid number of uses – must be greater than 0"
        exit 1
      fi
      
      time eval "
          for ((i = 0; i < $2; i++)); do
            $1 &>/dev/null
          done
      "
    }
  2. Then from an up-to-date main, run:

    > benchmark ./.github/scripts/buildActions.sh 1 
    
    real    0m15.119s
    user    0m20.259s
    sys     0m1.898s
    
    > benchmark ./.github/scripts/buildActions.sh 10
    
    real    2m38.679s
    user    3m32.631s
    sys     0m20.456s
  3. Then on Rory-AsyncGHActionsBuilds run:

    > benchmark ./.github/scripts/buildActions.sh 1
    
    real    0m5.416s
    user    0m26.921s
    sys     0m2.615s
    
    > benchmark ./.github/scripts/buildActions.sh 10
    
    real    0m53.952s
    user    4m24.140s
    sys     0m24.770s

It's not a perfect experiment, but it does appear to be definitively faster – roughly 1/3 the execution time of a synchronous build.

Tested On

n/a – bash only

@roryabraham roryabraham requested a review from a team as a code owner June 21, 2021 06:58
@roryabraham roryabraham self-assigned this Jun 21, 2021
@MelvinBot MelvinBot requested review from deetergp and removed request for a team June 21, 2021 06:58
@deetergp
Copy link
Contributor

Just a heads up git diff temp-test.. will compare what's committed to your feature branch against what's committed in temp-test. Simply doing git diff temp-test will compare whatever changes I made in your feature branch to what's committed to temp-test, which is probably what you meant, since your instructions did not expressly say to commit the changes to your feature branch after running npm run gh-actions-build

# This stores all the process IDs of the ncc commands so they can run in parallel
declare ASYNC_BUILDS

for ((i=0; i < ${#GITHUB_ACTIONS[@]}; i++)); do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAB: TIL that ${#GITHUB_ACTIONS[@]} and ${#GITHUB_ACTIONS[*]} will return the number of elements in the GITHUB_ACTIONS array. Very cool!


# Build the action in the background
ncc build "$ACTION" -o "$ACTION_DIR" &
ASYNC_BUILDS[$i]=$!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAB: I've used !$ before, but $! (returning the PID of the last command) is a new one on me.

Copy link
Contributor

@deetergp deetergp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me a minute to figure out what's going on, but it tests well and the code makes sense.

@deetergp deetergp merged commit 3f3a97f into main Jun 22, 2021
@deetergp deetergp deleted the Rory-AsyncGHActionsBuilds branch June 22, 2021 15:45
@OSBotify
Copy link
Contributor

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@OSBotify
Copy link
Contributor

🚀 Deployed to staging in version: 1.0.73-4🚀

platform result
🤖 android 🤖 success ✅
🖥 desktop 🖥 success ✅
🍎 iOS 🍎 success ✅
🕸 web 🕸 success ✅

@OSBotify
Copy link
Contributor

🚀 Deployed to production in version: 1.0.74-0🚀

platform result
🤖 android 🤖 success ✅
🖥 desktop 🖥 success ✅
🍎 iOS 🍎 success ✅
🕸 web 🕸 success ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants