test(e2e): separate android, iOS tests; add buildcache to iOS e2e #5239

mikehardy · 2021-04-30T22:39:56Z

Description

Allow separate runs of the two main native platforms instead of forcing re-run of both when one flakes

Add buildcache to iOS build in an attempt to speed it up

Tried ccache but it can't handle multiple source files at once and that's just about all Xcode does these days
Tried buildcache in "direct" mode (should be way faster...) but it doesn't work because "can't open dependency file".

This is worth 50% on the iOS build step, even with the firestore-ios-sdk-framework speedup involved.

Release Summary

conventional commits

Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
- Yes
My change supports the following platforms;
- Android
- iOS
My change includes tests;
- e2e tests added or updated in packages/\*\*/e2e
- jest tests added or updated in packages/\*\*/__tests__
I have updated TypeScript types that are affected by my change.
This is a breaking change;
- Yes
- No

Test Plan

CI passes with reasonable cache hit ratio on ccache

Think react-native-firebase is great? Please consider supporting the project with any of the below:

👉 Star this repo on GitHub ⭐️
👉 Follow React Native Firebase and Invertase on Twitter

Frequently one fails but not the other and it is a tremendous waste to have to re-run both all the time

even if we don't have a full cache match, a fallback should be okay in some cases

vercel · 2021-04-30T22:39:59Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/invertase/react-native-firebase/3ehVVRtg5pNJN68kwZoDW88o8J8L
✅ Preview: https://react-native-firebase-git-mikehardy-ci-e2e-grooming-invertase.vercel.app

codecov · 2021-04-30T22:41:06Z

Codecov Report

Merging #5239 (b4950df) into master (3396da2) will not change coverage.
The diff coverage is n/a.

❗ Current head b4950df differs from pull request most recent head 472c996. Consider uploading reports for the commit 472c996 to get more accurate results

@@           Coverage Diff           @@
##           master    #5239   +/-   ##
=======================================
  Coverage   88.86%   88.86%           
=======================================
  Files         109      109           
  Lines        3743     3743           
  Branches      360      360           
=======================================
  Hits         3326     3326           
  Misses        370      370           
  Partials       47       47

mikehardy · 2021-04-30T23:49:46Z

ccache somehow not being used for that many files, but sped up 10% currently

going to do some local tuning and see if it can be improved as it appears it is not even attempting to use cache most of the time 🤔 - locally it was


Fri, 30 Apr 2021 23:32:02 GMT   echo /usr/lib/ccache:/usr/local/opt/ccache/libexec >> $GITHUB_PATH
Fri, 30 Apr 2021 23:32:02 GMT   ccache --set-config=sloppiness=pch_defines,time_macros,file_macro,include_file_mtime,include_file_ctime,file_stat_matches,modules,clang_index_store,system_headers,ivfsoverlay
Fri, 30 Apr 2021 23:32:02 GMT   ccache --set-config=depend_mode=true
Fri, 30 Apr 2021 23:32:02 GMT   ccache --set-config=file_clone=true
Fri, 30 Apr 2021 23:32:02 GMT   ccache --set-config=inode_cache=true
Fri, 30 Apr 2021 23:32:02 GMT   ccache -p
Fri, 30 Apr 2021 23:32:02 GMT   ccache -s
Fri, 30 Apr 2021 23:32:02 GMT   shell: /bin/bash -e {0}
Fri, 30 Apr 2021 23:32:02 GMT   env:
Fri, 30 Apr 2021 23:32:02 GMT     CODECOV_TOKEN: ***
Fri, 30 Apr 2021 23:32:02 GMT     MD_APPLE_SDK_ROOT: /Applications/Xcode_12.4.app
Fri, 30 Apr 2021 23:32:02 GMT (default) absolute_paths_in_stderr = false
Fri, 30 Apr 2021 23:32:02 GMT (default) base_dir = 
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) cache_dir = /Users/runner/work/react-native-firebase/react-native-firebase/.ccache
Fri, 30 Apr 2021 23:32:02 GMT (default) compiler = 
Fri, 30 Apr 2021 23:32:02 GMT (default) compiler_check = mtime
Fri, 30 Apr 2021 23:32:02 GMT (default) compiler_type = auto
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) compression = true
Fri, 30 Apr 2021 23:32:02 GMT (default) compression_level = 0
Fri, 30 Apr 2021 23:32:02 GMT (default) cpp_extension = 
Fri, 30 Apr 2021 23:32:02 GMT (default) debug = false
Fri, 30 Apr 2021 23:32:02 GMT (default) debug_dir = 
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) depend_mode = true
Fri, 30 Apr 2021 23:32:02 GMT (default) direct_mode = true
Fri, 30 Apr 2021 23:32:02 GMT (default) disable = false
Fri, 30 Apr 2021 23:32:02 GMT (default) extra_files_to_hash = 
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) file_clone = true
Fri, 30 Apr 2021 23:32:02 GMT (default) hard_link = false
Fri, 30 Apr 2021 23:32:02 GMT (default) hash_dir = true
Fri, 30 Apr 2021 23:32:02 GMT (default) ignore_headers_in_manifest = 
Fri, 30 Apr 2021 23:32:02 GMT (default) ignore_options = 
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) inode_cache = true
Fri, 30 Apr 2021 23:32:02 GMT (default) keep_comments_cpp = false
Fri, 30 Apr 2021 23:32:02 GMT (default) limit_multiple = 0.8
Fri, 30 Apr 2021 23:32:02 GMT (default) log_file = 
Fri, 30 Apr 2021 23:32:02 GMT (default) max_files = 0
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) max_size = 500.0M
Fri, 30 Apr 2021 23:32:02 GMT (default) path = 
Fri, 30 Apr 2021 23:32:02 GMT (default) pch_external_checksum = false
Fri, 30 Apr 2021 23:32:02 GMT (default) prefix_command = 
Fri, 30 Apr 2021 23:32:02 GMT (default) prefix_command_cpp = 
Fri, 30 Apr 2021 23:32:02 GMT (default) read_only = false
Fri, 30 Apr 2021 23:32:02 GMT (default) read_only_direct = false
Fri, 30 Apr 2021 23:32:02 GMT (default) recache = false
Fri, 30 Apr 2021 23:32:02 GMT (default) run_second_cpp = true
Fri, 30 Apr 2021 23:32:02 GMT (/Users/runner/Library/Preferences/ccache/ccache.conf) sloppiness = include_file_mtime, include_file_ctime, time_macros, pch_defines, file_stat_matches, system_headers, clang_index_store, modules
Fri, 30 Apr 2021 23:32:02 GMT (default) stats = true
Fri, 30 Apr 2021 23:32:02 GMT (default) temporary_dir = /Users/runner/work/react-native-firebase/react-native-firebase/.ccache/tmp
Fri, 30 Apr 2021 23:32:02 GMT (default) umask = 
Fri, 30 Apr 2021 23:32:02 GMT cache directory                     /Users/runner/work/react-native-firebase/react-native-firebase/.ccache
Fri, 30 Apr 2021 23:32:02 GMT primary config                      /Users/runner/Library/Preferences/ccache/ccache.conf
Fri, 30 Apr 2021 23:32:02 GMT secondary config (readonly)         /usr/local/Cellar/ccache/4.2.1/etc/ccache.conf
Fri, 30 Apr 2021 23:32:02 GMT stats updated                       Fri Apr 30 23:32:02 2021
Fri, 30 Apr 2021 23:32:02 GMT stats zeroed                        Fri Apr 30 23:32:01 2021
Fri, 30 Apr 2021 23:32:02 GMT cache hit (direct)                     0
Fri, 30 Apr 2021 23:32:02 GMT cache hit (preprocessed)               0
Fri, 30 Apr 2021 23:32:02 GMT cache miss                             0
Fri, 30 Apr 2021 23:32:02 GMT cache hit rate                      0.00 %
Fri, 30 Apr 2021 23:32:02 GMT cleanups performed                     0
Fri, 30 Apr 2021 23:32:02 GMT files in cache                       414
Fri, 30 Apr 2021 23:32:02 GMT cache size                          34.1 MB
Fri, 30 Apr 2021 23:32:02 GMT max cache size                     500.0 MB

Fri, 30 Apr 2021 23:47:21 GMT Post job cleanup.
Fri, 30 Apr 2021 23:47:21 GMT Ccache stats:
Fri, 30 Apr 2021 23:47:21 GMT /usr/local/bin/ccache -s
Fri, 30 Apr 2021 23:47:21 GMT cache directory                     /Users/runner/work/react-native-firebase/react-native-firebase/.ccache
Fri, 30 Apr 2021 23:47:21 GMT primary config                      /Users/runner/Library/Preferences/ccache/ccache.conf
Fri, 30 Apr 2021 23:47:21 GMT secondary config (readonly)         /usr/local/Cellar/ccache/4.2.1/etc/ccache.conf
Fri, 30 Apr 2021 23:47:21 GMT stats updated                       Fri Apr 30 23:40:54 2021
Fri, 30 Apr 2021 23:47:21 GMT stats zeroed                        Fri Apr 30 23:32:01 2021
Fri, 30 Apr 2021 23:47:21 GMT cache hit (direct)                   138
Fri, 30 Apr 2021 23:47:21 GMT cache hit (preprocessed)               0
Fri, 30 Apr 2021 23:47:21 GMT cache miss                             0
Fri, 30 Apr 2021 23:47:21 GMT cache hit rate                    100.00 %
Fri, 30 Apr 2021 23:47:21 GMT multiple source files               2116
Fri, 30 Apr 2021 23:47:21 GMT cleanups performed                     0
Fri, 30 Apr 2021 23:47:21 GMT files in cache                       414
Fri, 30 Apr 2021 23:47:21 GMT cache size                          34.1 MB
Fri, 30 Apr 2021 23:47:21 GMT max cache size                     500.0 MB
Fri, 30 Apr 2021 23:47:21 GMT Save cache using key "ccache-2021-04-30T23:47:21.691Z".
Fri, 30 Apr 2021 23:47:21 GMT /usr/bin/tar --posix --use-compress-program zstd -T0 -cf cache.tzst -P -C /Users/runner/work/react-native-firebase/react-native-firebase --files-from manifest.txt
Fri, 30 Apr 2021 23:47:22 GMT Cache saved successfully

cold cache is 20% slower, warm cache is 50% faster

mikehardy added 3 commits April 30, 2021 17:00

test(e2e): split e2e into separate ios and android

3eed241

Frequently one fails but not the other and it is a tremendous waste to have to re-run both all the time

test(e2e): move xcode steps next to each other

95180c0

test(e2e): name the caching steps and try restore keys

5f9acaa

even if we don't have a full cache match, a fallback should be okay in some cases

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from ef9c584 to de4aa82 Compare April 30, 2021 22:58

vercel bot deployed to Preview April 30, 2021 22:58 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from de4aa82 to b2774a4 Compare May 1, 2021 03:06

vercel bot deployed to Preview May 1, 2021 03:06 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from b2774a4 to 567d596 Compare May 1, 2021 03:18

vercel bot deployed to Preview May 1, 2021 03:18 View deployment

vercel bot deployed to Preview May 1, 2021 03:47 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from e4bb19c to 15b21f4 Compare May 1, 2021 04:05

vercel bot deployed to Preview May 1, 2021 04:05 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from 15b21f4 to 2458a80 Compare May 1, 2021 04:30

vercel bot deployed to Preview May 1, 2021 04:30 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from 2458a80 to 323c1a9 Compare May 1, 2021 04:37

vercel bot deployed to Preview May 1, 2021 04:37 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from 323c1a9 to e0e2fe1 Compare May 1, 2021 04:51

vercel bot deployed to Preview May 1, 2021 04:51 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from e0e2fe1 to e6ba1ac Compare May 1, 2021 05:04

vercel bot deployed to Preview May 1, 2021 05:04 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from e6ba1ac to 76a7856 Compare May 1, 2021 05:28

vercel bot deployed to Preview May 1, 2021 05:28 View deployment

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from 76a7856 to abb8e7e Compare May 1, 2021 05:47

vercel bot deployed to Preview May 1, 2021 05:47 View deployment

mikehardy changed the title ~~test(e2e): separate android, iOS tests; add ccache to iOS e2e~~ test(e2e): separate android, iOS tests; add buildcache to iOS e2e May 1, 2021

mikehardy added the Workflow: Pending Merge Waiting on CI or similar label May 1, 2021

mikehardy mentioned this pull request May 1, 2021

test(e2e, ios): deterministically exclude leveldb #5245

Merged

8 tasks

test(e2e, ios): add buildcache to ios e2e run to speed up build

472c996

cold cache is 20% slower, warm cache is 50% faster

mikehardy force-pushed the @mikehardy/ci-e2e-grooming branch from abb8e7e to 472c996 Compare May 1, 2021 19:53

vercel bot deployed to Preview May 1, 2021 19:53 View deployment

mikehardy removed the Workflow: Pending Merge Waiting on CI or similar label May 1, 2021

mikehardy merged commit f6a9825 into master May 1, 2021

mikehardy deleted the @mikehardy/ci-e2e-grooming branch May 1, 2021 19:54

This was referenced May 4, 2021

test(e2e, ios): buildcache via an action vs in situ #5257

Merged

fix(ios): admob shows only non-personalized ads #5262

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(e2e): separate android, iOS tests; add buildcache to iOS e2e #5239

test(e2e): separate android, iOS tests; add buildcache to iOS e2e #5239

mikehardy commented Apr 30, 2021 •

edited

Loading

vercel bot commented Apr 30, 2021 •

edited

Loading

codecov bot commented Apr 30, 2021 •

edited

Loading

mikehardy commented Apr 30, 2021 •

edited

Loading

test(e2e): separate android, iOS tests; add buildcache to iOS e2e #5239

test(e2e): separate android, iOS tests; add buildcache to iOS e2e #5239

Conversation

mikehardy commented Apr 30, 2021 • edited Loading

Description

Release Summary

Checklist

Test Plan

vercel bot commented Apr 30, 2021 • edited Loading

codecov bot commented Apr 30, 2021 • edited Loading

Codecov Report

mikehardy commented Apr 30, 2021 • edited Loading

mikehardy commented Apr 30, 2021 •

edited

Loading

vercel bot commented Apr 30, 2021 •

edited

Loading

codecov bot commented Apr 30, 2021 •

edited

Loading

mikehardy commented Apr 30, 2021 •

edited

Loading