Skip to content

Latest commit

 

History

History
204 lines (138 loc) · 19.8 KB

cicd.md

File metadata and controls

204 lines (138 loc) · 19.8 KB
Raw

Continuous Integration and Delivery

Content

Workflows

Several workflows exist for different purposes:

Workflow Schedule/Trigger Checks native delivery web delivery Version bump Move release notes
commit commits of PRs ✔️ ❌ ❌ ❌ ❌
commit_main commits on main ❌ browserstack webnext ❌ ❌
delivery Monday (04:00), script ✔️ beta beta ✔️ ✔️
native_promotion Monday (02:00), script ❌ promotion ❌ ❌ ❌
web_promotion Monday (02:00), script ❌ ❌ promotion ❌ ❌
native_beta_delivery script ✔️ beta ❌ ✔️ ❌
native_production_delivery script ✔️ production ❌ ✔️ ❌
web_beta_delivery script ✔️ ❌ beta ✔️ ❌
web_production_delivery script ✔️ ❌ production ✔️ ❌
e2e_tests commits on main ❌ ❌ ❌ ❌ ❌

Steps executed if Checks is checked ✔️:

  • Linting
  • Prettier formatting
  • TypeScript checks
  • Unit testing with jest
  • Building the app
  • E2E tests

Steps executed if Version bump is checked ✔️:

  • Bump version: Bump the version(s) and create a tag and release on github

Failed Delivery

Sometimes it happens that one or multiple steps of our scheduled CI delivery workflow fails. In that case, you should not use the Restart Workflow from Start (as this will lead to just another failure since the version number was bumped before but not in the state of the failed delivery workflow such that it attempts to create the same releases again).

If the reason for a delivery to fail was just a transient error that was fixed in the meantime and doesn't require a code change (e.g. network error, API down, problems in the stores), you can use the Restart Workflow from Failed and in the best case the workflow should finish now.

In all other cases you can decide if there is an important change that should be delivered asap and can't wait another week. Examples are important bug fixes or big new features that should be released on multiple platforms in the same time. In that case you can trigger a delivery. It is possible to either just execute web_beta_delivery or native_beta_delivery or just run the whole delivery workflow again. If that is not the case, just waiting one week is also fine.

Triggering a Delivery

Normally the scheduled deliveries (see workflows) should be frequent enough for most purposes.

If you still have to deliver a new reason for example for an urgent bug fix or because of a failed CI delivery, the easiest way to deliver a new build to production or beta is to trigger the corresponding CircleCI workflows native_beta_delivery, native_production_delivery, web_beta_delivery and web_production_delivery:

  • Get a CircleCI Personal API Token.
  • Trigger a delivery using the tool trigger-pipeline: cd tools && yarn trigger-pipeline trigger <workflow-type> --api-token <api-token>

Services

deliverino (GitHub)

deliverino is a GitHub App and can be accessed and installed here. This bot bumps the version of the app when a new release is delivered. A private key in PEM format grants access to the bot. If the deliverino is installed for a specific repository then it has access to create commits there.

deliverino has the role of an Administrator. This is important when setting up Protected branches in GitHub. You have to disable "Include Administrators", else deliverino is not allowed to directly commit to the protected branch.

Google Play Store

You can visit the management website for the Play Store here. The Google Play Console is the product by Google for managing the App Store presence.

Adding Testers to the Beta Track

The Play Store has the concept of tracks to manage released versions of the app. The beta track is for public tests. Tests can be added via their Google E-Mail or by signing up at play.google.com/apps/testing/tuerantuer.app.integreat.

Metadata

The CI/CD pipeline uploads and overwrites metadata during the delivery step. You can read more about managing metadata for Android here.

App Store Connect

You can visit the management website for the Play Store here. App Store Connect is the product by Apple for managing the App Store presence.

For delivery an account without 2FA is required.

Adding Testers to TestFlight

The scheduled_native_production_workflow makes the builds directly available to TestFlights "App Store Connect Users". Those should not be confused with "External Tests" which require an approval by apple. Therefore, we currently only use "App Store Connect Users" as testers.

In order to add someone as "App Store Connect User" you have to add the Apple Account to App Store Connect and to TestFlight. This is a two-step process.

Metadata

The CI/CD pipeline uploads and overwrites metadata during the delivery step. You can read more about managing metadata for iOS here.

Authenticating

Authentication happens by setting the APP_STORE_CONNECT_API_KEY_CONTENT environment variable as documented above. For more information visit the documentation here.

BrowserStack

We are using BrowserStack to run our E2E tests on real iOS and Android devices. The general documentation about E2E tests and BrowserStack for native development can be found here.

Fastlane

Fastlane is a task-runner for triggering build relevant tasks. It offers integration with XCode and the Android SDK for building and delivering the app.

Fastlane Setup

  • Install Ruby >= 2.6.5
  • Make sure ruby --version reports the correct version.
  • Run bundle install --path vendor/bundle in the project root AND in ./android/ AND in ./ios/.
  • Run bundle exec fastlane --version.

Hint: You can run export FASTLANE_SKIP_UPDATE_CHECK=true to skip the changelog output.

Lanes

Lanes for Android live in ../native/android/fastlane and for iOS in ../native/ios/fastlane. Shared lanes are in ../native/fastlane.

An overview about FL lanes is available in several documents:

  • General - Responsible for delivering and uploading artifacts.
  • Android - Responsible for setting up the signing keys and building the Android app.
  • iOS - Responsible for setting up the certificates and building the iOS app.

Apple Certificates and Android Keystore

Fastlane is used to setup certificates and keystores. The detailed steps of the CI/CD pipeline are the same as those when manually building the app. Therefore, you can follow the documentation for Manual Builds to set up certificates for iOS and keystores for android.

Determining the Next Version

The next version of the app must be determined programmatically. The tool next-version can be used. More information on the version naming schema used can be found here.

Environment Variables and Dependencies

Variable Description Where do I get it from? Example Reference
BROWSERSTACK_ACCESS_KEY Access Key for BrowserStack Password Manager DEADBEEF Appium REST API
BROWSERSTACK_USERNAME Username for BrowserStack Password Manager 123546 Appium REST API
DELIVERINO_PRIVATE_KEY Base64 encoded PEM private key Password Manager Deliverino Settings Deliverino
SENTRY_AUTH_TOKEN Auth Token from Sentry for uploading sourcemaps and artifacts Generate this in your Sentry account with the scope project:releases deadbeef Sentry Authentication
MM_WEBHOOK URL which can be used to send notifications to our mattermost. Keep this private! Mattermost server settings https://chat.tuerantuer.org/hooks/... Mattermost Documentation

Android Variables

Variable Description Where do I get it from? Example Reference
GOOGLE_SERVICE_ACCOUNT_JSON JSON for authentication in the Google Play Console as Release Manager. This should expire after two years. Password Manager {...} Service Account Docu
CREDENTIALS_GIT_REPOSITORY_URL Git remote URL to the credentials repository which contains the Java Keystore Ask the team about this secret repository git@github.com:User/credentials.git -
CREDENTIALS_DIRECTORY_PATH Path where the credentials Git repository cloned to automatically by FL The developer can choose this freely /home/circleci/credentials -
CREDENTIALS_KEYSTORE_PATH Path to the OpenSSL AES256-CBC encrypted Java Keystore file - /home/circleci/credentials/.enc Look for the openssl enc command in the Android Fastlane file for more information
KEYSTORE_PATH Path to the decrypted Java Keystore file - /home/circleci/keystore.jks -
CREDENTIALS_KEYSTORE_PASSWORD Password for decrypting the keystore using OpenSSL password -
KEYSTORE_KEY_ALIAS Alias of the key within the Java Keystore You should look in the JKS file using keytool -list -v -keystore <jks> my-key -
KEYSTORE_KEY_PASSWORD Password of the key within the Java Keystore Password Manager 123456 -
KEYSTORE_PASSWORD Password of the JKS which can contain multiple keys Password Manager 123456 -

iOS Variables

Variable Description Where do I get it from? Example Reference
APP_STORE_CONNECT_API_KEY_ID Key ID for App Store Connect API Password Manager D83848D23 app_store_connect_api_key
APP_STORE_CONNECT_API_ISSUER_ID Issuer ID for App Store Connect API Password Manager 227b0bbf-ada8-458c-9d62-3d8022b7d07f app_store_connect_api_key
APP_STORE_CONNECT_API_KEY_CONTENT Key content for App Store Connect API Password Manager -----BEGIN EC PRIVATE KEY-----\nfewfawefawfe\n-----END EC PRIVATE KEY----- app_store_connect_api_key
MATCH_PASSWORD Password for accessing the certificates for the iOS app using Fastlane Match Password Manager 123456 Using a Git Repo

Skipping specific jobs

You can control which jobs should be skipped through environment variables. Set the variable SKIP_JOB_deliver_aschaffenburg_ios to "aschaffenburg" to skip the job with the name deliver_aschaffenburg_ios. You can also set it to "malte|aschaffenburg" in order to match multiple build configs or to "all" to match all build configs.

Environment variables can be set in the Project Settings of CircleCI.

Note: Some jobs like bump_version run only once for multiple build configs. Therefore, it does not make sense to set SKIP_JOB_bump_version to something other than "all".

Note: Most of the time job names contain the build config name as well, therefore setting e.g. the environment variable SKIP_JOB_deliver_ios won't work.

Hints and Quirks

CPU count aka. $TOTAL_CPUS

There is no obvious way for an application to know how many cores it has available in a CircleCI docker container. The usual ways of getting the CPU count reports the CPU count of the host. This causes out-of-memory issues as the host has a lot of cores. Therefore, all tools must set the worker limit to $TOTAL_CPUS. Set this variable in the .circleci/config.yml.