Address 'deploy' step in the CI/CD process on CircleCI (core repos)

[elnyry-sam-k]

Goal:

As an OSS codebase maintainer
I want to ensure all steps in the CI/CD process (using CircleCI) process of the core repos are relevant and up-to-date
so that it ensures quality of the repositories, services and only includes steps that are applicable

Tasks:
- [ ] Identify which of the options - either to make deploy step work or removing it, makes sense in this context Decided to fix the step

• Address the 'deploy' phase in the circleCI CI/CD process to either disable it and get it working with the core-team's dev environment (dev1 or dev2)
• Apply fixes to the mojaloop docker targets:
  • quoting-service
  • account-lookup-service - PR
    - [ ] als-oracle-pathfinder Out of scope for now
  • bulk-api-adapter - PR
  • central-event-processor - PR
  • central-ledger - PR
  • central-settlement - PR
  • email-notifier - PR
  • event-sidecar - PR
  • event-stream-processor - PR
  • finance-portal-backend-service - PR
  • finance-portal-ui - PR
  • ml-api-adapter - PR
  • operator-settlement - PR
  • settlement-management - PR
  • simulator - PR
  • transaction-requests-service - PR

Acceptance Criteria:

• 'Deploy' phase that is part of the release cycle in the CI/CD process (on CircleCI) is either removed or is made to work (on to core-team's dev environment)

Pull Requests:

• Fix/1107 circleci deploy quoting-service#154
• Fix/1107 circleci deploy account-lookup-service#245
• Fix/1107 circleci deploy bulk-api-adapter#23
• Fix/1107 circleci deploy central-event-processor#152
• Fix/1107 circleci deploy central-ledger#616
• Fix/1107 circleci deploy central-settlement#261
• fix/1107-circleci-deploy email-notifier#125
• fix/1107-circleci-deploy event-sidecar#21
• fix/1107-circleci-deploy event-stream-processor#24
• fix/1107-circleci-deploy finance-portal-backend-service#40
• fix/1107-circleci-deploy finance-portal-ui#31
• fix/1107-circleci-deploy ml-api-adapter#362
• fix/1107-circleci-deploy operator-settlement#11
• fix/1107-circleci-deploy settlement-management#9
• fix/1107-circleci-deploy simulator#151
• fix/1107-circleci-deploy transaction-requests-service#21

Follow-up:

• N/A

Dependencies:

• N/A

Accountability:

• Owner: TBC
• QA/Review: @mdebarros

[lewisdaly]

I was also looking into using CircleCI orbs (it's a feature they added about 1 year ago) to make some pieces of config that can be easily shared between our projects.

https://circleci.com/docs/2.0/creating-orbs/

This means that any major configuration changes in the future could be updated across all CI/CD pipelines without updating each and every repo.

[lewisdaly]

I managed to get an 'inline' CircleCI orb set up for this on the email notifier.

Just about everything looks like it will work, the only issue being I need the latest K8_CLUSTER_SERVER url, which is currently in CircleCI, but I'm pretty sure is outdated.

[lewisdaly]

@rmothilal @mdebarros in the past it looks like we deployed to a 'snapshot' cluster for snapshot releases, and a different cluster for normal releases.

Do we still want that functionality? Or we we happy to just deploy to dev1 to start off with?

[lewisdaly]

I managed to get everything working assuming a dev1 deployment locally.

My next challenge where I'm a little stuck is with the setting up kubectl auth. I was using my own credentials as a test, which obviously isn't a long term solution.

I've been working on setting up a proper service account for circle-ci, and creating a secret, etc. but am now rather stuck. If anyone has any input on this, that would be very helpful!

[lewisdaly]

Moving to blocked for now.

[lewisdaly]

@mdebarros has provided an access token for the ci/cd, so we are all good to go.

[lewisdaly]

• event-stream-processor isn't in the root values.yml file, so can't be overriden as part of the deployment.

[lewisdaly]

Ok, we're done with all the PRs!