Updating CI workflow (Deploying docker images to docker hub)

[walisc]

Ticket: N/A
Other Related Tickets: N/A

---

Describe of changes

This change update our github workflow by adding three new workflow, of which these are highlighted below

• OnPullRequest
• OnMerge
• Deploy Images to Docker Hub

OnPullRequest

This workflow is run when a pull request to dev or main is created. The jobs it executes are highlighted below

Below is a short description of each job

preprare

• Prepares the CI for the other jobs. This includes installing all necessary packages and software (e.g java, node, scala etc).
• Do note: These packages are cached (using the prepare file with the vesion number, and the yarn.lock file). If there are any strange behaviours related to these packages you could always try delete the cache (see Actions > Caches).

lint-check

• As the name suggests, this lints the project
• For java it uses the checkstyle plugin (with the already configure suppression files). For javascript it uses eslint

build-check

• This checks that the java, scala, and javascript apps are all building successfully

test

• Run all the test in the java apps , and in the javascript app
• Important note: - As it is right now these tests will fail, as unit test have not been updated. continue-on-error is set to true on this job though, so workflow should still complete regardless. Once the test are updated they will be need to set continue-on-error to false

smoke-test

• To implement
• This will eventually start JeMPI, and do some basic tests use a chrome daemon.

OnMerge

This workflow runs one a branch has been merged to dev or main. The jobs it executes are highlighted below

Below is a short description of each job

preprare

• Similar to prepare of OnPullRequst above

build-save-deploy-images

• As the name suggests, this build the docker images, then save then (as an artifacte), and the push to you local docker repository (if configured)
• A notte on saving
  • Once branch is merged it create a archive with docker images. This archive can be downloaded, and loaded with docker load --input <archive>
  • This name format of the archive is docker-images-[branch]-[hash]
  • To download them click on the workflow

• Lastly these archive are only rentained for 48hrs before being deleted
• A note of deployment
  • You will need to configure the following secret value (Setting > Setting and variables > Actions > Secrets (tab) > Repository secrets) for this to work
    • DOCKER_LOCAL_DEV_USER_NAME
    • DOCKER_LOCAL_PASSOWRD
  • And the following variable (Setting > Setting and variables > Actions > Variables (tab) > Repository variables)
    • DOCKER_LOCAL_HOST_NAME
  • If not configure this step will be skipped

Deploy Images to Docker Hub

This workflow executed the same workflow as OnMerge above, with a few differences.

1. It requires you to have a tag that will be used for your docker image
2. It is manually trigger, of which you have to specify the tag you want to use

4. It only works on the main branch
5. It pushes images to docker hub

In addition to this you will need to configure the following secret value (Setting > Setting and variables > Actions > Secrets (tab) > Repository secrets) for this to work
- DOCKER_HUB_USER_NAME
- DOCKER_HUB_PASSOWRD

How to test / configure

• The best way to test this is to trigger the aforementioned workflow, by creating pull request and merging. This should trigger the appropate workflows

Other notes:

• This will only properly work once on master, and secrets and variables are configure correctly

[walisc]

cc @rcrichton

[rcrichton]

I think triggering releases when tags are created is pretty standard, that's what we do in a lot of other repos. It also then ties into github releases. This PR will probably take a while to make it to main, are you able to run the script manually for now to generate beta release images on docker hub?

…

On Thu, Feb 15, 2024 at 9:03 AM Chido ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In .github/workflows/deploy-images-dockerhub.yml <#179 (comment)>: > @@ -0,0 +1,54 @@ +name: Deploy Images to DockerHub + +on: Hi @rcrichton <https://github.com/rcrichton> . Yes, I did think of this, but my worry was that it would trigger for any random tag/release creation (which might not be what we want). Throught it might be better to be more specific in regards to this. p.s you can also configure a local docker repo, of which when configure images are pushed to this on merge automatically (with the short commit hash as the tag). It's only when pushing to docker hub when someone will need to be more specific. — Reply to this email directly, view it on GitHub <#179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACBCUA2DHCGYNUAAJFYEXLYTWXKJAVCNFSM6AAAAABDJVBXR2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQOBRHE2DCMJXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- <http://www.jembi.org> *Ryan Crichton * *Head of Engineering * ***@***.*** Skype: ryan.graham.crichton Jembi Health Systems | Durban <https://www.jembi.org/contact>www.jembi.org *This e-mail contains proprietary and confidential information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail and then deleting same. If you are not the intended recipient you must not use, disclose, distribute, copy, print or rely on this e-mail. Jembi Health Systems NPO, its subsidiaries and associated companies is not liable for the security of information sent by e-mail and accepts no liability of whatsoever nature for any loss, damage or expense resulting, directly or indirectly, from the access of this e-mail or any attachments hereto.*

[walisc]

I think triggering releases when tags are created is pretty standard, that's what we do in a lot of other repos. It also then ties into github releases. This PR will probably take a while to make it to main, are you able to run the script manually for now to generate beta release images on docker hub?
…
On Thu, Feb 15, 2024 at 9:03 AM Chido @.> wrote: @.* commented on this pull request. ------------------------------ In .github/workflows/deploy-images-dockerhub.yml <#179 (comment)>: > @@ -0,0 +1,54 @@ +name: Deploy Images to DockerHub + +on: Hi @rcrichton https://github.com/rcrichton . Yes, I did think of this, but my worry was that it would trigger for any random tag/release creation (which might not be what we want). Throught it might be better to be more specific in regards to this. p.s you can also configure a local docker repo, of which when configure images are pushed to this on merge automatically (with the short commit hash as the tag). It's only when pushing to docker hub when someone will need to be more specific. — Reply to this email directly, view it on GitHub <#179 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACBCUA2DHCGYNUAAJFYEXLYTWXKJAVCNFSM6AAAAABDJVBXR2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQOBRHE2DCMJXGE . You are receiving this because you were mentioned.Message ID: @.**>
-- http://www.jembi.org Ryan Crichton * Head of Engineering * @. Skype: ryan.graham.crichton Jembi Health Systems | Durban https://www.jembi.org/contactwww.jembi.org This e-mail contains proprietary and confidential information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail and then deleting same. If you are not the intended recipient you must not use, disclose, distribute, copy, print or rely on this e-mail. Jembi Health Systems NPO, its subsidiaries and associated companies is not liable for the security of information sent by e-mail and accepts no liability of whatsoever nature for any loss, damage or expense resulting, directly or indirectly, from the access of this e-mail or any attachments hereto.

Hi @rcrichton ..okay, will make it push to docker hub on release.
Regarding being added to main. This shouldnt take too long, as we are meant to sync dev and main as part of the beta release. i.e once this is in, will be ported to main immediately (with all the other dev changes).

[walisc]

Hi @rcrichton . Have updated this to push to dockerhub on release 👍🏽

I think triggering releases when tags are created is pretty standard, that's what we do in a lot of other repos. It also then ties into github releases. This PR will probably take a while to make it to main, are you able to run the script manually for now to generate beta release images on docker hub?
…
On Thu, Feb 15, 2024 at 9:03 AM Chido @.> wrote: _@**._* commented on this pull request. ------------------------------ In .github/workflows/deploy-images-dockerhub.yml <#179 (comment)>: > @@ -0,0 +1,54 @@ +name: Deploy Images to DockerHub + +on: Hi @rcrichton https://github.com/rcrichton . Yes, I did think of this, but my worry was that it would trigger for any random tag/release creation (which might not be what we want). Throught it might be better to be more specific in regards to this. p.s you can also configure a local docker repo, of which when configure images are pushed to this on merge automatically (with the short commit hash as the tag). It's only when pushing to docker hub when someone will need to be more specific. — Reply to this email directly, view it on GitHub <#179 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACBCUA2DHCGYNUAAJFYEXLYTWXKJAVCNFSM6AAAAABDJVBXR2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQOBRHE2DCMJXGE . You are receiving this because you were mentioned.Message ID: _@_.>
-- http://www.jembi.org Ryan Crichton * Head of Engineering * @. Skype: ryan.graham.crichton Jembi Health Systems | Durban https://www.jembi.org/contactwww.jembi.org This e-mail contains proprietary and confidential information some or all of which may be legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail and then deleting same. If you are not the intended recipient you must not use, disclose, distribute, copy, print or rely on this e-mail. Jembi Health Systems NPO, its subsidiaries and associated companies is not liable for the security of information sent by e-mail and accepts no liability of whatsoever nature for any loss, damage or expense resulting, directly or indirectly, from the access of this e-mail or any attachments hereto.

Hi @rcrichton ..okay, will make it push to docker hub on release. Regarding being added to main. This shouldnt take too long, as we are meant to sync dev and main as part of the beta release. i.e once this is in, will be ported to main immediately (with all the other dev changes).

Hi @rcrichton . Have updated this to push to dockerhub on release 👍🏽

[rcrichton]

Looks good in general, just a few more comments.

[rcrichton]

It looks like we will use the default app images for docker hub. I don't think these will be suitable as there are already images names that we should reuse - https://hub.docker.com/search?q=jempi

[walisc]

Hmm...okay, noted. Will update as appropiate 👍🏽

[rcrichton]

Do we really need to store these artifacts? Why not rather push to dockerhub with a commit hash as the tag? Then they are always available.

[walisc]

It's mainly there for debugging purpose (hence the rentention only 2 days). Regarding pushing to docker hub, I think the plan was only to push images for main/releases, least dockerhub becomes too convoluted (and could also end up includes bugy images). @w-hayes thoughts?

[rcrichton]

I just think we are going to pay for storing these. I think pushing them to a tag or doing not saving at all makes the most sense.

[walisc]

Okay, will make it manual workflow (in case someone wants to use it to debug), and remove it from the event driven workflows. If this fine?

[rcrichton]

Cool, that sounds fine.

[walisc]

Hi @rcrichton ...have update this. If there are any other changes, do let me know, else would be great to merge it in.

[rcrichton]

Ok nice, looks good to me.

[rcrichton]

I'm happy with this, I think we can merge unless others need to review?