English documentation (#9)

* refacto documentation * rename make command from services-up to vio-edge-up * translate edge orchestrator documentation to english
octo-technology · Dec 27, 2022 · d8b3f8c · d8b3f8c
1 parent dffbbbb
commit d8b3f8c
Show file tree

Hide file tree

Showing 16 changed files with 300 additions and 301 deletions.
diff --git a/Makefile b/Makefile
@@ -35,14 +35,14 @@ deploy-hub_monitoring-azure:
 	ssh-add deployment/grafvio_id_rsa # chmod 400 deployment/grafvio_id_rsa
 	ansible-playbook deployment/ansible/update_grafana_dashboard.yml  -i deployment/ansible/inventory --extra-vars ansible_port=22000
 
-.PHONY: services-up ## 🐳 Start all services (mongodb, model_serving, supervisor, ui)
-services-up:
+.PHONY: vio-edge-up ## 🐳 Start all services (mongodb, model_serving, supervisor, ui)
+vio-edge-up:
 	docker-compose up -d --build
 
-.PHONY: services-up-raspberrypi ## 🐳 Start all services on RaspberryPI (mongodb, model_serving, supervisor, ui)
-services-up-raspberrypi:
+.PHONY: vio-edge-up-raspberrypi ## 🐳 Start all services on RaspberryPI (mongodb, model_serving, supervisor, ui)
+vio-edge-up-raspberrypi:
 	docker-compose -f docker-compose.raspberrypi.yml up -d
 
-.PHONY: services-down ## ❌ Stop all services (model_serving, supervisor, ui)
-services-down:
+.PHONY: vio-edge-down ## ❌ Stop all services (model_serving, supervisor, ui)
+vio-edge-down:
 	docker-compose down
diff --git a/README.md b/README.md
@@ -6,17 +6,25 @@
 
 Visual Inspection Orchestrator is a modular framework made to ease the deployment of VI usecases.
 
-Usecase example: Quality check of a product manufactured on an assembly line.
+*Usecase example: Quality check of a product manufactured on an assembly line.*
 
 VIO full documentation can be found [here](https://octo-technology.github.io/VIO/)
 
-## Features
 
-- [The edge orchestrator](docs/supervisor.md) 
+The VIO modules are split between:
+
+** Edge modules **: The VIO edge modules are deployed close to the object to inspect
+
+- [The edge orchestrator](docs/edge_orchestrator.md)
 - [The edge interface](docs/edge_interface.md)
-- [The edge model serving](docs/model_serving.md)
-- [The hub monitoring](docs/monitoring.md)
-- [The deployment tools](docs/deployment.md)
+- [The edge model serving](docs/edge_model_serving.md)
+- [The edge deployment playbook](docs/edge_deployment.md)
+
+** Hub modules **: The VIO hub modules are deployed in the cloud to collect data and orchestrate the edge fleet
+
+- [The hub monitoring](docs/hub_monitoring.md)
+- [The hub deployment playbook](docs/hub_deployment.md)
+
 
 ## Install the framework
 
@@ -30,25 +38,31 @@ Prerequisites:
 
 To launch the stack you can use the [Makefile](../Makefile) on the root of the repository which define the different target based on the [docker-compose.yml](../docker-compose.yml):
 
-- run all services (supervisor, model-serving, Mongo DB, UI) : `make services-up`
+- run all edge services (orchestrator, model-serving, interface, db) with local hub monitoring (grafana): `make vio-edge-up`
 
-- stop and delete all running services : `make services-down`
+- stop and delete all running services : `make vio-edge-down`
 
 To check all services are up and running you can run the command `docker ps`, you should see something like below:
 
  ![stack-up-with-docker](docs/images/stack-up-with-docker.png)
 
 Once all services are up and running you can access:
 
-- the swagger of the core API (OrchestratoAPI): [http://localhost:8000/docs](http://localhost:8000/docs)
-- the swagger of the model serving: [http://localhost:8501/docs](http://localhost:8501/docs)
-- the monitoring grafana: [http://localhost:4000/login](http://localhost:4000/login)
+- the swagger of the edge orchestrator API (OrchestratoAPI): [http://localhost:8000/docs](http://localhost:8000/docs)
+- the swagger of the edge model serving: [http://localhost:8501/docs](http://localhost:8501/docs)
+- the hub monitoring: [http://localhost:4000/login](http://localhost:4000/login)
 - the edge interface: [http://localhost:8080](http://localhost:8080)
 
 From the edge interface you can load a configuration and run the trigger button that will trigger the Core API and launch the following actions:
 
  ![vio-architecture-stack](docs/images/supervisor-actions.png)
 
+# Releases
+
+Build Type                    | Status                                                                                                                                                                           | Artifacts
+----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------
+**Docker images** | [![Status](https://github.com/octo-technology/VIO/actions/workflows/publication_vio_images.yml/badge.svg)](https://github.com/octo-technology/VIO/actions/workflows/publication_vio_images.yml/badge.svg) | [Github registry](https://github.com/orgs/octo-technology/packages)
+
 ## License
 
 VIO is licensed under [Apache 2.0 License](docs/LICENSE.md)

diff --git a/docs/CICD.md b/docs/CICD.md
@@ -1,61 +1,34 @@
-# La CICD
+# The CICD
 
-Nous utilisons les workflows de Github Actions pour l'intégration continue et pour le déploiement continu.
+We use Github Actions workflows for continuous integration and continuous deployment.
 
-Il existe 5 workflows :
+## The continuous integration workflows
 
-2 workflows de CI:
+- [ci_edge_interface.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/ci_edge_interface.yml): the CI of the edge_interface application is decomposed into 2 jobs
+
+        job lint_and_test_on_edge_interface: static code analysis of JavaScript code (no tests at the moment)
+        job build_and_push_images: building the Docker image of the application without publishing to a registry
 
-- [ci_edge_interface.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/ci_edge_interface.yml) :
-  intégration continue de l'application
-  edge_interface décomposée en 2 jobs
-    - job `lint_and_test_on_edge_interface` : analyse statique du code JavaScript (pas de tests pour le moment)
-    - job `build_and_push_images` : construction de l'image Docker de l'application sans publication dans une registry
-- [ci_edge_orchestrator.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/ci_edge_orchestrator.yml) :
-  intégration continue de l'application
-  edge_orchestrator décomposée en 2 jobs
-    - job `lint_and_test_on_edge_orchestrator` : analyse statique du code Python avec Flake8 suivie de l'exécution des
-      tests automatisés (unitaires, intégration et fonctionnels) avec le stockage des rapports de tests dans Github
-    - job `build_and_push_images` : construction de l'image Docker de l'application sans publication dans une registry
+- [ci_edge_orchestrator.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/ci_edge_orchestrator.yml): the CI of the edge_orchestrator application is decomposed into 2 jobs
+
+        job lint_and_test_on_edge_orchestrator: static code analysis with Flake8 followed by automated tests (unit, integration, and functional) with storing test reports in Github
+        job build_and_push_images: building the Docker image of the application without publishing to a registry
 
-Les deux workflows de CI (edge_[interface|orchestrator]_ci.yml) sont déclenchés sous l'une des conditions suivantes :
+The CI workflows (edge_[interface|orchestrator]_ci.yml) are triggered under one of the following conditions:
+- if a merge request with differences is opened on Github
+- if a commit on the master branch is pushed to Github
 
-- si une merge request comportant des différences est ouverte sur Github
-- si un commit sur la branche master est pushé sur Github
+## The release workflows
 
-3 worklows de release:
+- [publication_vio_images.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_vio_images.yml): publication of Docker edge_serving images by manual trigger job
+
+        build_and_push_images: building Docker images with publishing images to the Github registry  
 
-- [publication_vio_images.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_vio_images.yml) :
-  publication des images Docker edge_serving par déclenchement manuel
-    - job `build_and_push_images` : construction des images Docker avec publication des images dans la registry Github
-- [publication_vio_images_raspberry.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_vio_images_raspberry.yml) :
-  publication des images Docker edge_serving par déclenchement manuel
-    - job `build_and_push_images` : construction des images Docker spécifique au hardware Raspberry avec publication des images dans la registry Github
-- [publication_pages_gh-pages_branch.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_pages_gh-pages_branch.yml) :
-  génération et déploiment de la documentation
+- [publication_vio_images_raspberry.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_vio_images_raspberry.yml): publication of Docker edge_serving images by manual trigger job 
 
-Les 3 workflows de release sont déclenchés sous l'une des conditions suivantes :
+        build_and_push_images: building Docker images specific to Raspberry hardware with publishing images to the Github registry
 
-- si une release est crée depuis Github
+- [publication_pages_gh-pages_branch.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_pages_gh-pages_branch.yml): generation and deployment of documentation
 
-
-////////////////////////////// WIP //////////////////////////////
-
-Pour déployer une nouvelle version sur RaspberryPI, il faut d'abord créer des images Docker spécifiques pour le device
-en question.
-Afin de créer ces images, il suffit d'ajouter un tag Git, en suivant la
-convention [SemVer](https://semver.org/lang/fr/). Par exemple:
-
-```
-git tag rpi-1.2.1
-git push --tags
-```
-
-Une fois le tag poussé, cela déclenche une pipeline Gitlab CI qui va construire les images Docker pour RaspberryPI.
-Celles-ci seront stockées dans la registry Gitlab, et elles-mêmes taguées avec le même tag `rpi-1.2.1`.
-
-Enfin, il faut préciser à Azure IoT Hub qu'on souhaite déployer ces nouvelles versions sur les dispositifs Edge.
-Pour cela, il suffit mettre à jour les variables dans le fichier `deployment/ansible/setup_iot_hub_azure.yml` et
-relancer le playbook Ansible.
-
-////////////////////////////// WIP //////////////////////////////
+The release workflows are triggered under one of the following conditions:
+- if a release is created from Github
diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md
@@ -1,12 +1,12 @@
 # Contributing
 
-**Contribution rules**
+## Contribution rules
 - The code must be exhaustively tested.
 - Python test package: `pytest` (with possibility to use unittest mocks)
 - Code style: PEP8
 - Programming language for code and comments: English
 
-**Coding conventions**
+## Coding conventions
 - 120 character max per line
 - Use python 3.6 `fstring` instead of `format()` or `%s`
 - Directories, filenames, function and method names in `snake_case`
@@ -20,7 +20,7 @@ corresponding class).
  existing dead code.
 - Use [pathlib](https://docs.python.org/3/library/pathlib.html#module-pathlib) instead of native Python [os.path](https://docs.python.org/3/library/os.path.html)
 
-**Exception conventions**
+## Exception conventions
 - Create a custom exception in the module `exception.py` as follow:
 ```python
 class MyCustomException(Exception):
@@ -40,7 +40,7 @@ except MyCustomException as e:
     return True
 ```
 
-**Logging conventions**
+## Logging conventions
 - The logger should always be used at the class level and not the module level.
 - The logger should always be created through the _getChild_ method as a class attribute.
 - The created child logger should always be used in the class.
@@ -60,7 +60,7 @@ class MyClassWithLogging:
         self.logger.info('Doing something!')
 ```
 
-**Test conventions**
+## Test conventions
 - The same file hierarchy should be used between a project and the associated tests.
 ```
 my_project
@@ -123,7 +123,7 @@ class TestMyFunction:
 - Don't mistake a stub for a mock. A mock is used to assert that it has been called (see above example). A stub 
 is used to simulate the returned value.
 
-**Versioning strategy**
+## Versioning strategy
 - Git tutorial:
     - [Basic git tutorial](http://rogerdudler.github.io/git-guide/)
     - [Learn git branching](https://learngitbranching.js.org/)

diff --git a/docs/DOCUMENTATION.md b/docs/DOCUMENTATION.md
@@ -0,0 +1,23 @@
+# Documentation
+
+To update the documentation, feel free to modify / add markdown file in the `/docs` folder of the repository
+
+## Preview Locally
+
+To build locally your github pages site
+```shell
+$ mkdocs build
+```
+To test locally your github pages site
+```shell
+$ mkdocs serve
+```
+
+## Publish on github pages
+
+Simply commit your modification on your branch, issue a PR and the workflow [publication_pages_gh-pages_branch.yml](https://github.com/octo-technology/VIO/tree/main/.github/workflows/publication_pages_gh-pages_branch.yml) will be triggered automatically.
+
+Note: (Wrong behaviour) to manually push your modification directly to github pages you can execute the command:
+```shell
+$ mkdocs gh-deploy
+```
diff --git a/docs/documentation.md b/docs/documentation.md
diff --git a/docs/deployment.md → docs/edge_deployment.md b/docs/deployment.md → docs/edge_deployment.md
@@ -1,110 +1,11 @@
-# Deployment
+# Edge Deployment
 
-## Cloud - Infrastructure deployment on Azure
-
-This section allows you to create all the Azure infrastructure for VIO:
-- Storage resources (Storage Account + PostgreSQL)
-- The IoT Hub
-- An Azure function (`telemetry_saver`) to save Device-to-Cloud telemetry data in PostgreSQL
-- An Event Grid Topic to connect IoT Hub with the `telemetry_saver` Azure function
-
-### Prerequisites
-
-Before getting started, you need to install Ansible and its dependencies for Azure and PostgreSQL.
-
-```shell
-$ cd ./deployment/
-$ conda create -n ansible python=3
-$ conda activate ansible
-$ pip install -r requirements.txt
-$ ansible-galaxy collection install azure.azcollection
-$ ansible-galaxy collection install community.grafana
-```
-
-
-You'll also need :
-- The [Azure CLI](https://docs.microsoft.com/fr-fr/cli/azure/install-azure-cli)
-- The [Azure CLI IoT extension](https://github.com/Azure/azure-iot-cli-extension) extension
-- The [Azure Functions Core Tools](https://github.com/Azure/azure-functions-core-tools)
-
-On MacOS, these can be installed as follows:
-
-```shell
-$ brew update
-$ brew install azure-cli
-$ az extension add --name azure-iot
-$ brew tap azure/functions
-$ brew install azure-functions-core-tools@3
-```
-
-Once you have installed `azure-cli`, you can login to Azure using your Accenture account:
-
-```shell
-$ az login
-```
-
-Make sure you are using the Azure subscription `IX-Visual-Inspection-MDI`. You can check that with:
-```shell
-$ az account list --output table
-```
-
-If `IX-Visual-Inspection-MDI` is not the default subscription, you can switch to it with the following command:
-```shell
-$ az account set --subscription "IX-Visual-Inspection-MDI"
-```
-
-### Define the mandatory environment variables
-
-In order to create and configure all the Azure infrastructure, we need to define some environment variables: 
-
-```shell
-$ export REGISTRY_USERNAME=<your_gitlab_username>
-$ export REGISTRY_PASSWORD=<your_gitlab_container_registry_token>
-$ export POSTGRES_USERNAME=<postgres_admin_username>
-$ export POSTGRES_PASSWORD=<postgres_admin_password>
-$ export AZURE_STORAGE_CONNECTION_STRING=<azure_storage_connection_string>
-```
-
-#### Registry Username 
-firstname.lastname (i.e nicolas.dupont)
-
-#### Registry Password 
-You can find it on gitlab, click on your profil picture (top right corner) --> preferences. 
-On the ```Access Tokens``` category, you can generate a token. I suggest no expiration date, and you select all the scopes.
-Keep this token safe, once it's generated you cannot retrieve it on gitlab anymore.
-
-#### Postgres Username and Password.
-- To Get the Postgres username, go the Azure Portal, our subscription ```IX-Visual-Inspection-MDI``` --> our resource group ```vio-rg-dev``` --> the ``` vio-function-app-dev ``` function app.
-On the left side bar menu, click on ```Configuration``` and unhide the POSTGRES_USER field. You only need what's before the @. Here it's ```vioadmin```
-- To Ge the Postgres password, it's on the same page but unhide the POSTGRES_PASSWORD field.
-
-![postgres_username_password.png](images/postgres_username_password.png)
-
-
-### Create Azure Infrastructure
-
-The following command creates all the Azure IoT infrastructure for VIO.
-
-```shell
-$ ansible-playbook ansible/create_azure_cloud_infrastructure.yml -e 'ansible_python_interpreter=<path_to_your_conda_env_python_interpreter>'
-```
-
-## Cloud - Deploy Grafana dashboard and data-sources
-
-To deploy Grafana [dashboard](../monitoring/dashboards) and [data-sources](../monitoring/provisioning), run the following playbook :
-```shell
-$ ansible-playbook -i ansible/inventory/production.ini ansible/update_grafana_dashboard.yml --ask-pass
-```
-
-This will copy the files on the Grafana resource deployed in Azure and relaunch the grafana service to take into account the brand uploaded files. 
-
-
-## Edge - Raspberry Setup (Raspbian installation)
+## Raspberry Setup (Raspbian installation)
 The Raspberry can be set up thanks to this [Makefile](Makefile).
 
 First thing first, insert the SD card in your computer to mount it. **Before typing any command**, check that the SD card is effectively mounted on `/dev/disk2`, by typing:
 ```shell
-$ diskutil list 
+$ diskutil list
 ```
 
 Checklist before continuing:
@@ -234,9 +135,9 @@ $ diskutil unmountDisk $(MOUNTING_DIR)
 $ diskutil eject $(MOUNTING_DIR)
 ```
 
-## Edge - Install and configure the IoT Edge Agent on RaspberryPI
+## Install and configure the IoT Edge Agent on RaspberryPI
 
-In order to be managed by Azure IoT Hub, each edge device must install an IoT Edge Agent and _connect_ to the Hub. 
+In order to be managed by Azure IoT Hub, each edge device must install an IoT Edge Agent and _connect_ to the Hub.
 
 We use Ansible to automate the setup of the IoT Edge Agent.
 

diff --git a/docs/edge_interface.md b/docs/edge_interface.md
@@ -1,4 +1,4 @@
-# The Edge interface
+# Edge Interface
 
 1. Select a configuration
  ![edge_interface_config_screen](images/edge_interface_config_screen.png)