Feature/documentation for #43 #78 (#85)

* add files for the doc

* add doc traefik

* add TSimulus & reverse proxy doc #78, #43

* update doc

* fix typo

INSTALL.md

@@ -207,4 +207,10 @@ See [.gitlab-ci.sample.yml](.gitlab-ci.sample.yml) for an example CI setup with
 
 See the [user management documentation](doc/USERMANAGEMENT.md) for information on how to configure user identification and authorization (LDAP, RBAC, ...).
 
-See the [logs management documentation](doc/LOGGING.md) for information on how to configure logging.
+See the [logs management documentation](doc/LOGGING.md) for information on how to configure logging.
+
+See the [reverse proxy documentation](doc/REVERSEPROXY.md) for information on how to configure the Traefik reverse proxy with FADI.
+
+See the [security documentation](doc/SECURITY.md) for information on how to configure SSL.
+
+Seel the [TSimulus documentation](doc/TSIMULUS.md) for information on how to simulate sensors and generate realistic data with [TSimulus](https://github.com/cetic/TSimulus). 

README.md

@@ -20,7 +20,7 @@ Anywhere you can run [Kubernetes](https://kubernetes.io/), you should be able to
 1. [Install the framework on your workstation](INSTALL.md)
 2. Try [a simple use case](USERGUIDE.md) (or more [advanced usage examples](examples/README.md))
 
-You can find a more detailed explanation of FADI in the [presentation slideshow](https://fadi.presentations.cetic.be)
+You can find a more detailed explanation of FADI in the [presentation slideshow](https://fadi.presentations.cetic.be) and in the [documentation section](doc/README.md)
 
 ## FADI Helm Chart
 

doc/LOGGING.md

@@ -2,22 +2,22 @@ Logs management
 ==========
 
 <p align="left";>
-	<a href="https://www.elastic.co" alt="pgAdmin">
-	    <img src="/doc/images/logos/elk.png" align="center" alt"ELK logo" width="200px" />
+	<a href="https://www.elastic.co" alt="elk">
+	    <img src="/doc/images/logos/elk.png" align="center" alt="ELK logo" width="200px" />
     </a>
 </p>
 
 **[Elastic Stack](https://www.elastic.co)** is a group of open source products from Elastic designed to help users take data from any type of source and in any format and search, analyze, and visualize that data in real time. The product group is composed of: **Beats**, **Logstash**, **Elasticsearch** and **Kibana**.
 Despite each one of these four technologies being a separate project, they have been built to work together:
 
 * the process start with **[Beats](https://www.elastic.co/products/beats)**, it ships the logs from all services to 
-* **[Logstash](https://www.elastic.co/products/logstash)**, which parse, filter or/and transform the logs before storing them in 
+* **[Logstash](https://www.elastic.co/products/logstash)**, which parses, filters or/and transforms the logs before storing them in 
 * **[Elasticsearch](https://www.elastic.co/products/elasticsearch)** for indexing, which is connected to 
 * **[Kibana](https://www.elastic.co/products/kibana)** that provides visualisation of the various services' logs in a central web application interface
 
 ![Elastic-stack](/doc/images/installation/elastic_stack.png)
 
-To access the **Kibana** web interface, you have to. pass by nginx-ldapauth-proxy, you can use this command:
+To access the **Kibana** web interface, you have to go through the nginx-ldapauth-proxy, you can use this command:
 
 ```
 minikube service fadi-nginx-ldapauth-proxy
@@ -29,7 +29,6 @@ To create the index pattern and monitor the logs, follow these simple steps:
 1. In Kibana, open **Management** and then click **Index Patterns**.
 2. If this is your first index pattern, the **Create index pattern** page opens automatically. Otherwise, click **Create index pattern**. 
 3. Enter `filebeat*` in the Index pattern field.
-
   ![index_pattern](/doc/images/installation/index_pattern.png)
 
 4. Click **Next step**.
@@ -42,4 +41,4 @@ To create the index pattern and monitor the logs, follow these simple steps:
 
   ![Kibana Logs](/doc/images/installation/kibana_logs.png)
 
-For more details you can always visit the [Elastic-stack official documentation](https://www.elastic.co/guide/index.html).
+For more details you can always visit the [Elastic-stack official documentation](https://www.elastic.co/guide/index.html).

doc/README.md

@@ -2,6 +2,9 @@ FADI Documentation
 ==========
 
 * [Services logs management](LOGGING.md) - configure and setup a centralised logging system
-* [Users management](USERMANAGEMENT.md) - configure users, groups and access rights
+* [Users management](USERMANAGEMENT.md) - user identification and authorization (LDAP, RBAC, ...)
+* [Reverse proxy](doc/REVERSEPROXY.md) - Traefik reverse proxy configuration
+* [Security](doc/SECURITY.md) - SSL setup
+* [TSimulus](doc/TSIMULUS.md) - how to simulate sensors and generate realistic data with [TSimulus](https://github.com/cetic/TSimulus)
 
 For tutorials and examples, see the [examples section](examples/README.md)

doc/REVERSEPROXY.md

@@ -0,0 +1,84 @@
+Reverse Proxy
+==========
+
+<p align="left";>
+  <a href="https://traefik.io/" alt="traefik">
+    <img src="/doc/images/logos/traefik-logo.png" width="200px" />
+  </a>
+</p>
+
+* [1. Create the Traefik reverse proxy](#1-create-the-traefik-reverse-proxy)
+* [2. Configure the various services to use Traefik](#2-configure-the-various-services-to-use-traefik)
+
+This page provides information on how to configure FADI with the [Traefik](https://traefik.io/) Reverse Proxy.
+
+> Traefik is an open-source reverse proxy and load balancer for HTTP and TCP-based applications that is easy, dynamic, automatic, fast, full-featured, production proven, provides metrics, and integrates with every major cluster technology... No wonder it's so popular!
+
+Note: other Reverse Proxy than Traefik that could be used with FADI, check the list [here](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/).
+
+## 1. Create the Traefik reverse proxy
+
+To create the Traefik reverse proxy, you will need to add the `stable` Helm repository:
+
+```
+helm repo add stable https://kubernetes-charts.storage.googleapis.com
+```
+
+You can find information about this Helm chart [here](https://github.com/helm/charts/tree/master/stable/traefik) as the Traefik Helm chart is hosted in the stable helm repository.
+
+First, you will need to create a `clusterrole` for Traefik:
+
+```
+kubectl get clusterrole traefik-ingress-controller 2> /dev/null || kubectl create -f ./traefik/rbac-config.yaml
+```
+
+Take a look at the [sample file](/helm/traefik/rbac-config.yaml).
+
+Then, you can install Traefik with Helm: (If you want further information, you can follow this [tutorial](https://docs.traefik.io/v1.3/user-guide/kubernetes/#deploy-trfik-using-helm-chart))
+
+```
+helm upgrade --install traefik stable/traefik -f ./traefik/values.yaml --namespace kube-system --tiller-namespace tiller
+```
+
+The values file can be found [here](/helm/traefik/values.yaml).
+
+```
+loadBalancerIP: "yourLoadBalancerIP"
+ssl:
+  enabled: true
+dashboard:
+  enabled: true
+  domain: <yourdomain>
+  serviceType: NodePort
+  ingress:
+    annotations: {kubernetes.io/ingress.class: traefik}
+    path: /
+```
+
+See the [default values file](https://github.com/helm/charts/blob/master/stable/traefik/values.yaml) from the official repository for more configuration options.
+
+
+## 2. Configure the various services to use Traefik
+
+You will need to update ingress definitions for each service you want to expose, behind your domain name.
+
+See https://docs.traefik.io/providers/kubernetes-ingress/ for the documentation.
+
+Update the FADI `values.yaml` file. You can set all the service types to `ClusterIP` as all services are now exposed through an Ingress. 
+
+For instance, for Grafana:
+```
+grafana:
+  enabled: true
+  service:
+    type: ClusterIP
+  ingress:
+    enabled: true
+    annotations: {kubernetes.io/ingress.class: traefik}
+    path: /
+    hosts: [grafana.yourdomain]
+```
+
+You should now be able to access Grafana through the domain name you have chosen: `http(s)://grafana.yourdomain.com`
+
+Next you will also want to configure SSL access to your services. For that, have a look at the [security documentation](/doc/SECURITY.md). 

doc/SECURITY.md

@@ -0,0 +1,7 @@
+Security
+==========
+
+TODO
+
+* [cert-manager](https://cert-manager.io/docs/installation/kubernetes/).
+* ...

doc/TSIMULUS.md

@@ -0,0 +1,68 @@
+TSimulus: Sensors simulation for FADI
+==========
+
+<p align="left";>
+  <a href="https://tsimulus.readthedocs.io/en/latest" alt="tsimulus">
+    <img src="/doc/images/logos/TSimulus-logo.png" width="200px" />
+  </a>
+</p>
+
+* [0. Context](#0-context)
+* [1. Setup TSimulus](#1-setup-tsimulus)
+* [2. How to use TSimulus?](#2-how-to-use-tsimulus)
+
+## 0. Context
+
+The TSimulus library is used to simulate various sensors from industrial partners in the context of [many research projects](https://www.cetic.be/FADI). It allows to generate a sufficient amount of data to test the entire FADI platform. It is useful when there was a lack of real time data streams. In that context, TSimulus enables to already work on the analysis part without having the real data from the partners.
+
+This section explains how to setup TSimulus for FADI and gives the essential links to the documentation on how to use it.
+
+## 1. Setup TSimulus
+
+The **TSimulus as a Service** project aims at building a REST API in front of the [TSimulus](https://github.com/cetic/TSimulus) framework, and a set of configurable websocket routes to consume the TSimulus stream.
+
+The project is structured as a [sbt multiproject](https://www.scala-sbt.org/1.x/docs/Multi-Project.html), each part is runnable as standalone and the top level project orchestrates a complete deployment and coordination of each parts.
+
+For more information on the implementation, take a look at the documentation of [TSimulus as a Service](https://github.com/cetic/tsimulus-saas).
+
+To install TSimulus with FADI, you will need to modify a bit the [values.yaml](/helm/values.yaml) file. This will deploy all the TSimulus services (the TSimulus microservice and the [Swagger User Interface](https://swagger.io/tools/swagger-ui/)) on your Kubernetes cluster.
+
+First of all, update your [values.yaml](/helm/values.yaml) by activating the TSimulus services:
+
+```
+tsaas:
+  enabled: true
+  ingress:
+    enabled: true
+    hosts: [api-tsimulus.yourdomain]
+
+swaggerui:
+  enabled: true
+  swaggerui :
+    jsonUrl : https://raw.githubusercontent.com/cetic/tsimulus-saas/master/oas/api-doc/openapi.json
+    server :
+      url: http://api-tsimulus.yourdomain
+      description: "TSIMULUS API"
+  ingress:
+    enabled: true
+    hosts: [swagger-tsimulus.yourdomain]
+```
+
+You can also setup Ingress parts to use a reverse proxy. See the [previous section](doc/REVERSEPROXY.md).
+
+Then, run the [deploy.sh](/helm/deploy.sh) script to take the modifications into account:
+
+```
+cd helm
+./deploy.sh
+```
+
+You should now be able to access the Swagger User Interface (on a minikube setup: `minikube service fadi-swaggerui`):
+
+![](/doc/images/installation/tsaas-swaggerui.png)
+
+## 2. How to use TSimulus?
+
+* See the [TSimulus Documentation](https://tsimulus.readthedocs.io/en/latest/).
+* See the [TSimulus as a Service Documentation](https://github.com/cetic/tsimulus-saas).
+* (TODO) See the use cases.

doc/USERMANAGEMENT.md

@@ -82,12 +82,10 @@ host       database  user  address  auth-method  [auth-options]
 
 For example, to use LDAP authentication for local users, your configuration should look something like this :
 
-
 ```
 local      all  all  ldap  ldapserver=example.com  ldapport=389 [other-ldap-options]
 ```
 
-
 For more information about how to add LDAP authentication to PostgreSQL: [LDAP authentication in PostgreSQL](https://www.postgresql.org/docs/11/auth-ldap.html)
 
 For more information about pg-ldap-sync: [Use LDAP permissions in PostgreSQL](https://github.com/larskanis/pg-ldap-sync)