Skip to content
This repository has been archived by the owner on Feb 12, 2024. It is now read-only.

Update doc #144

Merged
merged 13 commits into from
Jan 20, 2022
9 changes: 3 additions & 6 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,13 +86,9 @@ cd fadi
Launch the Helm script, this will deploy all the FADI services on the Minikube cluster (and may take some time).

```bash
kubectl config set-context minikube
minikube addons enable ingress
cd helm
# you can edit values.yaml file to customise the stack
./deploy.sh
# specify the fadi namespace to see the different pods
kubectl config set-context minikube --namespace fadi
```

You can check everything is deploying/-ed in the Kubernetes dashboard:
Expand All @@ -117,13 +113,14 @@ kubectl get pods
To access a service in your browser, type for instance:

```
minikube service -n fadi fadi-nifi
kubectl port-forward service/fadi-grafana 8080:80
```
Then, you can access the service by typing in your browser [localhost:8080](http://localhost:8080)

You can list all the addresses by typing:

```
kubectl get ingress -n fadi
kubectl get ingressroute -n fadi
```

To update the FADI stack, re-type:
Expand Down
51 changes: 28 additions & 23 deletions USERGUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ Those components are configured in the following [sample config file](helm/value

The following instructions assume that you deployed FADI on your workstation inside minikube.

To access services through domain names, open a new terminal and enter this command to give Traefik an external IP address:
```
minikube tunnel
```
Don't forget to update your `hosts` file with Traefik's external IP address. You can find [here](https://phoenixnap.com/kb/how-to-edit-hosts-file-in-windows-mac-or-linux) a user guide for Linux, Mac and Windows.

Unless specified otherwise, all services can be accessed using the username and password pair: `admin` / `password1` , see the [user management documentation](doc/USERMANAGEMENT.md) for detailed information on how to configure user identification and authorization (LDAP, RBAC, ...).

See the [logs management documentation](doc/LOGGING.md) for information on how to configure the management of the various service logs.
Expand All @@ -43,20 +49,20 @@ First, setup the datalake by creating a table in the postgresql database.

To achieve this you need to:

* Head to the adminer interface, if you are using **minikube**, you can use the following command :
```
minikube service -n fadi fadi-adminer
```
* Head to the adminer interface

* Access to the adminer service and to the postgreSQL database using the following credentials:
* if you want to create a **Traefik ingress**, you can follow this [guide](doc/REVERSEPROXY.md#2-configure-the-various-services-to-use-traefik)
* else, you can use a port-forwarding to access the interface: `kubectl port-forward service/fadi-adminer 8081:80` and can access Adminer from your browser at [localhost:8081](http://localhost:8081)

* Access to the Adminer service and to the PostgreSQL database using the following credentials:

* System: `PostgreSQL`
* Server: `fadi-postgresql`
* Username: `admin`
* Password: `password1`
* Database: `postgres`

* In the adminer Browser, launch the Query tool by clicking "SQL command".
* In the Adminer Browser, launch the Query tool by clicking "SQL command".

* Copy/Paste the [table creation script](examples/basic/create_datalake_tables.sql) in the Query Editor.
![Postgres Server](examples/basic/images/adminer_create_table.png)
Expand Down Expand Up @@ -99,10 +105,9 @@ measure_ts,temperature
(...)
```

To start, head to the Nifi web interface, if you are using **minikube**, you can use the following command :
```
minikube service -n fadi fadi-nifi
```
To start, head to the Nifi web interface, type in your browser the `nifi.traefikIngress.host`. E.g. :

[http://nifi.fadi.cetic.be](http://nifi.fadi.cetic.be)

![Nifi web interface](examples/basic/images/nifi_interface.png)

Expand Down Expand Up @@ -162,7 +167,7 @@ So, create the following components :

![Nifi Ingest CSV and store in PostgreSQL](examples/basic/images/nifi_csv_to_sql_2.png)

See also [the nifi template](/examples/basic/basic_example_final_template.xml) that corresponds to this example.
See also [the NiFi template](/examples/basic/basic_example_final_template.xml) that corresponds to this example.
* To reuse the provided template (instead of designing our own template), you can:
* Click `Upload template` in the **Operate** frame, select the template, and upload it.
* From the Nifi menu, drag and drop the **Template** menu.
Expand All @@ -189,10 +194,10 @@ Once the measurements are stored in the database, we will want to display the re

[Grafana](http://grafana.com/) provides a dashboard and alerting interface.

Head to the Grafana interface, if you are using **minikube**, you can use the following command :
```
minikube service -n fadi fadi-grafana
```
Head to the Grafana web interface by typing in your browser the `grafana.traefikIngress.host`. E.g. :

[http://grafana.fadi.cetic.be](http://grafana.fadi.cetic.be)

(the default credentials are `admin`/`password1`)

![Grafana web interface](examples/basic/images/grafana_interface.png)
Expand Down Expand Up @@ -244,10 +249,10 @@ For more information on how to use Grafana, see the [official Grafana user guide

[Apache Superset](https://superset.incubator.apache.org) provides some interesting features to explore your data and build basic dashboards.

Head to the Superset interface, if you are using **minikube**, you can use the following command :
```
minikube service -n fadi fadi-superset
```
Head to the Superset web interface by typing in your browser the `superset.traefikIngress.host`. E.g. :

[http://superset.fadi.cetic.be](http://superset.fadi.cetic.be)

(the default credentials are `admin`/`password1`):

First we will define the datasource:
Expand Down Expand Up @@ -312,10 +317,10 @@ For more information on how to use Superset, see the [official Superset user gui

In this simple use case, we will try to access the data that is stored in the data lake.

Head to the Jupyter notebook interface, if you are using **minikube**, you can use the following command :
```
minikube service -n fadi proxy-public
```
Head to the Jupyter notebook interface by typing in your browser the `jupyter.traefikIngress.host`. E.g. :

[http://jupyter.fadi.cetic.be](http://jupyter.fadi.cetic.be)

Then, you can login using the default credentials `admin`/`password1`.

A Jupyter dashboard is shown.
Expand Down
18 changes: 9 additions & 9 deletions doc/USERMANAGEMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,15 +123,15 @@ auth:
userIdentityAttribute: cn
```

Then we make sure to pre-set the `nodePort`, let's say we want the node port to be 34567, our service configuration should look like this :
Then we make sure to pre-set the `traefikIngress`, let's say we want the domain name `nifi.test.local`, our `traefikIngress` configuration should look like this :

```yaml
service:
type: NodePort
nodePort: 34567
traefikIngress:
enabled: true
host: nifi.test.local
```

And then we set the properties as follows, the `nifi.properties.webProxyHost` variable should have the exact url with the exact port that we are going to use to access NIFI later, if our dns is nifi.example.cetic.be and/or the ip address 10.10.10.10, our configuration should look like this:
And then we set the properties as follows, the `nifi.properties.webProxyHost` variable should have the exact url that we are going to use to access NIFI later, if our dns is nifi.example.cetic.be, our configuration should look like this:


```yaml
Expand All @@ -140,7 +140,7 @@ And then we set the properties as follows, the `nifi.properties.webProxyHost` va
isNode: false
httpPort: null
httpsPort: 9443
webProxyHost: nifi.example.cetic.be:34567, 10.10.10.10:34567
webProxyHost: nifi.example.cetic.be
clusterPort: 6007
clusterSecure: true
```
Expand Down Expand Up @@ -209,13 +209,13 @@ We can no longer see the name of the group, and we now have a more restrictive c

> "phpLDAPadmin is a web app for administering Lightweight Directory Access Protocol (LDAP) servers."

In order to use [phpLDAPadmin](http://phpldapadmin.sourceforge.net/wiki/index.php/Main_Page), pass the configuration for the LDAP server through the environmental variable `_PHPLDAPADMIN_LDAP_HOSTS_`. To connect this service with the OpenLDAP server, pass **the name of the service** (`fadi-openldap`). To connect to the web application, run the following command:
In order to use [phpLDAPadmin](http://phpldapadmin.sourceforge.net/wiki/index.php/Main_Page), pass the configuration for the LDAP server through the environmental variable `_PHPLDAPADMIN_LDAP_HOSTS_`. To connect this service with the OpenLDAP server, pass **the name of the service** (`fadi-openldap`). To connect to the web application, run the following command in a separate shell:

```bash
minikube service fadi-phpldapadmin -n fadi
kubectl port-forward service/fadi-phpldapadmin 8082:80
```

The main page for phpLDAPadmin will open in your default browser where you can connect to your LDAP server and manage it.
Now, type [localhost:8082](http://localhost:8082) in your browser to access the main page of phpLDAPadmin where you can connect to your LDAP server and manage it.

<img src="images/installation/phpldapadmin.gif" alt="phpdapadmin" />

Expand Down
1 change: 1 addition & 0 deletions helm/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ helm repo add cetic https://cetic.github.io/helm-charts/ --force-update
helm repo update

# install/upgrade FADI
helm dep up
helm upgrade --install ${NAMESPACE} cetic/fadi -f ./values.yaml --namespace ${NAMESPACE}

printf "\n\nInstallation successful!\n"