Adding users at runtime (test build of Shiro/JDBC)

[Zarquan]

The multi-user testing has show that having to restart Zeppelin every time we want to add a new user account will cause problems for a live service.

It is worth spending some time (2days?) investigating ways of adding users dynamically, without restarting the system. A good candidate for this would probably be a SQL/JDBC component for Shiro, storing the account details in a database.

We have discussed this a few times and it has been on the back burner since 2020 (#166), but it hasn't had a strong use case until now. Probably because we were looking at it as a central store for accounts shared between all deployments as a way to solve the 'secret problem'.

This is looking to solve a different use case, the source of the account data would be in a text (JSON) file in the secret store, loaded into a local database at deployment time. At runtime Shiro would connect to the local database to authenticate users. New users would be added to both the upstream text (JSON) file and inserted into the live database. This would enable us to add new user accounts (human and test) without restarting a running system.

First step is to deploy a test build with Shiro connected to a small test database to see how easy it would be to configure.

[Zarquan]

I've tagged this with DR3 milestone because this would solve a specific problem we are likely to encounter once we accept multiple users. Specifically, being able to add a new users to a running live system without interrupting the service for existing users by having to restart Zeppelin.

[Zarquan]

Resources for configuring Shiro to use JDBC for authentication.

• https://gist.github.com/dominicfarr/9637357
• https://shiro.apache.org/static/1.3.2/apidocs/org/apache/shiro/realm/jdbc/JdbcRealm.html
• https://stackoverflow.com/questions/17440525/using-jdbcrealm-to-authenticate-user-with-shiro

[Zarquan]

Copy/paste discussion from Slack

Dave Morris 2:18 PM
12:19
Is the JDBC database also running on the Zeppelin node ?

Stelios Voutsinas 12:32 PM
In the last test deploy no, it runs on a remote node
12:33
I imagine we want a separate VM running the user database right?
12:33
I was thinking the place for this is probably the data project, similar to the Vault service node

Dave Morris 12:36 PM
Nope
12:36
Two reasons for putting the user accounts in database 1) to centralise the admin of the database 2) to enable us to add user accounts dynamically
12:36
doesn't work
12:37
It just creates another centralised thing that itself needs securing and backing up
12:37
2. can be done by having a local database on the Zeppelin node, and injecting user accounts into that
12:39
#665 (comment)

12:41
We might migrate parts of it to a central database in the end, but right now this is experimental work on a per deployment basis.

Stelios Voutsinas 12:46 PM
In that scenario, isn't the backup the JSON file in the secret store, in which case we could still do the same with a centralised database?
12:46
So adding a new user is still, updating the database & updating the JSON file

Dave Morris 12:46 PM
yes, we restore/create the database from the JSON file

[stvoutsin]

I've been working on this in https://github.com/stvoutsin/aglais/commits/feature/shiro-jdbc/
I've managed to get Shiro to authenticate with both a local database in the Zeppelin node, and a remote database accessible via public IP.
Next steps required before this can be PR'd would be to sort out how to pass in credentials to the Zeppelin configuration from the deployment scripts (using a secret function) & automating the creation of the MySQL database and inclusion of the tasks into the deployment scripts

[Zarquan]

Part of the motivation fro this is to be able to add 100+ users accounts to a test deployment at runtime.
We wouldn't want to add them to the main/live account database, so we need a local account database per deployment.
Same issue we had with putting the notebooks directory on a shared drive, we need to be able to make local changes that aren't reflected in the global state.

[Zarquan]

I've managed to get Shiro to authenticate with both a local database in the Zeppelin node, and a remote database accessible via public IP.

Yay, this is all good stuff to know.
Local database is the way to go for now.
Accessing a remote database over public IP address is going to be hard to make secure.

[Zarquan]

Do we need to make the database password a secret (as in a long term thing stored in a vault) ?

Three things need to know the database password

1. The client container running the Ansible scripts
2. The database itself
3. The Zeppelin Shiro configuration

In which case, use pwgen to create a secure random password at deployment time, store it in an Ansible var file on the client for the duration of the deployment, set it in the database config and the ZeppelinShiro config, and then .... delete it ?

People are using Vault for everything, which is a bad idea.

[stvoutsin]

I agree, if we go with a local database within the Zeppelin node, than we don't need to store the credentials. We'd probably only need Vault if we are running a centralised database

[stvoutsin]

This should be completed in the last MR (#687)

[Zarquan]

I'd like to see this working before we close it. I haven't had time to setup a test system and see it running, sorry. Hope to get a chance to do it next week.

[Zarquan]

Created a separate issue for saving the MariaDB credentials in a text file #689.

[Zarquan]

This is all done.