Possibility to have the zenoh/zenohd configuration splitted across multiple files.

[gabrik]

Hi, while developing the Zenoh Flow plugin for Zenoh we had the need of modifying the zenohd configuration in order to add the default configuration for the plugin.

This means manually editing a JSON file, sometimes through an SSH session with vi, and that's not funny.
Furthermore, this cannot be easily scripted in a maintainer script when creating a .deb or .rpm package.

So my suggestion is to have the possibility to "split" the configuration file, similarly to what Nginx does with the sites-available directory or the *.d directories in /etc (e.g., /etc/apt/sources.list.d). More info on the *.d here.

In this case, one can simply copy the default configuration for the plugin in the directory and it will be automatically used by zenoh, this is quite easy to do as a maintainer script in packages.

What do you think?

[p-avital]

Two issues with this, one purely semantic and the other halfway between technical and semantic:

• Pure semantic: with all the latest works on configuration, we've tried to avoid the environment having too much of an impact on configuration (we don't have a ZENOH_CFG environment variable for example, which would allow to not pass -c <file> to configure, but also allows us to see that users are using custom cfgs just from their launch commands when requesting support). Having a config folder would go against that approach. This is arguably easy to solve by having -c <dir.d> be used to load directories.
• For split configurations, we need to define (and implement) the merge operation. Say a config has two files with conflicting config, how do we handle that? How does merging deal with two arrays for the same key (should they be concatenated as we would expect of two objects for the same key? in what order? or should they be declared as conflict)?

[JEnoch]

Do we really want by default zenohd to automatically load a plugin as soon as it's installed ?
This could cause security issues if the plugin is not secured by default (e.g. MQTT not using TLS by default).
And some users may want to start several routers on the same host with different configurations (i.e. without some plugins).

So I would prefer the plugins to come with their own sub-config files (in /etc/zenohd/), but keeping /etc/zenohd/zenohd.json5 untouched. Up to the user to edit it, adding or uncommenting a line to add the plugin sub-config.
We could probably echo such instruction at plugin installation.

[cguimaraes]

I also share the view of @gabrik on this. It is much easier to handle separate files, especially if configurations for the plugins can be self-contained. It seems to me more aligned with what devops or sysadmins would expect to see.

@JEnoch would it be possible to have different configuration files but not to create the hierarchical dependency?

@JEnoch @p-avital as of today, do we have any example of conflicting configs between Zenoh and the plugins?

As of today, plugins are loaded during the bootstrap of zenohd. But it could be interesting that in the future we could load new plugins without restarting zenohd...for this it would much easier to have standalone configuration files for the each plugin.

[JEnoch]

It is much easier to handle separate files, especially if configurations for the plugins can be self-contained.

Sure, that's what my suggestion allows: you can have 1 conf file per plugin (and also per storage) that is included in the full config for zenohd.

would it be possible to have different configuration files but not to create the hierarchical dependency?

The config for zenohd is structured, so it's hierachical in a way. Why would you not want a hierarchical dependency for sub-parts of the config ?

as of today, do we have any example of conflicting configs between Zenoh and the plugins?

No. Today their can't be conflict as all is defined in a single JSON5 structure where an attribute can only be defined once.

But it could be interesting that in the future we could load new plugins without restarting zenohd

That can be done via the admin space, via a PUT on the config key expression (in the same way we can add storages at runtime).

[cguimaraes]

If plugins can be already loaded dynamically in runtime with a PUT on the admin space, I am expecting that plugins configuration are also provided afterwards. In this case, there is no hierarchical dependency for sub-configurations with the parent zenoh-config.json.

Also, if plugin configurations are standalone and there are no conflicts in terms of properties, then I think it is reasonable to satisfy @gabrik request in the initial post.

It is a requirement that he has as a Zenoh user, and it seems that the problems we are raising are in fact no problems as of today (or even envisioned future).

My two cents.

[JEnoch]

If plugins can be already loaded dynamically in runtime with a PUT on the admin space, I am expecting that plugins configuration are also provided afterwards. In this case, there is no hierarchical dependency for sub-configurations with the parent zenoh-config.json.

"dependency" is probably not the correct word. There is an hierarchical structure of the configuration:

{
   transport: { ... },
   scouting: { ... },
    // ...
   plugins: {
      plugin1: { ... },
      plugin2: { ... },
      storage_manager: {
         volumes: {
            vol1: { ... },
            vol2:  { ... },
        },
        storages: {
           storage1: { ... },
           storage2: { .... },
       }
   },
}

Each sub-part being optional, and some might be updated at runtime (e.g. to add/remove a plugin or a storage).

@p-avital point was that if several files redefine the full config, some sub-parts might be conflicting and the merge into a single config might be complex.
My point is that we should rather authorise split only for sub-parts in a hierarchical way, starting from a single top-level file. Thus no possible conflict.

Also, if plugin configurations are standalone and there are no conflicts in terms of properties, then I think it is reasonable to satisfy @gabrik request in the initial post.

Actually there are 2 requests in the initial post:

1. split a config in several files
2. automatically load an "installed" plugin, without changing the default /etc/zenohd/zenohd.json5

Wrt. point 2: prior to 0.6.0-beta.1 zenohd was searching the plugins libraries on filesystem and automatically loading them.
We removed this feature in 0.6.0-beta.1 for security concerns. I think those concerns still apply.