feat: add /usr/share as a config location

[PureTryOut]

/usr/share/ is becoming more and more common for applications to store their default configuration in. Either the application itself or distributions can use this location to ship a config that users will not touch, and instead they are expected to overwrite (parts of) the config in /etc/. This means this config file can be safely updated with new required values without having to worry about overwriting user-made change.

This is useful in general but especially in case of immutable distributions who don't touch anything in /etc and /home

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Please upload report for BASE (dev@9a8fd1f). Learn more about missing BASE report.

❗ Current head ac01002 differs from pull request most recent head a10d5c6

Please upload reports for the commit a10d5c6 to get more accurate results.

Files	Patch %	Lines
ovos_config/config.py	0.00%	5 Missing ⚠️
ovos_config/locations.py	0.00%	4 Missing ⚠️
ovos_config/models.py	0.00%	4 Missing ⚠️
ovos_config/meta.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##             dev    #128   +/-   ##
=====================================
  Coverage       ?   0.00%           
=====================================
  Files          ?      10           
  Lines          ?     826           
  Branches       ?       0           
=====================================
  Hits           ?       0           
  Misses         ?     826           
  Partials       ?       0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[NeonDaniel]

We have documented /etc config as not user-managed so I'm not sure that this new path adds anything functionally different. SYSTEM is read-only to the user and USER is where user overrides are allowed. Would overriding those existing envvars would be sufficient for distros where /etc or /home defaults aren't appropriate without increasing the complexity of the already complex config stack?

[j1nx]

I agree with @PureTryOut that more and more linux packages and/ or their developers use /user/share as their default shipped location for configuration files.

It will look like this;
~/.config/ovos/ovos.conf - User OVOS configuration
/etc/ovos/ovos.conf - System OVOS configuration
/usr/share/ovos/ovos.conf - Default OVOS configuration (Basically the default with everything from this ovos-config package)

However, while following best practices in Linux world then perhaps we could also look into using the drop-in config fragment systems they use.

/usr/share/ovos/ovos.conf.d/
/etc/ovos/ovos.conf.d/
~/.config/ovos/ovos.conf.d/

Any *.conf files in those directories with only the configuration fragments of interest will be applied (overwritten from the above) in alphabetically order.

A distribution or package maintainer that wants to ship with specific different configuration values other then the ovos-config default values could use the drop-in directory.

In this case for @PureTryOut he could ship his package with own config file located in /usr/share/ovos/ovos.conf.d/

I also agree with @mikejgray that users and configuration files sometimes do not go well (RTFM) however above appears to be the new Linux way / best practices.

[builderjer]

To me, this is the best solution. I love the idea of /ovos.conf.d
raspOVOS could ship /usr/share/ovos/ovos.conf.d/raspOVOS.conf with the specific changes there.
Or when using my custom location, just add another conf file to ~/.config/ovos/ovos.conf.d/personal_location.conf
That would make it WAY easier with multiple changes.

[PureTryOut]

SYSTEM is read-only to the user

System being /etc, no it's not. Sure, any configuration done through OVOS won't be written there, but a system administrator (read: someone with root rights) most definitely can. And by putting the config file in /etc you as a program are implying that it contains values that can be overwritten.

/etc is used by system administrators to configure a service independent of the account that is running. However sometimes distributions will want to ship their own configs and they need to know that they can safely update them without having to worry about overwriting user-made changes.
For example a distribution having an image for the Raspberry Pi could decide to by default enable a specific STT or TTS that is known to work well with that device. A user can then choose to overwrite it (either system-wide if an admin or for their specific user) with some other option. Then the distribution decides that some better option has come along and changes their distro configuration to use the new option and deprecates/removes the old option. The user is at this point still overwriting it so it won't be used, until they go back to their distro-default by removing their own configuration, at which point the newly chosen STT/TTS from the distro is used rather than the old one which at that point might have been deprecated or even removed entirely.

If this wasn't an option and the user has at any point changed their config in /etc, the distro can't update the default config any more because that would overwrite user changes. Meanwhile they might remove an STT option from their distro at which point the installation breaks for the user with no easy way to get back to a working state, because there is no config any more that follows whatever the distro intends.

[JarbasAl]

config fragments is not a bad idea and deserves it's own issue to discuss/implement separately from this one!

[mikejgray]

My 2 cents, as I was tagged: if adding /usr/share makes it easier to ship for a particular distro and conforms more closely to standard Linux practices I'm all for it. However, as a non-blocking separate PR, I'd like to see the number of config file locations decrease by 1 to offset that. IMO we have too many places for config files already and the order of precedence is confusing. This is frequently a source of many user problems and precedence is confusing in many areas - for example, the AWS SDKs and CLI has 10 places to configure credentials, in order of what overrides what, and this is often a considerable problem when troubleshooting permissions in applications and services.

Long-term the solution is to have a user-friendly web UI and eventually a voice UI for handling configurations, but if it's possible to simplify an ever-expanding number of config files, we should do it. Separating config into fragments where each service is responsible for owning and updating its own configuration is also a good practice and I'm very much in support of that - especially since OVOS has morphed into a fully distributed architecture at this point.

I have some thoughts about using something like etcd or even a relational database for config versus flat files. That has been thoroughly addressed elsewhere and I am squarely in the minority, so I'll refrain from echoing those ideas here, especially since they really have nothing to do with this PR and the excellent suggestion and work from @PureTryOut . 🙂

[JarbasAl]

However, as a non-blocking separate PR, I'd like to see the number of config file locations decrease by 1 to offset that. IMO we have too many places for config files already and the order of precedence is confusing.

we can drop the remote config maybe, any backend implementation can manage the user config instead

the drawbacks here are that we lose the ability to block specific updates from remote, and need to update the ipgeo plugin. but those we can handle in their respective repos

and as mentioned in chat, i want to drop the whole ovos.conf eventually, but thats another discussion

[PureTryOut]

In my mind there would ideally be this order of configs, the earlier being overwritten by things later in the chain:

Distribution (/usr/share/ovos/ovos.conf) -> System (/etc/ovos/ovos.conf) -> User (XDG base dir, so unless the relevant env specified, $HOME/.config/ovos/ovos.conf).

One possible problem however would be that I don't know if it would be possible for pip packages to ship their config file to any of these locations. This is relevant for skills and plugins of course. I suppose we could keep the current "default" which just means in-code configuration if no config option for it is found in any of the above mentioned locations. Then we have 4 locations in total which I think is alright. Definitely don't do a ~/.ovos like that AWS tool lol.

[builderjer]

Pip can definitely place a file in the user's $HOME directory. I do it often.

[JarbasAl]

Pip can definitely place a file in the user's $HOME directory. I do it often.

on that note, this is useful for individual skills that want to ship default skill settings.json