Serverless services

[javfg]

This PR adds Serverless services to Reva. It is the first step before the big PR for the Notifications Service.

This adds a new type of service which doesn't have a listener (neither http or grp). As the Notifications service is communicating via NATS only with other backend services, it doesn't need to be exposed.

It could later on be used to implement other services which don't need http/grp endpoints, like metrics.

[update-docs]

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

[labkode]

@javfg can you add an example of how the config would look like for an example service?

[gmgigi96]

This is awesome, we could use the "serverless" services for running some internal reva procedures!

Regarding the code, I have a comment regarding the management of the services. In the HTTP and GRPC services, revad is handling properly the graceful shutdown when a signal is received (check https://github.com/cs3org/reva/blob/master/cmd/revad/internal/grace/grace.go#L258), so every service does not have to implement its own signal handler.

Also, how do you deal with errors in a single service? We don't have any way of propagating this to the Serverless struct. If we have a way then, do we need to restart the single service? Do we need to panic the entire reva?

[javfg]

This is awesome, we could use the "serverless" services for running some internal reva procedures!

Regarding the code, I have a comment regarding the management of the services. In the HTTP and GRPC services, revad is handling properly the graceful shutdown when a signal is received (check https://github.com/cs3org/reva/blob/master/cmd/revad/internal/grace/grace.go#L258), so every service does not have to implement its own signal handler.

I looked into this initially and thought about adding some way to deal with the shutdown for serverless in grace.go too, but in the end I did not because I'm not sure if there is a way to deal with them in a unified way.

Maybe we could discuss this during next week? I thought perhaps using context (which serverless services don't have right now), and maybe using context.WithCancel to start the services?

Also, how do you deal with errors in a single service? We don't have any way of propagating this to the Serverless struct. If we have a way then, do we need to restart the single service? Do we need to panic the entire reva?

Not dealing with them at all (a broken service crashes the whole Reva instance), and indeed this is something that is missing too. I'm not sure about how we handle them in rhttp/rgrpc. Another thing to discuss, I could use some input.

[javfg]

@javfg can you add an example of how the config would look like for an example service?

Sure! Here is the config for the notifications service:

[serverless.services.notifications]
nats_address = ""
nats_token = "TOKEN"
nats_template_subject = "reva-notifications-template"
nats_notification_subject = "reva-notifications-notification"
nats_trigger_subject = "reva-notifications-trigger"
storage_driver = "sql"
grouping_interval = 5
grouping_maxsize = 5

[labkode]

@javfg thanks!

@gmgigi96 has a good point. Even if they are server less, I expect that these services will create network connections to upstream services and maybe open local files. These cases should be handled gracefully if possible.

Adding the following methods shouldn't be difficult. We can split the interface into two: Gracer and Listener

Stop() error
GracefulStop() error

[glpatcern]

@javfg can you add an example of how the config would look like for an example service?

Sure! Here is the config for the notifications service:

Thanks! I'd push this in the example folder, the PRs tend to get lost...