Remodel data storage

[TheMarex]

Current we support two data backends hidden behind BaseDatafacade. Both codes uses different code paths for loading data into memory, ideally it should be the same.

The whole data backend should be re-written:

• Move all relevant code to libosrm-storage
• Implement only one data facade
• Make shared memory named (e.g. by profile name)
• There should only be one datafacade (abstraction at different level)
• Make loading new data as transparent to the server component as possible

@danpat @daniel-j-h extend if you have any more points

[TheMarex]

Will also fix #1894

[frodrigo]

This will allow multiple shared memory on same host ?

[daniel-j-h]

Yes, via named shared memory; e.g. for multiple profiles.

[danpat]

We need to implement a notification/wait strategy for data reloading as well. There is currently some confusion about which process is responsible for cleaning up no-longer-used shared memory.

The workflow should be:

1. osrm-datastore loads a new block of shared memory.
2. Processes using the old block are notified of a new block and begin the switch.
3. osrm-datastore waits until all users are switched, then removes the old block.

Currently, the readers remove the old block themselves, and osrm-datastore returns as soon as it's done loading. The "notification" is implemented by osrm_impl/osrm-routed checking for new data when a query comes in; this should be changed to an async notification method, and readers should not be responsible for cleaning up old shared memory once they've swapped - this would be difficult to synchronize when there are many readers, and is currently broken :-/

[emiltin]

It would be great to be able to run multiple instances of osrm-routed with different profiles, ie osrm-datastore should be able to keep multiple data source in memory at once.

[daniel-j-h]

@danpat you refactored the data facades - care to have a look here and either update what still needs to be done or close this and split off a different issue with remaining task.

[danpat]

#3165 consolidated the logic behind the two datafacades - we now only have a single file-loading codepath.

The only thing we haven't done is make the shared memory regions named.