Memory Usage

[tooneamelter]

I mapped the planet. planet-latest.osm.pbf is 59G, osrm dir is as follows:
-rw-rwxr--+ 1 osrm osrm 59G Jul 23 14:10 planet-latest.osrm
-rw-rw-r-- 1 osrm osrm 61G Jul 23 19:41 planet-latest.osrm.cell_metrics
-rw-rw-r-- 1 osrm osrm 401M Jul 23 19:17 planet-latest.osrm.cells
-rw-rwxr--+ 1 osrm osrm 4.1G Jul 23 15:47 planet-latest.osrm.cnbg
-rw-rwxr--+ 1 osrm osrm 4.1G Jul 23 19:04 planet-latest.osrm.cnbg_to_ebg
-rw-rw-r-- 1 osrm osrm 68K Jul 23 19:24 planet-latest.osrm.datasource_names
-rw-rwxr--+ 1 osrm osrm 23G Jul 23 19:18 planet-latest.osrm.ebg
-rw-rwxr--+ 1 osrm osrm 5.9G Jul 23 19:07 planet-latest.osrm.ebg_nodes
-rw-rwxr--+ 1 osrm osrm 6.7G Jul 23 16:19 planet-latest.osrm.edges
-rw-rwxr--+ 1 osrm osrm 5.9G Jul 23 19:14 planet-latest.osrm.enw
-rwxrwx---+ 1 osrm osrm 25G Jul 23 19:05 planet-latest.osrm.fileIndex
-rw-rwxr--+ 1 osrm osrm 27G Jul 23 16:20 planet-latest.osrm.geometry
-rw-rwxr--+ 1 osrm osrm 4.7G Jul 23 16:19 planet-latest.osrm.icd
-rw-rwxr--+ 1 osrm osrm 7.0K Jul 23 19:14 planet-latest.osrm.maneuver_overrides
-rw-rw-r-- 1 osrm osrm 24G Jul 23 19:42 planet-latest.osrm.mldgr
-rw-rwxr--+ 1 osrm osrm 196M Jul 23 14:10 planet-latest.osrm.names
-rw-rwxr--+ 1 osrm osrm 15G Jul 23 15:46 planet-latest.osrm.nbg_nodes
-rw-rw-r-- 1 osrm osrm 3.9G Jul 23 19:17 planet-latest.osrm.partition
-rw-rwxr--+ 1 osrm osrm 6.0K Jul 23 14:10 planet-latest.osrm.properties
-rw-rwxr--+ 1 osrm osrm 100M Jul 23 16:27 planet-latest.osrm.ramIndex
-rw-rwxr--+ 1 osrm osrm 4.0K Jul 23 16:08 planet-latest.osrm.restrictions
-rw-rwxr--+ 1 osrm osrm 3.5K Jul 23 13:32 planet-latest.osrm.timestamp
-rw-rwxr--+ 1 osrm osrm 24K Jul 23 16:19 planet-latest.osrm.tld
-rw-rwxr--+ 1 osrm osrm 55K Jul 23 16:19 planet-latest.osrm.tls
-rw-rwxr--+ 1 osrm osrm 1.9G Jul 23 16:08 planet-latest.osrm.turn_duration_penalties
-rw-rwxr--+ 1 osrm osrm 12G Jul 23 16:08 planet-latest.osrm.turn_penalties_index
-rw-rwxr--+ 1 osrm osrm 1.9G Jul 23 16:08 planet-latest.osrm.turn_weight_penalties

Running osrm-routed this way, via systemd:
root@gisserver:/usr/sbin# cat /etc/systemd/system/osrm-routed.service
[Unit]
Description=Open Source Routing Machine
Wants=network-online.target
After=network.target network-online.target

[Service]
ExecStart=/usr/local/bin/osrm-routed --algorithm=MLD /srv/downloads/planet-latest.osrm
User=osrm
Group=osrm
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

I'm wondering if there is a better way to run osrm-routed so it doesn't chew up as much memory. I have 512GB and 56 Cores:

[mjjbell]

You might want to try the --mmap option, which will allow you to run osrm-routed in more memory constrained environments at the expense of some paging.

A memory reduction could be achieved by not loading datasets that your use-case doesn't need (#5218), but that has not been implemented yet, and requires some codebase refactoring as a preliminary step.

[isemichastnov]

@tooneamelter Can you pls shed some light - how much RAM did it use while preprocessing the entire planet pbf?

[merschformann]

Can you pls shed some light - how much RAM did it use while preprocessing

I shared some insights from my last time processing the planet osm here @isemichastnov . I hope it helps.
But maybe you already found your answer in the meantime. 😅

[SiarheiFedartsou]

Closed as stale.