Pelias Who's on First Data Importer

This repository is part of the Pelias project. Pelias is an open-source, open-data geocoder built by Mapzen that also powers Mapzen Search. Our official user documentation is here.

Overview

pelias-whosonfirst is a tool used for importing Who's On First data from local files into a Pelias ElasticSearch store.

Requirements

Node.js 4 or higher is required

Types

There are two major categories of Who's on First data supported: hierarchy (or admin) data, and venues.

Hierarchy data represents things like cities, countries, counties, boroughs, etc.

Venues represent individual places like the Statue of Liberty, a gas station, etc. Venues are subdivided by country, and sometimes regions within a country.

Currently, the supported hierarchy types are:

• borough
• continent
• country
• county
• dependency
• disputed
• localadmin
• locality
• macrocounty
• macroregion
• neighbourhood
• region

Other types may be included in the future.

This page has a description of all the types supported by Who's on First.

Configuration

The sole Pelias configuration option available is imports.whosonfirst.datapath. It should point to the root of where Who's On First data has been downloaded.

Downloading the Data

• The enclosed script download_data.js will download the required bundles and place the data into the datapath configured in pelias-config in the required directory layout. To install the required node module dependencies and run the download script:

npm install
npm run download

## or

npm run download -- --admin-only # to only download hierarchy data, and not venues (venues require around 100GB of disk space)

Warning: Who's on First data is big. Just the hierarchy data is tens of GB, and the full dataset is over 100GB on disk. Additionally, Who's on First uses one file per record. In addition to lots of disk space, you need lots of free inodes. On Linux/Mac, df -ih can show you how many free inodes you have.

Expect to use a few million inodes for Who's on First. You probably don't want to store multiple copies of the Who's on First data due to its disk requirements.

Usage

To install the required node module dependencies and execute the importer, run:

$> npm install
$> npm start