Skip to content

uglykidmat/geonames

BranchesTags

Repository files navigation

Geonames v2

"Reverse/Geocoding Webservices" : this repository contains the Geonames Controller, built on a Symfony 6.3.1 skeleton.

The general information can be found here : https://www.notion.so/gtrsuite/Geonames-5561d74e241c4fee8dcf0ee39c4a1221

Deployment

1. Download/Installation

  git clone https://github.com/Gatoreviews/geonames.git
  composer install

2. Update database configuration in your .env files.

  DATABASE_URL="postgresql://{{db_user}}:{{db_password}}@{{127.0.0.1:5432}}/{{db_name}}?serverVersion=15&charset=utf8"

3. then run these commands to create the database :

php bin/console doctrine:database:create
php bin/console doctrine:migrations:migrate

the database should then be ready for hydration.

4. Run these commands in order :

symfony server:start -d to start the server.

Countries hydration

tl;dr

Hydration
php bin/console app:cu
php bin/console app:clu
php bin/console app:clvu
php bin/console app:cgu
php bin/console app:cbu
php bin/console app:adcu AD AE AF AG AI AL AM AO AR AS AT AU AX AZ BA BB BD BE BF BG BH BI BJ BM BN BO BQ BR BS BT BW BY BZ CA CD CF CG CH CI CL CM CN CO CR CU CV CY CZ DE DJ DK DM DO DZ EC EE EG EH ER ES ET FI FJ FM FO FR GA GB GD GE GF GH GM GN GP GQ GR GT GU GW GY HK HN HR HT HU ID IE IL IN IQ IR IS IT JM JO JP KE KG KH KM KN KP KR KW KZ LA LB LC LI LK LR LS LT LU LV LY MA MC MD ME MG MH MK ML MM MN MP MQ MR MS MT MU MV MW MX MY MZ NA NC NE NG NI NL NO NP NR NZ OM PA PE PF PG PH PK PL PM PR PS PT PW PY QA RE RO RS RU RW SA SB SC SD SE SH SI SJ SK SL SM SN SO SR ST SV SY SZ TD TF TG TH TJ TL TM TN TO TR TT TW TZ UA UG UM US UY UZ VC VE VI VN VU WF WS YE YT ZA ZM ZW
php bin/console app:wmf 7535697
___DELETE COUNTRY 'FO' DUPLICATES???___
php bin/console app:adaltu
php bin/console app:adlu AD AE AF AG AI AL AM AO AR AS AT AU AX AZ BA BB BD BE BF BG BH BI BJ BM BN BO BQ BR BS BT BW BY BZ CA CD CF CG CH CI CL CM CN CO CR CU CV CY CZ DE DJ DK DM DO DZ EC EE EG EH ER ES ET FI FJ FM FO FR GA GB GD GE GF GH GM GN GP GQ GR GT GU GW GY HK HN HR HT HU ID IE IL IN IQ IR IS IT JM JO JP KE KG KH KM KN KP KR KW KZ LA LB LC LI LK LR LS LT LU LV LY MA MC MD ME MG MH MK ML MM MN MP MQ MR MS MT MU MV MW MX MY MZ NA NC NE NG NI NL NO NP NR NZ OM PA PE PF PG PH PK PL PM PR PS PT PW PY QA RE RO RS RU RW SA SB SC SD SE SH SI SJ SK SL SM SN SO SR ST SV SY SZ TD TF TG TH TJ TL TM TN TO TR TT TW TZ UA UG UM US UY UZ VC VE VI VN VU WF WS YE YT ZA ZM ZW
API/Export
php bin/console app:adapi {lang} {countryCode}
php bin/console app:ade {locale} {level}

1. update the countries

  • basic country information. This performs a purge of the "geonames_country" table and fills it up with fresh information from Geonames. As of november 2023, there were 250 entries.
php bin/console app:cu

2. update the countries' locales (translated names)

  • Countries names translated into different languages. The table "country_locale" must first be updated with geonames information :
php bin/console app:clu

ℹ️ This command runs a few subprocesses which manage the different files containing the Ids, since loading them all at once caused a timeout error. Importing these files should result in about 43000 new entries in the database.

To update all the names ("locales") for a specific country, you need its geonamesId (find it here https://www.geonames.org/countries/, click on the desired country name, then on the name again, the ID will be in the URL). Then run :

php bin/console app:clsu 3017382

to get all of France's names, for example.

3. Update country level

⚠️ make sure you have 'geonames_country_level.json' in your 'base_data' root folder :

/country/level/update

or

php bin/console app:clvu

4. Update GeoJson data

⚠️ Make sure the file 'geonames_geojson.json' is in the 'base_data' root folder :

/geojson/update

or

php bin/console app:cgu

Will update the database entries (countries or administrative divisions) if their geonameID is found.

5. Country Barycenters

⚠️ Make sure the file 'geonames_country_barycenters.json' is in the 'base_data' root folder. Then visit : /country/barycenters/update and the country table will be updated accordingly.

To update a specific country's barycenter :

/country/barycenter/{string countryCode}

or

php bin/console app:cbu

This script will compute the approximate barycenter of the country and update it in the database.

Subdivisions hydration

  • Update your GEONAMES_TOKEN variable in the correct .envfile.
  • Import/Update : The command line arguments are countryCode, featureCode ("ADM1","ADM2","ADM3") and "startRow" which sets the start of the geonames Response content. Example :
php bin/console app:adu FR ADM1 1

will yield the first level Administrative Divisions of France, starting from row 1. Updates are done by batch of 1000 entries, so the next logical steps would be to run the same command with the second argument increased by 1000, like :

php bin/console app:adu DE ADM1 1000
php bin/console app:adu IT ADM3 2000
php bin/console app:adu JP ADM2 1
php bin/console app:adu ES ADM2 1000

Multiple countries' ADM can be imported at once :

php bin/console app:adu DE,ES,FI,CH ADM1 1

Update subdivisions by children :

php bin/console app:adcu {countrycodes}

This command will run a database hydration by parent <-> children geonames Search.

It is important for your current database to empty of administrative divisions from the countries you are trying to import.
The list of countries must be separated by a blank space.

The command to run a FULL hydration is : ⚠️ this command can take a while to finish and can be hard on the RAM.

Make sure you are not running on an 8Gb RAM Laptop like me.

php bin/console app:adcu AD AE AF AG AI AL AM AO AR AS AT AU AX AZ BA BB BD BE BF BG BH BI BJ BM BN BO BQ BR BS BT BW BY BZ CA CD CF CG CH CI CL CM CN CO CR CU CV CY CZ DE DJ DK DM DO DZ EC EE EG EH ER ES ET FI FJ FM FO FR GA GB GD GE GF GH GM GN GP GQ GR GT GU GW GY HK HN HR HT HU ID IE IL IN IQ IR IS IT JM JO JP KE KG KH KM KN KP KR KW KZ LA LB LC LI LK LR LS LT LU LV LY MA MC MD ME MG MH MK ML MM MN MP MQ MR MS MT MU MV MW MX MY MZ NA NC NE NG NI NL NO NP NR NZ OM PA PE PF PG PH PK PL PM PR PS PT PW PY QA RE RO RS RU RW SA SB SC SD SE SH SI SJ SK SL SM SN SO SR ST SV SY SZ TD TF TG TH TJ TL TM TN TO TR TT TW TZ UA UG UM US UY UZ VC VE VI VN VU WF WS YE YT ZA ZM ZW

Some administrative divisions have an alternative admincode. To import them, ⚠️ make sure the file 'geonames_alternative_divisions.json' is in the 'base_data' folder, then visit

/administrativedivisions/alternatives/update

or

php bin/console app:adaltu

Quick command to import multiple geonameIds :

php bin/console app:adbu {geoid},{geoid},{geoid},...

the Ids must be separated by a comma.

Locales update :

php bin/console app:adlu {CountryCode}

As for the subdivisions update above, it is possible to import every translation for every country in one command :

php bin/console app:adlu AD AE AF AG AI AL AM AO AR AS AT AU AX AZ BA BB BD BE BF BG BH BI BJ BM BN BO BQ BR BS BT BW BY BZ CA CD CF CG CH CI CL CM CN CO CR CU CV CY CZ DE DJ DK DM DO DZ EC EE EG EH ER ES ET FI FJ FM FO FR GA GB GD GE GF GH GM GN GP GQ GR GT GU GW GY HK HN HR HT HU ID IE IL IN IQ IR IS IT JM JO JP KE KG KH KM KN KP KR KW KZ LA LB LC LI LK LR LS LT LU LV LY MA MC MD ME MG MH MK ML MM MN MP MQ MR MS MT MU MV MW MX MY MZ NA NC NE NG NI NL NO NP NR NZ OM PA PE PF PG PH PK PL PM PR PS PT PW PY QA RE RO RS RU RW SA SB SC SD SE SH SI SJ SK SL SM SN SO SR ST SV SY SZ TD TF TG TH TJ TL TM TN TO TR TT TW TZ UA UG UM US UY UZ VC VE VI VN VU WF WS YE YT ZA ZM ZW

This command can take a moment to execute, depending on the country's depth.

Security information

  • /status is publicly available.
  • /geonames/search is accessible via a Bearer Token.
  • /geonames/api is accessible via a Bearer Token.
  • /* any other page is accessible via basic_auth.

Api Documentation

/api/doc.json for a json output /api/doc for the Nelmio API documentation /api for the standard API documentation

Usage/Examples

The Geoname controller has a few functions :

POST endpoint

  /geonames/search

Handles POST requests : the content must be a JSON string following this structure (for a single entry ; if more are needed, just separate the {} with a ,) :

[
  {
    "elt_id": "4M SP04801", #unique element id
    "country_code": "FR",   #2-letter country code. If not found in a list, returns an empty content.
    "zip_code": "30900",    #zipcode respecting the country's format
    "lat": 43.818134,       #latitude
    "lng": 4.347509         #longitude
  },
  {
    ...
  }
]

The search will be on the coordinates, and use the postalcode/countrycode couple to find the subdivisions. This URL is protected by a token, if not provided you will encounter a 401 error.

GET endpoints

Hydrate the database first, for every country code needed. Example for Japan :

/administrativedivisions/locales/update/JP

then call the endpoint :

  /administrativedivisions/api/{lang}/{countryCode}

The inline command for this API endpoint (JSON generation) is :

  php bin/console app:adapi {lang} {countryCode}

Country list :

  /country/list/{lang}

A GET request on this URL will return the country list with the following information : countryCode, geonameId, name. The name will be translated in {lang}, for example /fr, /de, etc

Example : /administrativedivisions/api/it/FR Handles GET requests for getting all the subdivision names of a country in a specific language.

EXPORT endpoint

  /administrativedivisions/export/{locale}/{level}
php bin/console app:ade {locale} {level}

Example : /administrativedivisions/export/fr/0, /administrativedivisions/export/it/1 Handles GET requests for getting all the subdivisions in a specific language, filtered by level. 0 represents the countries, 1,2,3 the subdivisions. The result page contains JSON information, and a file with the same content will be created in the /var/geonames_export_data/ folder.

Translations

ℹ️ "Translations" act as locales overrides. The following endpoints are used to manage these translations. Use different HTTP methods on the translation API endpoint /translation:

  • GET : paginated list
  • POST : bulk creation
  • PATCH : bulk update
  • DELETE : bulk deletion

The expected content should follow this syntax, geonameId and locale are required :

[{
	"geonameId": "123456",
	"name": "name_in_locale_language",
	"countryCode": "FR",
	"fcode": "ADM1",
	"locale": "fr"
},
 {
	"geonameId": "123457",
	"name": "name_in_locale_language",
	"countryCode": "UK",
	"fcode": "COUNTRY",
	"locale": "de"
}]

Calling POST with it will create new entries, PATCH will modify existing entries (if found), and DELETE will delete them. For an initial database hydration, just copy the JSON content from 'geonames_translation.json' in your 'base_data' folder, and use it as request on the POST endpoint.

The /export URL will serve a .csv file with all the current translation entries.

/translation/export

Subdivisions

  1. To clean database entries, use app:adp followed by the featureCode.
php bin/console app:adp ADM1
  1. Translations list
  • Calling the endpoint /alternatives/{locale} (locale being a 2-letter code) will return a list of every subdivions names in the selected locale.

Global search in Symfony database :

/search/{string geoquery}-{string featureCode}

Search for a keyword geoquery (eg. "New York", "Chambéry", etc) associated with a featureCode (ADM1,ADM1H,ADM2,ADM2,ADM3,ADM3,ADM4,ADM4,ADM5,ADM5,ADMD,ADMD,LTER,PC,PCLD,PCLF,PCLH,PCLI,PCLI,PCLS,PRSH,TERR,Z,ZNB). See https://www.geonames.org/export/codes.html for more information.

Add a geonames entry to the local database :

/globalgetjson/{int geonamesId}

Information on a specific Geonames Id :

/geonamesid/{int geonamesId}

Search by postal code :

/postalcodesearch/{int postalcode}

Search by nearby postal code :

/nearbypostalcode/{string countrycode}-{int postalcode}

Search by Latitude and Longitude :

/latLng/{int lat}-{int lng}

Countries

  1. Translations list
  • Calling the endpoint /countrynames/{locale} (locale being a 2-letter code) will return a list of every country names in the selected locale.
  1. Other

Search for some information on a country :

/country/{string countryCode}

Information about all country-levels :

/country/level/get

Information about a specific country (countryCode being a 2-letter string) :

/country/level/get/{string countryCode}

GeoJson information

/geojson/get

will return the information from every entry in the database.

/geojson/get/{int geonameId}

will return the information for a specific geonameID, if found in the database.

API Search

To get information directly from the Geonames API, the endpoints will be under /geonamesapi/ :

/geonamesapi/postalcodesearch/{string postalcode}

returns a list of (10 by default) postal codes and places for the placename/postalcode query

/geonamesapi/postalcodelookup/{string postalcode}-{string countrycode}

returns a list of places for the given postalcode in JSON format, sorted by postalcode,placename

Commands

Terminal commands :

php bin/console Latlngsearch <lat> <lng>

returns a json string of the Geonames location closest the latitude and longitude provided.

Running the tests :

php vendor/bin/phpunit --testdox

About

Temp

Resources

Readme
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 5

Languages