fix(legacy-plugin-chart-country-map): higher quality geojson and re-arrange controls

[ktmud]

🏆 Enhancements
🐛 Bug Fix

Programmatically generate higher-quality GeoJSON files for country maps as noted in #955 (comment)

Updated all current GeoJSON files, as well as the Storybook.

Check the Jupyter notebook for details.

This should fix #569, as well as apache/superset#12987, apache/superset#4568, apache/superset#11666, and maybe more...

Next step is to update the Country Map Tools doc in Superset.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/superset/superset-ui/AnHkbRj3rugKp1VV95ou1DZveaWS
✅ Preview: https://superset-ui-git-fork-ktmud-country-map-superset.vercel.app

[codecov]

Codecov Report

Merging #1045 (0d85855) into master (93d7fb4) will increase coverage by 0.33%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #1045      +/-   ##
==========================================
+ Coverage   27.74%   28.07%   +0.33%     
==========================================
  Files         428      436       +8     
  Lines        8758     8904     +146     
  Branches     1314     1357      +43     
==========================================
+ Hits         2430     2500      +70     
- Misses       6155     6217      +62     
- Partials      173      187      +14     

Impacted Files	Coverage Δ
.../legacy-plugin-chart-country-map/src/CountryMap.js	0.00% <0.00%> (ø)
...egacy-plugin-chart-country-map/src/controlPanel.ts	0.00% <0.00%> (ø)
...s/legacy-plugin-chart-country-map/src/countries.ts	0.00% <0.00%> (ø)
...acy-plugin-chart-country-map/src/transformProps.js	0.00% <ø> (ø)
plugins/plugin-chart-echarts/src/utils/controls.ts	75.00% <0.00%> (-25.00%)	⬇️
plugins/plugin-chart-table/src/TableChart.tsx	41.37% <0.00%> (-8.63%)	⬇️
plugins/plugin-chart-table/src/transformProps.ts	61.62% <0.00%> (-4.63%)	⬇️
...lugins/plugin-chart-table/src/utils/formatValue.ts	64.28% <0.00%> (-2.39%)	⬇️
...gin-chart-echarts/src/Timeseries/transformProps.ts	41.81% <0.00%> (-2.19%)	⬇️
plugins/plugin-chart-table/src/buildQuery.ts	60.00% <0.00%> (-1.71%)	⬇️
... and 21 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93d7fb4...0d85855. Read the comment docs.

[kristw]

1. Have to be careful about the property name for backward compatibility. If the granularity of the map region is administrative level 1, NAME_1 should be presented. Could you confirm that it is true for all files?
2. Many fields such as wikipedia or mapcolor9 may be not necessary. Can remove some fields to reduce the size of the geojson file.

For example, Belgium,

OLD

properties: {
  "ID_0": 23,
  "ISO": "BE-BRU",
  "NAME_0": "Belgium",
  "ID_1": 1,
  "NAME_1": "Bruxelles",
  "ID_2": 1,
  "NAME_2": "Bruxelles",
  "TYPE_2": "Hoofdstedelijk Gewest|Région Capitale",
  "ENGTYPE_2": "Capital Region",
  "NL_NAME_2": null,
  "VARNAME_2": "Brussel Hoofstadt|Brusselse Hoofdstedelijke Gewest|Brüssel|Bruxelas|Région de Bruxelles-Capitale|Brussels|Bruselas"
}

NEW

properties: {
  "featurecla": "Admin-1 scale rank",
  "scalerank": 7,
  "adm1_code": "BEL-3474",
  "diss_me": 3474,
  "iso_3166_2": "BE-BRU",
  "wikipedia": null,
  "iso_a2": "BE",
  "adm0_sr": 1,
  "name": "Brussels",
  "name_alt": "Bruselas|Brussel Hoofstadt|Brusselse Hoofdstedelijke Gewest|Brüssel|Bruxelas|Région de Bruxelles-Capitale",
  "name_local": null,
  "type": "Hoofdstedelijk Gewest|Région Capitale",
  "type_en": "Capital Region",
  "code_local": null,
  "code_hasc": "BE.BU",
  "note": "Formerly part of Brabant (since 1962)",
  "hasc_maybe": null,
  "region": "Capital Region",
  "region_cod": null,
  "provnum_ne": 0,
  "gadm_level": 1,
  "check_me": 20,
  "datarank": 8,
  "abbrev": null,
  "postal": null,
  "area_sqkm": 0,
  "sameascity": 9,
  "labelrank": 9,
  "name_len": 8,
  "mapcolor9": 1,
  "mapcolor13": 8,
  "fips": "BE11",
  "fips_alt": "BE",
  "woe_id": 55965974,
  "woe_label": "Capital Region of Brussels, BE, Belgium",
  "woe_name": "Brussels",
  "latitude": 50.8332,
  "longitude": 4.36266,
  "sov_a3": "BEL",
  "adm0_a3": "BEL",
  "adm0_label": 2,
  "admin": "Belgium",
  "geonunit": "Brussels Capital Region",
  "gu_a3": "BCR",
  "gn_id": 6693370,
  "gn_name": "(Bruxelles-Capitale)",
  "gns_id": -2148026,
  "gns_name": "Krimpen aan den IJssel, Gemeente",
  "gn_level": 2,
  "gn_region": null,
  "gn_a1_code": "BE.BRU",
  "region_sub": null,
  "sub_code": null,
  "gns_level": 2,
  "gns_lang": null,
  "gns_adm1": null,
  "gns_region": "NL11",
  "min_label": 8.7,
  "max_label": 11.0,
  "min_zoom": 8.7,
  "wikidataid": "Q240",
  "name_ar": "إقليم بروكسل العاصمة",
  "name_bn": "ব্রাসেল্‌স",
  "name_de": "Region Brüssel-Hauptstadt",
  "name_en": "Brussels-Capital Region",
  "name_es": "Región de Bruselas-Capital",
  "name_fr": "région de Bruxelles-Capitale",
  "name_el": "Περιοχή των Βρυξελλών",
  "name_hi": "ब्रुसेल्स",
  "name_hu": "Brüsszel fővárosi régió",
  "name_id": "Daerah Ibu Kota Brussel",
  "name_it": "Regione di Bruxelles-Capitale",
  "name_ja": "ブリュッセル首都圏地域",
  "name_ko": "브뤼셀",
  "name_nl": "Brussels Hoofdstedelijk Gewest",
  "name_pl": "Region Stołeczny Brukseli",
  "name_pt": "Região de Bruxelas-Capital",
  "name_ru": "Брюссельский столичный регион",
  "name_sv": "Bryssel",
  "name_tr": "Brüksel Başkent Bölgesi",
  "name_vi": "Bruxelles",
  "name_zh": "布鲁塞尔-首都大区",
  "ne_id": 1159315379,
  "ISO": "BE-BRU",
  "NAME_1": "Brussels"
}

[ktmud]

Have to be careful about the property name for backward compatibility. If the granularity of the map region is administrative level 1, NAME_1 should be presented. Could you confirm that it is true for all files?

Adding just NAME_1 seems to suffice for backward compatibility---I didn't change any piece of current JavaScript code because it needs to support geojson files people may have generated using the old method.

Many fields such as wikipedia or mapcolor9 may be not necessary. Can remove some fields to reduce the size of the geojson file.

I don't think they matter much with gzip, but sure I can remove them for cleanliness.

[kristw]

Adding just NAME_1 seems to suffice for backward compatibility---I didn't change any piece of current JavaScript code because it needs to support geojson files people may have generated using the old method.

I meant could you confirm all the new files have field NAME_1? It might be a bit easier to check locally.

I don't think they matter much with gzip, but sure I can remove them for cleanliness.

Some clean up would be nice.

[ktmud]

I meant could you confirm all the new files have field NAME_1? It might be a bit easier to check locally.

Yes, they are globally set by creating an alias to the always defined "name" column from the original dataset.

[ktmud]

@kristw I did update the JavaScript a little to improve error-handling. Would you like to take another look?

[kristw]

lgtm

[j-manu]

@ktmud This has removed several states of India. Example: Telangana (IN-TG . Incorrectly labelled as IN-AN previously)

India Map data can be found in - https://github.com/datameet/maps with CC BY 4.0 license

[ktmud]

@j-manu Do you mind working on a PR to get this fixed? It should be easy to edit the Jupyter notebook and swap the geometry accordingly.

[j-manu]

@ktmud I haven't worked with Jupyter / superset or GeoJSON. I'm a user of superset via docker and did some digging because the version I have has Telangana labelled incorrectly. Then I discovered this commit which removed Telangana completely

It will be easier if I can assist you in fixing this.

1. The shape files for the states of India are available here https://github.com/datameet/maps/tree/master/States
   I have generated the GeoJSON from this and verified it has the right states (GeoJSON attached)
   states.geojson.zip

2. The states.geojson is not in the format that superset expects.
   a: No translations
   b: No ISO codes
   c: Pretty big in size (45 MB)

I'm not sure how (2)can be solved. The ISO codes can be found here - https://www.iso.org/obp/ui#iso:code:3166:IN
and I'm attaching a CSV which has that data (I have changed the names to ASCI)
States ISO Codes - Sheet1.csv

[ktmud]

@junlincc Does Preset has resource to pick this up?

[j-manu]

@ktmud I have created an issue for this - #1135 - so that it can be tracked