Choropleth: allow to use custom maps #4599

kravets-levko · 2020-01-29T09:25:07Z

What type of PR is this? (check all applicable)

Feature

Description

Allow user to type arbitrary GeoJSON URL; existing build-it maps are available as dropdown options. Also, now instead of setting "Key Type" (type of key field from query result) user should map query result field to one of GeoJSON field.

Also fixed few small bugs in visualization.

Related Tickets & Documents

https://discuss.redash.io/t/choropleth-map-per-regions/4437

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

kravets-levko · 2020-01-29T09:27:46Z

The only thing that really makes this feature completely useless is this:

@arikfr Do you have any ideas what we can do with it?

arikfr · 2020-01-29T09:35:18Z

@kravets-levko not sure 🤔 The simplest solution? Give up on dynamic values there and keep adding different maps to our build. We should just make it very simple and documented process, so anyone can make a PR for this.

…t be saved in options, only keys)

kravets-levko · 2020-01-29T10:32:07Z

@arikfr How about adding connect-src *; to default CSP rules? According to docs, it will allow only requests triggered by JS code (and it fixes the issue mentioned in my first comment - I already tried it on my dev instance).

…'t edit them; refine code a bit

kravets-levko · 2020-01-29T22:30:43Z

Everything works fine, except of that CSP issue ☝️

arikfr · 2020-01-30T09:24:50Z

@arikfr How about adding connect-src *; to default CSP rules? According to docs, it will allow only requests triggered by JS code (and it fixes the issue mentioned in my first comment - I already tried it on my dev instance).

Allowing requests to be triggered by JS code sounds like a potential risk, and the kind of risk CSP supposed to prevent.

I thought of a different solution: have a Proxy endpoint in Redash that loads this JSON endpoint and returns it. The complex part is to make this endpoint secure by checking that this URL is indeed in the visualization definition.

arikfr · 2020-01-30T09:26:37Z

The complex part is to make this endpoint secure by checking that this URL is indeed in the visualization definition.

But this actually doesn't have to be that hard:

Make a call to /api/resource-proxy with params: URL to load & visualization id.
In the handler, load the visualization, check that it actually defines the URL.

Only issue: what to do before saving? :|

kravets-levko · 2020-01-30T09:36:18Z

Proxy is an option of course.

Only issue: what to do before saving? :|

What if proxy instead of checking visualization object will just allow to load any JSON files? Even if we'll need such a feature in other visualization (maybe one day?) - most likely we'll not need anything except of JSON. So proxy may load URL, check if it returned JSON and then pass that JSON to frontend. And of course allow only GET methods. Does it sound safe?

arikfr · 2020-01-30T09:38:12Z

Thinking of it; making only GET requests should be fine.

kravets-levko · 2020-02-11T10:38:43Z

@arikfr I added a basic implementation of proxy to load custom maps in 813d97a Most likely it has to be improved, so please review it - I'll appreciate any suggestions about it 🙇‍♂️

arikfr · 2020-02-11T10:44:50Z

redash/handlers/__init__.py

@@ -22,6 +23,14 @@ def status_api():
    return jsonify(status)


+@routes.route("/resource-proxy", methods=["GET"])


Let's put this in the api namespace (i.e. /api/resource-proxy) to avoid issues in environments like our deployment preview.

Done c7b1345

kravets-levko · 2020-09-24T11:51:25Z

Another relevant idea to discuss (and possibly implement): here I added some more metadata to Choropleth maps, but having it in our codebase will make it harder to add custom maps. The idea is to embed those metadata into GeoJSON files (it's allowed by GeoJSON specs). It will solve problem with custom maps, but also there are few cons: 1) map name couldn't be embedded because it's needed before the file is loaded, and 2) it will be a bit harder to find those mappings (e.g. to fix them).

jimsparkman · 2020-10-07T20:07:07Z

Is this still being considered? Would love to see this feature added.

kravets-levko · 2020-10-07T20:10:23Z

Hi @jimsparkman! Yes, I'll resume my work on it soon. I have to re-think some details first

opioom · 2020-11-27T10:30:48Z

@kravets-levko are u able to estimate any deadline? We use redash for our clients (big hospitals in PL) and we need to analyze the patients diagnosis' distribution. This would be a great feature for us to have it in redash. Currently the client has to speak with ESRI which of course cost a lot of money.

kravets-levko · 2020-11-27T11:43:17Z

Hi @opioom! Sorry, no any estimates for now. I totally understand how useful this feature is, but ATM I'm loaded with other stuff.

dengc367 · 2020-12-18T11:19:00Z

great job. when to merge to the master branch?

chrispruitt · 2020-12-28T21:01:58Z

Nice work on this! Any update yet on when this will be merged?

opioom · 2021-02-26T11:25:04Z

it's been 6 months since my last post so....do we have any progress? Any other options like workaround that could help to me visualize the PL regions with any data?

mathieubossaert · 2021-02-27T17:46:25Z

it's been 6 months since my last post so....do we have any progress? Any other options like workaround that could help to me visualize the PL regions with any data?

Waiting for this great feature too, for the moment I "dirty" replaced map files (countries.geo.json and japan.prefecture.geo.json) by files with same structure and names but with my own data (square grid and municipalities).

I created specific volumes for it.

    volumes:
      - /opt/redash/maps/countries.geo.json:/app/client/dist/data/0962607.countries.geo.json
      - /opt/redash/maps/japan.prefectures.geo.json:/app/client/dist/data/650f3ee.japan.prefectures.geo.json

edouardjmn · 2021-03-31T10:39:48Z

@mathieubossaert Thank you so much! Works well 🙌🏻

opioom · 2021-08-30T19:18:12Z

@mathieubossaert that's sounds like a solution! Do you have any detailed guide how to do that? I desperate need that function for healthcare analysis.

mathieubossaert · 2021-09-21T13:17:23Z

Hi @opioom , I will document this trick / workaround after the release of v10.

mathieubossaert · 2021-12-07T12:55:25Z

@opioom
As promised on september.
We "simply" need to change the content of the 3 geo.json files (keeping same names) used by redash, with your own content.
In my case I needed to show 3 kind of maps :

cities and villages from Occitanie
10km square mesh.
french "Departements"

So I replace Japan prefectures, counties and USA geojson files with mine..
Let's say my geo.json files are located on the host into /opt/redash/maps/ directory.
Their names are

communes_occitanie.geo.json
mailles_10km.geo.json
departements.geo.json

We "simply" need to mount server container geojson files to the hosted mines.
In the end of the server section of the docker-compose file, add

    volumes:
      - /opt/redash/maps/communes_occitanie.geo.json:/app/client/dist/data/0962607.countries.geo.json
      - /opt/redash/maps/mailles_10km.geo.json:/app/client/dist/data/dac34f3.japan.prefectures.geo.json
      - /opt/redash/maps/departements.geo.json:/app/client/dist/data/eed73d1.usa-albers.geo.json

Before that, you can check the name of the files used by redash container (name changes for a file on V10 upgrade on my instance)

docker exec -t -i redash_server_1 ls /app/client/dist/data/

0962607.countries.geo.json
eed73d1.usa-albers.geo.json
dac34f3.japan.prefectures.geo.json

opioom · 2021-12-10T21:43:26Z

Thanks a lot @mathieubossaert!! Will try to follow your guide on my test env.
Edit: I've made a small modification because it didn't work via docker compose. I simply replaced json files inside redash_server (docker cp) with my own json files. The only fix I have to make are missing labels on a map, but it's probably related to the json file structure.

mathewtrivett · 2022-06-05T12:00:29Z

Realise this is an old PR. I was wondering whether it is possible to customise the polygons / geojson used by the choropleth visualisation? Would be happy to contribute to get this integrated.

guidopetri · 2023-07-21T19:38:55Z

@kravets-levko , thanks for the PR! We've updated a lot of things now that we're Community-driven so - if you're still interested (or @mathewtrivett ) in getting this merged - would you mind rebasing off master to re-run the CI, as well as updating merge conflicts?

We're trying to clean up our PR todo list, so if you're not interested, that's fine - we'll close the PR in about a week if we don't hear back. If you're interested in reopening the PR afterwards, we would also very much welcome that.

justinclift · 2023-07-22T08:29:25Z

Lets keep this one around for a while longer. It'll probably need to be reworked, but it's too important to just lose to history in the short term. 😄

Choropleth: allow to use custom maps

8f7d1d8

kravets-levko added Frontend Visualizations labels Jan 29, 2020

kravets-levko requested review from arikfr and gabrieldutra January 29, 2020 09:25

kravets-levko self-assigned this Jan 29, 2020

Use separate input for custom map URL (pre-defined map URLs should no…

6260601

…t be saved in options, only keys)

kravets-levko added 5 commits January 29, 2020 13:05

Keep last custom map URL when selecting predefined map type

4911764

Use cache for geoJson requests

3b29f0c

Don't handle bounds changes while loading geoJson data

3f280b1

Choropleth: fix map "jumping" on load; don't save bounds if user didn…

6187448

…'t edit them; refine code a bit

Improve cache; fix typo

b331c4c

kravets-levko marked this pull request as ready for review January 29, 2020 22:29

weekly-digest bot mentioned this pull request Feb 3, 2020

Weekly Digest (27 January, 2020 - 3 February, 2020) #4604

Closed

kravets-levko mentioned this pull request Feb 11, 2020

Add korea map Choropleth #4637

Closed

1 task

Use proxy to load custom maps (to bypass CSP)

813d97a

arikfr reviewed Feb 11, 2020

View reviewed changes

kravets-levko added 3 commits February 11, 2020 12:49

Load pre-defined maps directly; move proxy to /api namespace

c7b1345

Optimize Japan Perfectures map (remove irrelevant GeoJson properties)

bc9dd81

Merge branch 'master' into choropleth-custom-map

4357872

weekly-digest bot mentioned this pull request Feb 17, 2020

Weekly Digest (10 February, 2020 - 17 February, 2020) #4652

Closed

arikfr mentioned this pull request Feb 18, 2020

(WIP) feat: Added Australia to Choropleth visualisation #4374

Closed

1 task

vivekr-bi approved these changes Apr 6, 2020

View reviewed changes

gabrieldutra mentioned this pull request Apr 15, 2020

Visualizations customizable settings #4793

Merged

1 task

kravets-levko mentioned this pull request Sep 21, 2020

Some Choropleth improvements/refactoring #5186

Merged

8 tasks

guidopetri added the 1-week-review label Jul 21, 2023

guidopetri removed the 1-week-review label Aug 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Choropleth: allow to use custom maps #4599

Choropleth: allow to use custom maps #4599

kravets-levko commented Jan 29, 2020 •

edited

Loading

kravets-levko commented Jan 29, 2020

arikfr commented Jan 29, 2020

kravets-levko commented Jan 29, 2020

kravets-levko commented Jan 29, 2020

arikfr commented Jan 30, 2020

arikfr commented Jan 30, 2020

kravets-levko commented Jan 30, 2020

arikfr commented Jan 30, 2020

kravets-levko commented Feb 11, 2020

arikfr Feb 11, 2020

kravets-levko Feb 11, 2020

kravets-levko commented Sep 24, 2020

jimsparkman commented Oct 7, 2020

kravets-levko commented Oct 7, 2020

opioom commented Nov 27, 2020

kravets-levko commented Nov 27, 2020

dengc367 commented Dec 18, 2020

chrispruitt commented Dec 28, 2020

opioom commented Feb 26, 2021

mathieubossaert commented Feb 27, 2021 •

edited

Loading

edouardjmn commented Mar 31, 2021

opioom commented Aug 30, 2021 •

edited

Loading

mathieubossaert commented Sep 21, 2021

mathieubossaert commented Dec 7, 2021 •

edited

Loading

opioom commented Dec 10, 2021 •

edited

Loading

mathewtrivett commented Jun 5, 2022

guidopetri commented Jul 21, 2023

justinclift commented Jul 22, 2023

		@@ -22,6 +23,14 @@ def status_api():
		return jsonify(status)


		@routes.route("/resource-proxy", methods=["GET"])

Choropleth: allow to use custom maps #4599

Are you sure you want to change the base?

Choropleth: allow to use custom maps #4599

Conversation

kravets-levko commented Jan 29, 2020 • edited Loading

What type of PR is this? (check all applicable)

Description

Related Tickets & Documents

Mobile & Desktop Screenshots/Recordings (if there are UI changes)

kravets-levko commented Jan 29, 2020

arikfr commented Jan 29, 2020

kravets-levko commented Jan 29, 2020

kravets-levko commented Jan 29, 2020

arikfr commented Jan 30, 2020

arikfr commented Jan 30, 2020

kravets-levko commented Jan 30, 2020

arikfr commented Jan 30, 2020

kravets-levko commented Feb 11, 2020

arikfr Feb 11, 2020

Choose a reason for hiding this comment

kravets-levko Feb 11, 2020

Choose a reason for hiding this comment

kravets-levko commented Sep 24, 2020

jimsparkman commented Oct 7, 2020

kravets-levko commented Oct 7, 2020

opioom commented Nov 27, 2020

kravets-levko commented Nov 27, 2020

dengc367 commented Dec 18, 2020

chrispruitt commented Dec 28, 2020

opioom commented Feb 26, 2021

mathieubossaert commented Feb 27, 2021 • edited Loading

edouardjmn commented Mar 31, 2021

opioom commented Aug 30, 2021 • edited Loading

mathieubossaert commented Sep 21, 2021

mathieubossaert commented Dec 7, 2021 • edited Loading

opioom commented Dec 10, 2021 • edited Loading

mathewtrivett commented Jun 5, 2022

guidopetri commented Jul 21, 2023

justinclift commented Jul 22, 2023

kravets-levko commented Jan 29, 2020 •

edited

Loading

mathieubossaert commented Feb 27, 2021 •

edited

Loading

opioom commented Aug 30, 2021 •

edited

Loading

mathieubossaert commented Dec 7, 2021 •

edited

Loading

opioom commented Dec 10, 2021 •

edited

Loading