Integrate Kepler.gl as a new visualization type

[mistercrunch]

Kepler.gl is an amazing geospatial React application. It offers a very
well designed user interface to compose geo layers and slice and dice
geospatial data.

As it turns out, Kepler is designed to be embedded as part of React
apps. It took me only a few hours to get to a working prototype.

Next steps:

• allow to save the state, without the data
• restore the state, and merge with the new data

* tweak styles, make a light style that looks decent within Superset, Kepler uses stylable components, so that should be possible

[codecov-io]

Codecov Report

Merging #6403 into master will decrease coverage by 0.79%.
The diff coverage is 5.76%.

@@            Coverage Diff            @@
##           master    #6403     +/-   ##
=========================================
- Coverage   65.43%   64.64%   -0.8%     
=========================================
  Files         432      428      -4     
  Lines       21319    20696    -623     
  Branches     2347     2291     -56     
=========================================
- Hits        13951    13378    -573     
+ Misses       7252     7192     -60     
- Partials      116      126     +10

Impacted Files	Coverage Δ
...ts/src/visualizations/presets/DeckGLChartPreset.js	0% <ø> (ø)	⬆️
superset/assets/src/explore/controls.jsx	42.4% <ø> (-0.46%)	⬇️
superset/assets/src/explore/controlPanels/index.js	76.92% <ø> (ø)	⬆️
...ets/src/visualizations/Kepler/KeplerChartPlugin.js	0% <0%> (ø)
...perset/assets/src/visualizations/Kepler/Kepler.jsx	0% <0%> (ø)
superset/assets/src/visualizations/Kepler/store.js	0% <0%> (ø)
...assets/src/visualizations/Kepler/transformProps.js	0% <0%> (ø)
superset/viz.py	71.24% <40%> (-0.7%)	⬇️
superset/assets/src/explore/validators.js	0% <0%> (-68.75%)	⬇️
superset/assets/src/explore/store.js	61.84% <0%> (-31.91%)	⬇️
... and 117 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5243182...ff32008. Read the comment docs.

[mistercrunch]

Improved the solution to support saving the state properly, using its own redux store, autozoom, readonly and more.

[mistercrunch]

This is to align Kepler's deck.gl dep with Superset's. It's important as having 2 versions of deck.gl errors out.

[heshan0131]

Accidentally found this PR. Great work! let us know if you need any help on our side. I found your next item is to change the styling. You can follow This issue, it will make overriding style a lot easier!

[vylc]

@mistercrunch any movement on this in getting to the next version of deck?

[mistercrunch]

@heshan0131 I missed the notification on your previous comment, but thank you for your note! The main things I had questions around are:

• are there any events I could hook onto when controls are changing? for now I had to hook up something to the onSaveMap event. Superset assumes that the visualization controls are outside the visualization, but passes a method to the visualization to allow it to change the value of a superset control. For now what I do is dump the state of Kepler inside a Superset TextControl, but we had to tell the users to go and push that button prior to saving... I wish I have a generic onKeplerStateChange prop or something like that...
• is there a light style mode as opposed to the dark theme? I dug through the Kepler code and found some reference to a light theme, but it seemed to apply only to some modals... Maybe we just need a dark mode for Superset :)

[heshan0131]

@mistercrunch kepler.gl currently doesn't have a prop to listen on onStateChange event, but you are right, we should provide more callbacks such as onStateChange onMouseHover etc. We are slowly learning how people use the framework. Before we add that, you can add middleware to your store and use KeplerGlSchema to serialize the state then diff it.

Something like:

import {KeplerGlSchema} from 'kepler.gl/schemas';
import {taskMiddleware} from 'react-palm/tasks';

let previousValue = null;

const stateChangeMiddleware = store => next => action => {
  const returnValue = next(action);

  const keplerState = store.getState().keplerGl.map;
  const stateToSave = KeplerGlSchema.getConfigToSave(keplerState);
  const hash = JSON.stringify(stateToSave);
    
  if (previousValue !== hash) {
     // do something here
     console.log('kepler.gl state has changed');
     previousValue = hash;
   }

    return returnValue;
};

const middlewares =[taskMiddleware, stateChangeMiddleware];
const enhancers = [applyMiddleware(...middlewares)];
const store = createStore(reducers, {}, compose(...enhancers));

return store;

[heshan0131]

Also we don't have a complete light theme yet (also on our todo list)
@macrigiuseppe has created a custom theme example which contains a fairly reasonable light theme, just need a little bit touch up. Worth a try

[mistercrunch]

@heshan0131 this is super helpful, thanks for the input and the great tool!

[mistercrunch]

@heshan0131, I made it work! I had to adapt the code a little and rate limit the changes as it made the map interactions sluggish (we update a control onChange and causes a chain of renders).

[mistercrunch]

Looks like theme support is in a newer version. I have a branch trying to upgrade kepler + deck to latest

[mistercrunch]

Tried theming kepler 1.0 (not released yet) and it looks promising but didn't work out of the box

[heshan0131]

@heshan0131, I made it work! I had to adapt the code a little and rate limit the changes as it made the map interactions sluggish (we update a control onChange and causes a chain of renders).

You can try skip ActionTypes.LAYER_HOVER and ActionTypes.UPDATE_MAP. This will significantly reduce state serialization being called

using import {ActionTypes} from 'kepler.gl/actions'; to get access to the constant

[mistercrunch]

Good input, thought we want to capture the viewport in the state, so the rate limiting option seems like like the best option for now.

[mistercrunch]

@betodealmeida @xtinec @vylc Kepler properly save its state into Superset after the changes I added recently

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[chintanp]

@mistercrunch Any ideas when this will make it to Superset?

[mistercrunch]

It should be easy to re-do this based on the new plugin interface.
https://preset.io/blog/2020-07-02-hello-world/

[DrissiReda]

@mistercrunch I still had issues with that. I tried using both kepler and deckgl then I tried replacing deckgl with kepler, these are the errors I get:

• When replaced : "export 'DeckGLChartPreset' was not found in '@superset-ui/legacy-preset-chart-kepler'
• When using both: "export 'KeplerChartPreset' was not found in '@superset-ui/legacy-preset-chart-kepler'
• When using both: "export 'KeplerChartPlugin' was not found in '@superset-ui/legacy-preset-chart-kepler'
• When using both: "export 'Kepler' was not found in '@superset-ui/legacy-preset-chart-kepler'

What exactly should I use, the README.md of the kepler plugin is a copy paste of the deckgl plugin.