Implement count_cat and custom colormapping in BokehJS

[jbednar]

There are several open and fairly serious usability issues (#126, #127, #136, #251) that all stem from the fact that Datashader provides its own colormapping facilities beyond what is available from interactive plotting libraries where the Datashader plot is embedded. Specifically, Datashader provides eq_hist for parameter-free color normalization, and count_cat [now generalized to by] mixing of color categories, along with support for arbitrary Python functions for normalization. These functions are extremely useful and powerful, but if users choose to use them (e.g. via datashade() in HoloViews), the plotting library cannot support interactive features like it will if you use the plotting library's colormapping (e.g. by adding rasterize() and letting Bokeh do the colormapping). Thus if datashader does the colormapping, there is not currently any support for having colorbars, legends, and hovering of data values; all Bokeh sees is the final RGBA colors, and not the data values needed to support such interactive features.

@jlstevens and I propose that the best solution to this problem is probably just to go ahead and reimplement Datashader's colormapping from within BokehJS. The colormapping is a relatively inexpensive operation, applying to a matrix of bins (aggregated data) rather than to the raw data, so the speed and scalability of Datashader shouldn't be needed for this task. Plus it's not all that much code, compared to the rest of Datashader. And having the colormapping be performed client-side in the browser will not only instantaneously fix all the above issues, but such features will work even in a static export, because they will no longer require any server processing. The Datashader-specific colormapping will still be available, when rendering to static images, but now interactivity can be fully supported.

Specific tasks expected:

• Provide an eq_hist function in BokehJS, in a way that supports all the usual features provided by the existing linear and log mappings.
• Provide some way to map the output of count_cat (a stack of 2D arrays) into an RGBA image. Could first require a new HoloViews Element (an ImageStack?), or could be a way to visualize an Overlay of Images, e.g. via the old Compositor approach. Needs some research and brainstorming!
• Provide a way to supply an arbitrary normalization function, to handle cases like cube roots? Presumably the function would need to be written in JavaScript, but that doesn't seem too onerous a restriction given how simple most such functions would be (other than eq_hist).

[jlstevens]

This sounds good to me though I suspect I want to tackle the last task and then the first task listed above. In other words, work towards a generalized and more flexible colormapping approach in Bokeh that can then support functionality such as eq_hist.

The middle task about a potential new element/compositing will need more discussion as you point out.

[jbednar]

Sure; could go either way.

[jbednar]

eq_hist is now available in Bokeh, and using the Bokeh version is now preferred in nearly all cases that aren't categorical. For categorical plots, Datashader's shade (really colorize) is still required.

[jbednar]

Implemented in bokeh/bokeh#12356 . Still needs an example in Datashader.