Default color cycle is too short

[jlstevens]

As part of #1518 I noted that our current color cycle is very short:

For comparison, the matplotlib 2.0 color cycle has 10 colors:

One suggestion is to extend our color cycle by appending the matplotlib purple, brown and any other distinguishable colors.

Another suggestion is to use the large color cycle in datashader that is the union of Set1,Set2 and Set3 (minus colors that are too similar).

Personally, I don't find all these colors quite distinguishable enough so I prefer the first option (extending our current cycle, especially as it is likely to cause less disruption).

[ea42gh]

http://colorbrewer2.org/#type=qualitative&scheme=Paired&n=12

you might also consider a sequential scheme. Note that colorblind schemes might be a useful alternative

[jlstevens]

@ea42gh We are considering just extending the existing color cycle. Do you have any particular issues with it (other than length) that would lead you to recommend something else entirely different?

[jlstevens]

@jbednar @philippjfr I don't think we ended up doing this for 1.8. Should we try to do it for 1.8.1? And if so, should we pretend we did do it so it can fall under the style_17 compatibility switch?

[jbednar]

I think it would fall under style_17 in any case, so it's not pretending, because that switch needs to restore whatever 1.7 did. Do you mean, so that we don't also have to make a style_18 switch? Sure.

[jlstevens]

Do you mean, so that we don't also have to make a style_18 switch? Sure.

Right.

[jlstevens]

To get the discussion going, here is my attempt to extend the cycle to 12 colors:

['#30a2da', '#fc4f30', '#e5ae38', '#6d904f', #8b8b8b',
cadetblue', 'brown', 'orange', 'LightCoral', 'ForestGreen', 'purple', 'FireBrick']

Rendered in bokeh using a dark theme (15 lines so the last 3 start the repeat):

And matplotlib with a light theme:

I think even this would be a definite improvement though the 3rd color is currently a bit too close to 'orange' for my liking. For now I'm using HTMl color names but I'll turn those strings to hex codes once we decide on the cycle we want.

[philippjfr]

It also seems like 7 and 12 are fairly close. Here's my suggestion with colors taken from Category10/tab10:

['#30a2da', '#fc4f30', '#e5ae38', '#6d904f', '#8b8b8b', '#9467bd',
 '#8c564b', '#e377c2', '#1f77b4', '#d62728', '#17becf', '#bcbd22']

[jlstevens]

Looks good! I think I would swap the last one (i.e 12) with the brown (7) to try and balance light and dark colors out a bit.

Otherwise, I would be happy to merge such a change. Certainly a lot better than what we had before!

[jlstevens]

Do we still want to control this change behind a config switch so people can go back to the short color cycle to make sure their plots don't suddenly look different?

[jbednar]

I have no objection, apart from wanting the cycle never to end, and thus to have as many colors after the ones here as are even remotely distinguishable, but I know you two don't agree...

[philippjfr]

Trying to find a principled way of ordering, e.g. by maximizing distance in lab color space. Think that's going to be worth it?

[jlstevens]

I have no objection, apart from wanting the cycle never to end...

Even 16 million colors isn't quite infinite. ;-p

[jlstevens]

Trying to find a principled way of ordering, e.g. by maximizing distance in lab color space. Think that's going to be worth it?

I wouldn't spend too much time on it but it might be worth having a go in case it results in a clearly superior looking cycle. That said the first 5 colors are still fixed for backwards compatibility...

[philippjfr]

Wrote a little visualization that might help, both showing distances in lab space:

[jlstevens]

Great! The problem is the first 5 entries are going down on y and those are ones we don't want to change.

Edit: Actually, thinking about it, that is a good thing. The issue is to avoid similar things being too close together as we can't avoid similar colors as we expand the space. It isn't obvious to me what we should change...

[philippjfr]

Right the smallest transition is from green to grey, which we are forced to keep for now.

[jbednar]

Be careful to use "fire" rather than "hot" in the one on the left if you want to draw valid conclusions. :-)

It's certainly possible to use a principled approach, e.g. taking a perceptually uniform color space and sampling it in a fractal-like way that chooses the first five most distinguishable colors, then fills in between them, then fills in between those, etc., gradually adding less discriminable colors. But even that isn't likely to be valid, because the perceptually uniform color spaces I know of are defined using large color patches that don't really apply to plotting lines or points. Our ability to detect the color of isolated thin lines or points varies quite dramatically depending on size. So really the only principled way to do it would be to run some psychophysical experiments testing discriminability for small dot sizes and thin lines, which there may be some research on but isn't accounted for in the major uniform color spaces (Lab, Luv, etc.). And it would be nice to run such testing on color blind people as well, e.g. so that the first five colors are color-blind safe, and then the rest relax that criterion (as making the entire cycle colorblind safe would drastically limit the number of colors available). That all adds up to a major research project, which I think someone should do, but which I don't personally have funding for...

[philippjfr]

Haha, the distance matrix is impossible to read anyway, so I'm mostly judging on the difference curve. Switching 7 and 12 as Jean-Luc suggested seems reasonable:

For future reference here's the code:

%%opts RGB Curve HeatMap [width=500 height=450 shared_axes=False] {+axiswise}
from colormath.color_objects import sRGBColor, LabColor
from colormath.color_conversions import convert_color
from colormath.color_diff import delta_e_cie2000

def hex_to_rgb(value):
    value = value.lstrip('#')
    lv = len(value)
    return tuple(int(value[i:i + lv // 3], 16)/255. for i in range(0, lv, lv // 3))

colors = ['#30a2da', '#fc4f30', '#e5ae38', '#6d904f', '#8b8b8b', '#9467bd',
          '#bcbd22', '#e377c2', '#1f77b4', '#d62728', '#17becf', '#8c564b']

rgbs = [hex_to_rgb(c) for c  in colors]

labs = [(c,convert_color(sRGBColor(*hex_to_rgb(c)), LabColor)) for c in colors]

cross = [(c1, c2, delta_e_cie2000(v1, v2)) for c1, v1 in labs for c2, v2 in labs]

diffs = [delta_e_cie2000(c1[1], c2[1]) for c1, c2 in zip(labs[:-1], labs[1:])]

rgb = hv.RGB(np.array(rgbs)[np.newaxis], bounds=(-1, -5, 11, 0))

hv.HeatMap(cross).options(xrotation=90, invert_yaxis=True, colorbar=True, cmap='hot') + hv.Curve(diffs) * rgb

[jbednar]

The one thing you can read clearly from the matrix is that the fifth color (gray) is hard to distinguish from everything. It would be a breaking change to get rid of it, but I do think that particular color is a liability.

[philippjfr]

Last variant I'll post:

[jlstevens]

The problem with the last one is that when it cycles, you'll have two blues next to each other.

[philippjfr]

The problem with the last one is that when it cycles, you'll have two blues next to each other.

Good point, should have included the cycle.

[philippjfr]

Okay last one:

colors = ['#30a2da', '#fc4f30', '#e5ae38', '#6d904f', '#8b8b8b', '#17becf',
          '#9467bd', '#d62728', '#1f77b4', '#e377c2', '#8c564b', '#bcbd22']

Guess we should vote or something?

[jbednar]

The last one there seems fine to me, except that I vote to remove gray entirely as not being a color and being hard to distinguish from all colors.

[jlstevens]

I think removing the gray should be an issue for 2.0 as removing it breaks backwards compatibility. Otherwise, I agree that it should be removed.

I should also mention that the last one also seems fine to me too.

[philippjfr]

The extended cycle is now, merged, moving to the 2.0 milestone to eventually replace the color cycle entirely.