Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Styler builtins: highlight_between #39821

Merged
merged 69 commits into from
Apr 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
22653c6
merged
attack68 Feb 16, 2021
6c73f62
merged
attack68 Feb 16, 2021
fe08398
upodate docs and consistency
attack68 Feb 17, 2021
aaa15bc
upodate docs and consistency
attack68 Feb 17, 2021
a362fbf
add and update tests
attack68 Feb 17, 2021
6ab6dca
highlight range now accepts sequence
attack68 Feb 18, 2021
411ef6b
highlight range now accepts sequence
attack68 Feb 18, 2021
85e5241
highlight range now accepts sequence
attack68 Feb 18, 2021
4f105d6
examples
attack68 Feb 18, 2021
26f6d12
reorder
attack68 Feb 18, 2021
e7d9d19
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 19, 2021
92eb5fd
adj tests
attack68 Feb 19, 2021
cbea15a
adj tests
attack68 Feb 19, 2021
caef3a5
whats new
attack68 Feb 19, 2021
dc12b0c
to_numpy()
attack68 Feb 19, 2021
95c1a68
docstring fix
attack68 Feb 19, 2021
d965635
test fix
attack68 Feb 19, 2021
b30b099
improve coding remove conditional
attack68 Feb 19, 2021
d947c00
docs links
attack68 Feb 20, 2021
f248797
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 23, 2021
5ffe49f
add tests to new modules.
attack68 Feb 23, 2021
6b5bc67
add examples and input validation
attack68 Feb 23, 2021
c9d70ed
add examples and input validation
attack68 Feb 23, 2021
8baf65f
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 23, 2021
2971556
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 25, 2021
869154e
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 27, 2021
f409884
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Feb 28, 2021
bc1eb6b
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 1, 2021
e086fc7
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 2, 2021
1aefc6b
name change: highlight_range to highlight_between
attack68 Mar 4, 2021
7f8d335
split PR
attack68 Mar 5, 2021
e4abcbe
split PR
attack68 Mar 5, 2021
3522160
split PR
attack68 Mar 5, 2021
ed4a758
split PR
attack68 Mar 5, 2021
5e982c7
split PR
attack68 Mar 5, 2021
cc4148b
split PR
attack68 Mar 5, 2021
41c1c38
split PR
attack68 Mar 5, 2021
a36bc5b
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 5, 2021
5fc8124
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 5, 2021
0bd7323
remove reference to highlight_quantile
attack68 Mar 5, 2021
b14504d
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 5, 2021
7c7a337
link see alsos
attack68 Mar 5, 2021
f80d012
doc edit
attack68 Mar 6, 2021
0228ab7
doc edit
attack68 Mar 6, 2021
2a81ea1
Merge branch 'master' into range_builtin
attack68 Mar 8, 2021
472d321
remove bool option from inclusive
attack68 Mar 10, 2021
d03b905
Merge branch 'master' into range_builtin
attack68 Mar 10, 2021
37b131f
use operator
attack68 Mar 10, 2021
3533763
remove misleading random note from test fixtures
attack68 Mar 11, 2021
c69f29d
remove misleading random note from test fixtures
attack68 Mar 11, 2021
7731978
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 14, 2021
9168be7
mypy fix
attack68 Mar 14, 2021
0c92a50
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 15, 2021
86fb87c
filesnames
attack68 Mar 15, 2021
53bcc1b
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 15, 2021
bbb63d4
add arg shape validation function
attack68 Mar 17, 2021
6f59dbf
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 17, 2021
6dd8137
mypy updates
attack68 Mar 17, 2021
8bf57aa
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 20, 2021
e9a4840
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 20, 2021
4a575d4
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 22, 2021
f6858d1
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 23, 2021
45be30a
integrate into new test structure
attack68 Mar 23, 2021
dcc651d
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Mar 24, 2021
c779961
remove axis from validate arg func
attack68 Mar 29, 2021
adf607d
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Apr 9, 2021
210a00d
no bool in inclusive
attack68 Apr 9, 2021
73861f6
revert random explicit
attack68 Apr 10, 2021
3c57373
Merge remote-tracking branch 'upstream/master' into range_builtin
attack68 Apr 12, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added doc/source/_static/style/hbetw_axNone.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/_static/style/hbetw_basic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/_static/style/hbetw_props.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/source/_static/style/hbetw_seq.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions doc/source/reference/style.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ Builtin styles
Styler.highlight_null
Styler.highlight_max
Styler.highlight_min
Styler.highlight_between
Styler.background_gradient
Styler.bar

Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.3.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@ Other enhancements
- :meth:`.Styler.background_gradient` now allows the ability to supply a specific gradient map (:issue:`22727`)
- :meth:`.Styler.clear` now clears :attr:`Styler.hidden_index` and :attr:`Styler.hidden_columns` as well (:issue:`40484`)
- Builtin highlighting methods in :class:`Styler` have a more consistent signature and css customisability (:issue:`40242`)
- :meth:`.Styler.highlight_between` added to list of builtin styling methods (:issue:`39821`)
- :meth:`Series.loc.__getitem__` and :meth:`Series.loc.__setitem__` with :class:`MultiIndex` now raising helpful error message when indexer has too many dimensions (:issue:`35349`)
- :meth:`pandas.read_stata` and :class:`StataReader` support reading data from compressed files.
- Add support for parsing ``ISO 8601``-like timestamps with negative signs to :meth:`pandas.Timedelta` (:issue:`37172`)
Expand Down
156 changes: 156 additions & 0 deletions pandas/io/formats/style.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from contextlib import contextmanager
import copy
from functools import partial
import operator
from typing import (
Any,
Callable,
Expand All @@ -21,6 +22,7 @@
FrameOrSeries,
FrameOrSeriesUnion,
IndexLabel,
Scalar,
)
from pandas.compat._optional import import_optional_dependency
from pandas.util._decorators import doc
Expand Down Expand Up @@ -1352,6 +1354,7 @@ def highlight_null(
--------
Styler.highlight_max: Highlight the maximum with a style.
Styler.highlight_min: Highlight the minimum with a style.
Styler.highlight_between: Highlight a defined range with a style.
"""

def f(data: DataFrame, props: str) -> np.ndarray:
Expand Down Expand Up @@ -1399,6 +1402,7 @@ def highlight_max(
--------
Styler.highlight_null: Highlight missing values with a style.
Styler.highlight_min: Highlight the minimum with a style.
Styler.highlight_between: Highlight a defined range with a style.
"""

def f(data: FrameOrSeries, props: str) -> np.ndarray:
Expand Down Expand Up @@ -1446,6 +1450,7 @@ def highlight_min(
--------
Styler.highlight_null: Highlight missing values with a style.
Styler.highlight_max: Highlight the maximum with a style.
Styler.highlight_between: Highlight a defined range with a style.
"""

def f(data: FrameOrSeries, props: str) -> np.ndarray:
Expand All @@ -1459,6 +1464,157 @@ def f(data: FrameOrSeries, props: str) -> np.ndarray:
f, axis=axis, subset=subset, props=props # type: ignore[arg-type]
)

def highlight_between(
self,
subset: IndexLabel | None = None,
color: str = "yellow",
axis: Axis | None = 0,
left: Scalar | Sequence | None = None,
right: Scalar | Sequence | None = None,
inclusive: str = "both",
props: str | None = None,
) -> Styler:
"""
Highlight a defined range with a style.

.. versionadded:: 1.3.0

Parameters
----------
subset : IndexSlice, default None
A valid slice for ``data`` to limit the style application to.
color : str, default 'yellow'
Background color to use for highlighting.
axis : {0 or 'index', 1 or 'columns', None}, default 0
If ``left`` or ``right`` given as sequence, axis along which to apply those
boundaries. See examples.
left : scalar or datetime-like, or sequence or array-like, default None
Left bound for defining the range.
right : scalar or datetime-like, or sequence or array-like, default None
Right bound for defining the range.
inclusive : {'both', 'neither', 'left', 'right'}
Identify whether bounds are closed or open.
props : str, default None
CSS properties to use for highlighting. If ``props`` is given, ``color``
is not used.

Returns
-------
self : Styler

See Also
--------
Styler.highlight_null: Highlight missing values with a style.
Styler.highlight_max: Highlight the maximum with a style.
Styler.highlight_min: Highlight the minimum with a style.

Notes
-----
If ``left`` is ``None`` only the right bound is applied.
If ``right`` is ``None`` only the left bound is applied. If both are ``None``
all values are highlighted.

``axis`` is only needed if ``left`` or ``right`` are provided as a sequence or
an array-like object for aligning the shapes. If ``left`` and ``right`` are
both scalars then all ``axis`` inputs will give the same result.

This function only works with compatible ``dtypes``. For example a datetime-like
region can only use equivalent datetime-like ``left`` and ``right`` arguments.
Use ``subset`` to control regions which have multiple ``dtypes``.

Examples
--------
Basic usage

>>> df = pd.DataFrame({
... 'One': [1.2, 1.6, 1.5],
... 'Two': [2.9, 2.1, 2.5],
... 'Three': [3.1, 3.2, 3.8],
... })
>>> df.style.highlight_between(left=2.1, right=2.9)

.. figure:: ../../_static/style/hbetw_basic.png

Using a range input sequnce along an ``axis``, in this case setting a ``left``
and ``right`` for each column individually

>>> df.style.highlight_between(left=[1.4, 2.4, 3.4], right=[1.6, 2.6, 3.6],
... axis=1, color="#fffd75")

.. figure:: ../../_static/style/hbetw_seq.png

Using ``axis=None`` and providing the ``left`` argument as an array that
matches the input DataFrame, with a constant ``right``

>>> df.style.highlight_between(left=[[2,2,3],[2,2,3],[3,3,3]], right=3.5,
... axis=None, color="#fffd75")

.. figure:: ../../_static/style/hbetw_axNone.png

Using ``props`` instead of default background coloring

>>> df.style.highlight_between(left=1.5, right=3.5,
... props='font-weight:bold;color:#e83e8c')

.. figure:: ../../_static/style/hbetw_props.png
"""

def f(
data: FrameOrSeries,
props: str,
left: Scalar | Sequence | np.ndarray | FrameOrSeries | None = None,
right: Scalar | Sequence | np.ndarray | FrameOrSeries | None = None,
inclusive: bool | str = True,
) -> np.ndarray:
if np.iterable(left) and not isinstance(left, str):
left = _validate_apply_axis_arg(
left, "left", None, data # type: ignore[arg-type]
)

if np.iterable(right) and not isinstance(right, str):
right = _validate_apply_axis_arg(
right, "right", None, data # type: ignore[arg-type]
)

# get ops with correct boundary attribution
if inclusive == "both":
ops = (operator.ge, operator.le)
elif inclusive == "neither":
ops = (operator.gt, operator.lt)
elif inclusive == "left":
ops = (operator.ge, operator.lt)
elif inclusive == "right":
ops = (operator.gt, operator.le)
else:
raise ValueError(
f"'inclusive' values can be 'both', 'left', 'right', or 'neither' "
f"got {inclusive}"
)

g_left = (
ops[0](data, left)
if left is not None
else np.full(data.shape, True, dtype=bool)
)
l_right = (
ops[1](data, right)
if right is not None
else np.full(data.shape, True, dtype=bool)
)
return np.where(g_left & l_right, props, "")

if props is None:
props = f"background-color: {color};"
return self.apply(
f, # type: ignore[arg-type]
axis=axis,
subset=subset,
props=props,
left=left,
right=right,
inclusive=inclusive,
)

@classmethod
def from_custom_template(cls, searchpath, name):
"""
Expand Down
74 changes: 73 additions & 1 deletion pandas/tests/io/formats/style/test_highlight.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import numpy as np
import pytest

from pandas import DataFrame
from pandas import (
DataFrame,
IndexSlice,
)

pytest.importorskip("jinja2")

Expand Down Expand Up @@ -70,3 +73,72 @@ def test_highlight_minmax_ext(df, f, kwargs):
df = -df
result = getattr(df.style, f)(**kwargs)._compute().ctx
assert result == expected


@pytest.mark.parametrize(
"kwargs",
[
{"left": 0, "right": 1}, # test basic range
{"left": 0, "right": 1, "props": "background-color: yellow"}, # test props
{"left": -100, "right": 100, "subset": IndexSlice[[0, 1], :]}, # test subset
{"left": 0, "subset": IndexSlice[[0, 1], :]}, # test no right
{"right": 1}, # test no left
{"left": [0, 0, 11], "axis": 0}, # test left as sequence
{"left": DataFrame({"A": [0, 0, 11], "B": [1, 1, 11]}), "axis": None}, # axis
{"left": 0, "right": [0, 1], "axis": 1}, # test sequence right
],
)
def test_highlight_between(styler, kwargs):
expected = {
(0, 0): [("background-color", "yellow")],
(0, 1): [("background-color", "yellow")],
}
result = styler.highlight_between(**kwargs)._compute().ctx
assert result == expected


@pytest.mark.parametrize(
"arg, map, axis",
[
("left", [1, 2], 0), # 0 axis has 3 elements not 2
("left", [1, 2, 3], 1), # 1 axis has 2 elements not 3
("left", np.array([[1, 2], [1, 2]]), None), # df is (2,3) not (2,2)
("right", [1, 2], 0), # same tests as above for 'right' not 'left'
("right", [1, 2, 3], 1), # ..
("right", np.array([[1, 2], [1, 2]]), None), # ..
],
)
def test_highlight_between_raises(arg, styler, map, axis):
msg = f"supplied '{arg}' is not correct shape"
with pytest.raises(ValueError, match=msg):
styler.highlight_between(**{arg: map, "axis": axis})._compute()


def test_highlight_between_raises2(styler):
msg = "values can be 'both', 'left', 'right', or 'neither'"
with pytest.raises(ValueError, match=msg):
styler.highlight_between(inclusive="badstring")._compute()

with pytest.raises(ValueError, match=msg):
styler.highlight_between(inclusive=1)._compute()


@pytest.mark.parametrize(
"inclusive, expected",
[
(
"both",
{
(0, 0): [("background-color", "yellow")],
(0, 1): [("background-color", "yellow")],
},
),
("neither", {}),
("left", {(0, 0): [("background-color", "yellow")]}),
("right", {(0, 1): [("background-color", "yellow")]}),
],
)
def test_highlight_between_inclusive(styler, inclusive, expected):
kwargs = {"left": 0, "right": 1, "subset": IndexSlice[[0, 1], :]}
result = styler.highlight_between(**kwargs, inclusive=inclusive)._compute()
assert result.ctx == expected