Pattern matching callbacks & ServersideOutput

[jonasvdd]

Hi, firstly, amazing work with these dash add-ons. 🚀

I'm one of the core developers of plotly-resampler, a library that performs back-end data-aggregation to still retain snappy and interactive plotly figures withholding large amounts of data. At the request of some end-users (predict-idlab/plotly-resampler#91, predict-idlab/plotly-resampler#56, predict-idlab/plotly-resampler#27) I'm looking into the serialization capabilities of our plotly-resampler figures for dash apps, so there is no need for a global variable in which we store our figures.

As for now I was able to get a minimal working example where I cache the FigureResampler, i.e., the plotly-resampler go.Figure thanks to your amazing ServerSideOutput component. See example ⬇️

import dash
import dash_bootstrap_components as dbc
import numpy as np
import plotly.graph_objects as go
from dash import Input, Output, State, dcc, html
from dash_extensions.enrich import (
    DashProxy,
    ServersideOutput,
    ServersideOutputTransform,
)
from plotly_resampler import FigureResampler
from trace_updater import TraceUpdater

# Some data that will be used for the plotly-resampler figures
x = np.arange(1_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

# Globals
app = DashProxy(
    __name__,
    external_stylesheets=[dbc.themes.LUX],
    transforms=[ServersideOutputTransform()],
)

app.layout = html.Div(
    [
        dbc.Container(html.H1("plotly-resamper test"), style={"textAlign": "center"}),
        html.Button("plot chart", id="plot-button", n_clicks=0),
        html.Hr(),
        # The graph and it's needed components to serialize and update efficiently
        dcc.Graph(id="graph-id"),
        dcc.Loading(dcc.Store(id="store")),
        TraceUpdater(id="trace-updater", gdID="graph-id"),
    ]
)


# The callback used for dynamically updating the graph its front-end view based on
# the end-users view-of-interest (i.e. its zoom ranges)
@app.callback(
    Output("trace-updater", "updateData"),
    Input("graph-id", "relayoutData"),
    State("store", "data"),
    prevent_initial_call=True,
)
def update_fig(relayoutdata, fig):
    if fig is None:
        raise dash.exceptions.PreventUpdate()
    return fig.construct_update_data(relayoutdata)


# The callback used to construct and store the graph's data on the serverside
@app.callback(
    Output("graph-id", "figure"),
    ServersideOutput("store", "data"),
    Input("plot-button", "n_clicks"),
    prevent_initial_call=True,
    memoize=True,
)
def plot_graph(n_clicks):
    ctx = dash.callback_context
    if len(ctx.triggered) and "plot-button" in ctx.triggered[0]["prop_id"]:
        fig: FigureResampler = FigureResampler(go.Figure())
        fig.add_trace(go.Scattergl(name="sin1"), hf_x=x, hf_y=noisy_sin)
        fig.add_trace(go.Scattergl(name="sin2"), hf_x=x, hf_y=noisy_sin * 0.999999**x)
        return fig, fig
    else:
        raise dash.exceptions.PreventUpdate()


if __name__ == "__main__":
    app.run_server(debug=True, port=9023)

As I want to take the next step towards dynamically creating / removing graphs (e.g., instead of the plot-graph button, we use an add-graph button to render graph on some state fields that the user entered), I would love to also create a working example where we use your ServersideOutput, and this Is where I'm stuck now.

My first try to create such an example would withhold this method (see first method in snippet ⬇️), in which we dynamically create a dcc.Store and the figure variable, which we want to (1) store on the server and (2) link to the dynamically created dcc.Store. This would enable to use pattern-matching callbacks for performing graph-updates (see second method in snippet ⬇️).

@app.callback(
    Output("container", "children"),
    Input("add-chart", "n_clicks"),
    State("container", "children"),
    prevent_initial_call=True,
)
def append_graph(n_clicks: int, div_children: List[html.Div]) -> List[html.Div]:
    """
    This function is called when the button is clicked. It adds a new graph to the div.
    """
    # This is the graph which we want to store via the ServersideOutput
    figure = FigureResampler(go.Figure(go.Scattergl(x=x, y=noisy_sin)))

    # Create a uuid for the graph and add it to the global graph dict,
    uid = str(uuid4())
    new_child = html.Div(
        children=[
            dcc.Graph(id={"type": "dynamic-graph", "index": uid}, figure=figure),
            TraceUpdater(id={"type": "dynamic-updater", "index": uid}, gdID=uid),
            dcc.Store(id={"type": "store", "index": uid}),
            # HOW do i link a dynamically created dcc.Store to a ServerSideStore callback?
            # Should I extract this dcc.store to an external variable and apply some 
            # manual code to it?
        ],
    )
    div_children.append(new_child)
    return div_children

@app.callback(
    Output({"type": "dynamic-updater", "index": MATCH}, "updateData"),
    Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    State({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
)
def update_pattern_match_figure(relayoutdata: dict, graph: dict):
    if graph is not None and relayoutdata is not None:
        return graph.construct_update_data(relayoutdata)

So my question is: How can I use your ServerSideOutput on dynamically created dcc.store objects (via triggering / callbacks or within a method, without using the callback logic itself)? In your documentation, I did not immediately find something on pattern-matching callbacks, which would be highly useful for the more advanced users of your toolkit. I hope this somehow makes sense!

Thanks again for your amazing work! 💪🏼
Cheers, Jonas

[prokie]

Hi @jonasvdd, I got it working like this.

"""
Minimal dynamic dash app example.
"""
import numpy as np
import plotly.graph_objects as go
import trace_updater
from dash.exceptions import PreventUpdate
from dash_extensions.enrich import (
    MATCH,
    DashProxy,
    Input,
    Output,
    ServersideOutput,
    ServersideOutputTransform,
    State,
    dcc,
    html,
    TriggerTransform,
    Trigger,
)
from dash import callback_context
from plotly_resampler import FigureResampler

x = np.arange(1_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

app = DashProxy(
    __name__,
    transforms=[
        ServersideOutputTransform(),
        TriggerTransform(),
    ],
)


app.layout = html.Div(
    [
        html.Div(
            children=[
                html.Button("Add Chart", id="add-chart", n_clicks=0),
            ]
        ),
        html.Div(id="container", children=[]),
    ]
)


@app.callback(
    Output("container", "children"),
    Input("add-chart", "n_clicks"),
    State("container", "children"),
    prevent_initial_call=True,
)
def add_graph(n_clicks: int, div_children: list[html.Div]):
    new_child = html.Div(
        children=[
            dcc.Graph(id={"type": "dynamic-graph", "index": n_clicks}),
            trace_updater.TraceUpdater(
                id={"type": "dynamic-updater", "index": n_clicks},
                gdID=f"{n_clicks}",
            ),
            dcc.Store(id={"type": "store", "index": n_clicks}),
            dcc.Interval(
                id={"type": "interval", "index": n_clicks}, max_intervals=1, interval=1
            ),
        ],
    )
    div_children.append(new_child)
    return div_children


@app.callback(
    ServersideOutput({"type": "store", "index": MATCH}, "data"),
    Trigger({"type": "interval", "index": MATCH}, "n_intervals"),
    prevent_initial_call=True,
)
def display_graphs() -> list[html.Div]:
    index = callback_context.triggered_id["index"]
    figure = FigureResampler(go.Figure(go.Scatter(x=x, y=noisy_sin)))

    figure.register_update_graph_callback(
        app=app,
        graph_id={"type": "dynamic-graph", "index": index},
        trace_updater_id={"type": "dynamic-updater", "index": index},
    )

    return figure


@app.callback(
    Output({"type": "dynamic-graph", "index": MATCH}, "figure"),
    Input({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
)
def update_figure(fig):
    return fig


@app.callback(
    Output({"type": "dynamic-updater", "index": MATCH}, "updateData"),
    Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    State({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
)
def update_figure_A(relayoutdata: dict, fig: FigureResampler):
    if fig is not None:
        return fig.construct_update_data(relayoutdata)
    raise PreventUpdate


if __name__ == "__main__":
    app.run_server(debug=True)

[jonasvdd]

Hi @prokie, indeed that fixes it! I further cleaned up your example and added some comments, which can be found here