Introduce SPORES in v0.7.0 as a generalisable mode

docs/advanced/mode.md

@@ -65,9 +65,6 @@ For this reason, `horizon` must always be equal to or larger than `window`.
 
 ## SPORES mode
 
-!!! warning
-    SPORES mode has not yet been re-implemented in Calliope v0.7.
-
 `SPORES` refers to Spatially-explicit Practically Optimal REsultS.
 This run mode allows a user to generate any number of alternative results which are within a certain range of the optimal cost.
 It follows on from previous work in the field of `modelling to generate alternatives` (MGA), with a particular emphasis on alternatives that vary maximally in the spatial dimension.
@@ -77,18 +74,10 @@ As an example, if you wanted to generate 10 SPORES, all of which are within 10%
 
 ```yaml
 config.build.mode: spores
-config.solve:
-    # The number of SPORES to generate:
-    spores_number: 10
-    # The cost class to optimise against when generating SPORES:
-    spores_score_cost_class: spores_score
-    # The initial system cost to limit the SPORES to fit within:
-    spores_cost_max: .inf
-    # The cost class to constrain to be less than or equal to `spores_cost_max`:
-    spores_slack_cost_group: monetary
-parameters:
-    # The fraction above the cost-optimal cost to set the maximum cost during SPORES:
-    slack: 0.1
+# The number of SPORES to generate:
+config.solve.spores.number: 10:
+# The fraction above the cost-optimal cost to set the maximum cost during SPORES:
+parameters.spores_slack: 0.1
 ```
 
 You will now also need a `spores_score` cost class in your model.

docs/examples/loading_tabular_data.py

@@ -1,7 +1,6 @@
 # ---
 # jupyter:
 #   jupytext:
-#     custom_cell_magics: kql
 #     text_representation:
 #       extension: .py
 #       format_name: percent

docs/examples/modes.py

@@ -0,0 +1,190 @@
+# ---
+# jupyter:
+#   jupytext:
+#     text_representation:
+#       extension: .py
+#       format_name: percent
+#       format_version: '1.3'
+#       jupytext_version: 1.16.4
+#   kernelspec:
+#     display_name: calliope_docs_build
+#     language: python
+#     name: calliope_docs_build
+# ---
+
+# %% [markdown]
+# # Running models in different modes
+#
+# Models can be built and solved in different modes:
+
+# - `plan` mode.
+# In `plan` mode, the user defines upper and lower boundaries for technology capacities and the model decides on an optimal system configuration.
+# In this configuration, the total cost of investing in technologies and then using them to meet demand in every _timestep_ (e.g., every hour) is as low as possible.
+# - `operate` mode.
+# In `operate` mode, all capacity constraints are fixed and the system is operated with a receding horizon control algorithm.
+# This is sometimes known as a `dispatch` model - we're only concerned with the _dispatch_ of technologies whose capacities are already fixed.
+# Optimisation is limited to a time horizon which
+# - `spores` mode.
+# `SPORES` refers to Spatially-explicit Practically Optimal REsultS.
+# This run mode allows a user to generate any number of alternative results which are within a certain range of the optimal cost.
+
+# In this notebook we will run the Calliope national scale example model in these three modes.
+
+# More detail on these modes is given in the [_advanced_ section of the Calliope documentation](https://calliope.readthedocs.io/en/latest/advanced/mode/).
+
+# %%
+
+import plotly.express as px
+import plotly.graph_objects as go
+import xarray as xr
+
+import calliope
+
+# We update logging to show a bit more information but to hide the solver output, which can be long.
+calliope.set_log_verbosity("INFO", include_solver_output=False)
+
+# %% [markdown]
+# ## Running in `plan` mode.
+
+# %%
+# We subset to the same time range as operate mode
+model_plan = calliope.examples.national_scale(time_subset=["2005-01-01", "2005-01-10"])
+model_plan.build()
+model_plan.solve()
+
+# %% [markdown]
+# ## Running in `operate` mode.
+
+# %%
+model_operate = calliope.examples.national_scale(scenario="operate")
+model_operate.build()
+model_operate.solve()
+
+# %% [markdown]
+# Note how we have capacity variables as parameters in the inputs and only dispatch variables in the results
+
+# %%
+model_operate.inputs[["flow_cap", "storage_cap", "area_use"]]
+
+# %%
+model_operate.results
+
+# %% [markdown]
+# ## Running in `spores` mode.
+
+# %%
+# We subset to the same time range as operate/plan mode
+model_spores = calliope.examples.national_scale(
+    scenario="spores", time_subset=["2005-01-01", "2005-01-10"]
+)
+model_spores.build()
+model_spores.solve()
+
+# %% [markdown]
+# Note how we have a new `spores` dimension in our results.
+
+# %%
+model_spores.results
+
+# %% [markdown]
+# We can track the SPORES scores used between iterations using the `spores_score_cumulative` result.
+# This scoring mechanism is based on increasing the score of any technology-node combination where the
+
+# %%
+# We do some prettification of the outputs
+model_spores.results.spores_score_cumulative.to_series().where(
+    lambda x: x > 0
+).dropna().unstack("spores")
+
+# %% [markdown]
+# ## Visualising results
+#
+# We can use [plotly](https://plotly.com/) to quickly examine our results.
+# These are just some examples of how to visualise Calliope data.
+
+# %%
+# We set the color mapping to use in all our plots by extracting the colors defined in the technology definitions of our model.
+# We also create some reusable plotting functions.
+colors = model_plan.inputs.color.to_series().to_dict()
+
+
+def plot_flows(results: xr.Dataset) -> go.Figure:
+    df_electricity = (
+        (results.flow_out.fillna(0) - results.flow_in.fillna(0))
+        .sel(carriers="power")
+        .sum("nodes")
+        .to_series()
+        .where(lambda x: x != 0)
+        .dropna()
+        .to_frame("Flow in/out (kWh)")
+        .reset_index()
+    )
+    df_electricity_demand = df_electricity[df_electricity.techs == "demand_power"]
+    df_electricity_other = df_electricity[df_electricity.techs != "demand_power"]
+
+    fig = px.bar(
+        df_electricity_other,
+        x="timesteps",
+        y="Flow in/out (kWh)",
+        color="techs",
+        color_discrete_map=colors,
+    )
+    fig.add_scatter(
+        x=df_electricity_demand.timesteps,
+        y=-1 * df_electricity_demand["Flow in/out (kWh)"],
+        marker_color="black",
+        name="demand",
+    )
+    return fig
+
+
+def plot_capacity(results: xr.Dataset, **plotly_kwargs) -> go.Figure:
+    df_capacity = (
+        results.flow_cap.where(results.techs != "demand_power")
+        .sel(carriers="power")
+        .to_series()
+        .where(lambda x: x != 0)
+        .dropna()
+        .to_frame("Flow capacity (kW)")
+        .reset_index()
+    )
+
+    fig = px.bar(
+        df_capacity,
+        x="nodes",
+        y="Flow capacity (kW)",
+        color="techs",
+        color_discrete_map=colors,
+        **plotly_kwargs,
+    )
+    return fig
+
+
+# %% [markdown]
+# ## `plan` vs `operate`
+# Here, we compare flows over the 10 days.
+# Note how flows do not match as the rolling horizon makes it difficult to make the correct storage charge/discharge decisions.
+
+# %%
+fig_flows_plan = plot_flows(
+    model_plan.results.sel(timesteps=model_operate.results.timesteps)
+)
+fig_flows_plan.update_layout(title="Plan mode flows")
+
+
+# %%
+fig_flows_operate = plot_flows(model_operate.results)
+fig_flows_operate.update_layout(title="Operate mode flows")
+
+# %% [markdown]
+# ## `plan` vs `spores`
+# Here, we compare installed capacities between the baseline run (== `plan` mode) and the SPORES.
+# Note how the baseline SPORE is the same as `plan` mode and then results deviate considerably.
+
+# %%
+fig_flows_plan = plot_capacity(model_plan.results)
+fig_flows_plan.update_layout(title="Plan mode capacities")
+
+# %%
+fig_flows_spores = plot_capacity(model_spores.results, facet_col="spores")
+fig_flows_spores.update_layout(title="SPORES mode capacities")

docs/examples/national_scale/notebook.py

@@ -109,7 +109,7 @@
 
 # %% [markdown]
 # #### Plotting flows
-# We do this by combinging in- and out-flows and separating demand from other technologies.
+# We do this by combining in- and out-flows and separating demand from other technologies.
 # First, we look at the aggregated result across all nodes, then we look at each node separately.
 
 # %%

docs/hooks/dummy_model/model.yaml

@@ -5,12 +5,19 @@ overrides:
       time_cluster: cluster_days.csv
     config.build:
       add_math: ["storage_inter_cluster"]
+  spores:
+    config:
+      init.name: SPORES solve mode
+      build.mode: spores
+      solve.spores.number: 2
+    parameters:
+      spores_slack: 0.1
 
 config.init.name: base
 
 nodes:
-  A.techs: {demand_tech, conversion_tech, supply_tech, storage_tech}
-  B.techs: {demand_tech, conversion_tech, supply_tech, storage_tech}
+  A.techs: { demand_tech, conversion_tech, supply_tech, storage_tech }
+  B.techs: { demand_tech, conversion_tech, supply_tech, storage_tech }
 
 techs:
   tech_transmission:

docs/hooks/generate_math_docs.py

@@ -67,7 +67,7 @@ def on_files(files: list, config: dict, **kwargs):
             f"{override}.yaml",
             textwrap.dedent(
                 f"""
-            Pre-defined additional math to apply {custom_documentation.name} math on top of the [base mathematical formulation][base-math].
+            Pre-defined additional math to apply {custom_documentation.name} __on top of__ the [base mathematical formulation][base-math].
             This math is _only_ applied if referenced in the `config.init.add_math` list as `{override}`.
             """
             ),

mkdocs.yml

@@ -132,6 +132,7 @@ nav:
       - examples/milp/index.md
       - examples/milp/notebook.py
     - examples/loading_tabular_data.py
+    - examples/modes.py
     - examples/piecewise_constraints.py
     - examples/calliope_model_object.py
     - examples/calliope_logging.py

src/calliope/backend/backend_model.py

@@ -57,6 +57,7 @@ class BackendModelGenerator(ABC):
         "default",
         "type",
         "title",
+        "sense",
         "math_repr",
         "original_dtype",
     ]
@@ -65,6 +66,8 @@ class BackendModelGenerator(ABC):
     _PARAM_DESCRIPTIONS = extract_from_schema(MODEL_SCHEMA, "description")
     _PARAM_UNITS = extract_from_schema(MODEL_SCHEMA, "x-unit")
     _PARAM_TYPE = extract_from_schema(MODEL_SCHEMA, "x-type")
+    objective: str
+    """Optimisation problem objective name."""
 
     def __init__(
         self, inputs: xr.Dataset, math: CalliopeMath, build_config: config_schema.Build
@@ -170,6 +173,14 @@ def add_objective(
             objective_dict (parsing.UnparsedObjective): Unparsed objective configuration dictionary.
         """
 
+    @abstractmethod
+    def set_objective(self, name: str) -> None:
+        """Set a built objective to be the optimisation objective.
+
+        Args:
+            name (str): name of the objective.
+        """
+
     def log(
         self,
         component_type: ALL_COMPONENTS_T,
@@ -928,13 +939,7 @@ def has_integer_or_binary_variables(self) -> bool:
 
     @abstractmethod
     def _solve(
-        self,
-        solver: str,
-        solver_io: str | None = None,
-        solver_options: dict | None = None,
-        save_logs: str | None = None,
-        warmstart: bool = False,
-        **solve_config,
+        self, solve_config: config_schema.Solve, warmstart: bool = False
     ) -> xr.Dataset:
         """Optimise built model.
 
@@ -943,17 +948,10 @@ def _solve(
         values at optimality.
 
         Args:
-            solver (str): Name of solver to optimise with.
-            solver_io (str | None, optional): If chosen solver has a python interface, set to "python" for potential
-                performance gains, otherwise should be left as None. Defaults to None.
-            solver_options (dict | None, optional): Solver options/parameters to pass directly to solver.
-                See solver documentation for available parameters that can be influenced. Defaults to None.
-            save_logs (str | None, optional): If given, solver logs and built LP file will be saved to this filepath.
-                Defaults to None.
+            solve_config: (config_schema.Solve): Calliope Solve configuration object.
             warmstart (bool, optional): If True, and the chosen solver is capable of implementing it, an existing
                 optimal solution will be used to warmstart the next solve run.
                 Defaults to False.
-            **solve_config: solve configuration overrides.
 
         Returns:
             xr.Dataset: Dataset of decision variable values if the solution was optimal/feasible,
@@ -1144,7 +1142,7 @@ def track_constraints(self, constraints_to_track: list):
         valid_constraints = shadow_prices.intersection(self.available_constraints)
         if invalid_constraints:
             model_warn(
-                f"Invalid constraints {invalid_constraints} in `config.solve.shadow_prices`. "
+                f"Invalid constraints {invalid_constraints} in `config_schema.solve.shadow_prices`. "
                 "Their shadow prices will not be tracked."
             )
         # Only actually activate shadow price tracking if at least one valid