From 0379dcae0a3c1c4fd99fe5862763a540f4e79858 Mon Sep 17 00:00:00 2001 From: odow Date: Mon, 20 Jan 2025 12:38:03 +1300 Subject: [PATCH 1/5] Update --- docs/make.jl | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/make.jl b/docs/make.jl index 0d8a72dbed8..bd378cad048 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -39,6 +39,7 @@ const _HAS_GUROBI = try catch false end +@show _HAS_GUROBI const _GUROBI_EXCLUDES = String[] if !_HAS_GUROBI From 5bd4fe006dbd2f092af02ba6ae55d93aea72caaa Mon Sep 17 00:00:00 2001 From: odow Date: Mon, 20 Jan 2025 15:37:54 +1300 Subject: [PATCH 2/5] Add markdown files --- .../algorithms/benders_decomposition.md | 566 ++++++++++++++++++ .../algorithms/tsp_lazy_constraints.md | 296 +++++++++ docs/src/tutorials/linear/callbacks.md | 220 +++++++ .../tutorials/linear/multiple_solutions.md | 175 ++++++ 4 files changed, 1257 insertions(+) create mode 100644 docs/src/tutorials/algorithms/benders_decomposition.md create mode 100644 docs/src/tutorials/algorithms/tsp_lazy_constraints.md create mode 100644 docs/src/tutorials/linear/callbacks.md create mode 100644 docs/src/tutorials/linear/multiple_solutions.md diff --git a/docs/src/tutorials/algorithms/benders_decomposition.md b/docs/src/tutorials/algorithms/benders_decomposition.md new file mode 100644 index 00000000000..1d77cac9746 --- /dev/null +++ b/docs/src/tutorials/algorithms/benders_decomposition.md @@ -0,0 +1,566 @@ +```@meta +EditURL = "benders_decomposition.jl" +``` + +# [Benders decomposition](@id benders_decomposition_classical) + +_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ +[_Download the source as a `.jl` file_](benders_decomposition.jl). + +**This tutorial was originally contributed by Shuvomoy Das Gupta.** + +This tutorial describes how to implement [Benders decomposition](https://en.wikipedia.org/wiki/Benders_decomposition) +in JuMP. It uses the following packages: + +````@example benders_decomposition +using JuMP +import Gurobi +import HiGHS +import Printf +```` + +## Theory + +Benders decomposition is a useful algorithm for solving convex optimization +problems with a large number of variables. It works best when a larger problem +can be decomposed into two (or more) smaller problems that are individually +much easier to solve. + +This tutorial demonstrates Benders decomposition on the following +mixed-integer linear program: +```math +\begin{aligned} +\text{min} \ & c_1(x) + c_2(y) \\ +\text{subject to} \ & f_1(x) \in S_1 \\ + & f_2(y) \in S_2 \\ + & f_3(x, y) \in S_3 \\ + & x \in \mathbb{Z}^m \\ + & y \in \mathbb{R}^n \\ +\end{aligned} +``` +where the functions $f$ and $c$ are linear, and the sets $S$ are inequality +sets like $\ge l$, $\le u$, or $= b$. + +Any mixed integer programming problem can be written in the form above. + +If there are relatively few integer variables, and many more continuous +variables, then it may be beneficial to decompose the problem into a small +problem containing only integer variables and a linear program containing +only continuous variables. Hopefully, the linear program will be much easier +to solve in isolation than in the full mixed-integer linear program. + +For example, if we knew a feasible solution for ``\bar{x}``, we could obtain a +solution for ``y`` by solving: +```math +\begin{aligned} +V_2(\bar{x}) = & \text{min} \ & c_2(y)\\ + & \text{subject to} \ & f_2(y) \in S_2 \\ + & & f_3(x, y) \in S_3 \\ + & & x = \bar{x} & \ [\pi] \\ + & & y \in \mathbb{R}^n \\ +\end{aligned} +``` +Note that we have included a "copy" of the `x` variable to simplify computing +$\pi$, which is the dual of $V_2$ with respect to $\bar{x}$. + +Because this model is a linear program, it is easy to solve. + +Replacing the ``c_2(y)`` component of the objective in our original problem +with ``V_2`` yields: +```math +\begin{aligned} +V_1 = & \text{min} \ & c_1(x) + V_2(x) \\ +& \text{subject to} \ & f_1(x) \in S_1 \\ +& & x \in \mathbb{Z}^m. +\end{aligned} +``` +This problem looks a lot simpler to solve because it involves only $x$ and a +subset of the constraints, but we need to do something else with ``V_2`` first. + +Because ``\bar{x}`` is a constant that appears on the right-hand side of the +constraints, ``V_2`` is a convex function with respect to ``\bar{x}``, and the +dual variable ``\pi`` is a subgradient of ``V_2(x)`` with respect to ``x``. +Therefore, if we have a candidate solution ``x_k``, then we can solve +``V_2(x_k)`` and obtain a feasible dual vector ``\pi_k``. Using these values, +we can construct a first-order Taylor-series approximation of ``V_2`` about +the point ``x_k``: +```math +V_2(x) \ge V_2(x_k) + \pi_k^\top (x - x_k). +``` +By convexity, we know that this inequality holds for all ``x``, and we call +these inequalities _cuts_. + +Benders decomposition is an iterative technique that replaces ``V_2(x)`` with +a new decision variable ``\theta``, and approximates it from below using cuts: +```math +\begin{aligned} +V_1^K = & \text{min} \ & c_1(x) + \theta \\ + & \text{subject to} \ & f_1(x) \in S_1 \\ + & & x \in \mathbb{Z}^m \\ + & & \theta \ge M \\ + & & \theta \ge V_2(x_k) + \pi_k^\top(x - x_k) & \quad \forall k = 1,\ldots,K. +\end{aligned} +``` +This integer program is called the _first-stage_ subproblem. + +To generate cuts, we solve ``V_1^K`` to obtain a candidate first-stage +solution ``x_k``, then we use that solution to solve ``V_2(x_k)``. Then, using +the optimal objective value and dual solution from ``V_2``, we add a new cut +to form ``V_1^{K+1}`` and repeat. + +### Bounds + +Due to convexity, we know that ``V_2(x) \ge \theta`` for all ``x``. Therefore, +the optimal objective value of ``V_1^K`` provides a valid _lower_ bound on the +objective value of the full problem. In addition, if we take a feasible +solution for ``x`` from the first-stage problem, then ``c_1(x) + V_2(x)`` +is a valid _upper_ bound on the objective value of the full problem. + +Benders decomposition uses the lower and upper bounds to determine when it has +found the global optimal solution. + +## Monolithic problem + +As an example problem, we consider the following variant of +[The max-flow problem](@ref), in which there is a binary variable to decide +whether to open each arc for a cost of 0.1 unit, and we can open at most 11 +arcs: + +````@example benders_decomposition +G = [ + 0 3 2 2 0 0 0 0 + 0 0 0 0 5 1 0 0 + 0 0 0 0 1 3 1 0 + 0 0 0 0 0 1 0 0 + 0 0 0 0 0 0 0 4 + 0 0 0 0 0 0 0 2 + 0 0 0 0 0 0 0 4 + 0 0 0 0 0 0 0 0 +] +n = size(G, 1) +model = Model(HiGHS.Optimizer) +set_silent(model) +@variable(model, x[1:n, 1:n], Bin) +@variable(model, y[1:n, 1:n] >= 0) +@constraint(model, sum(x) <= 11) +@constraint(model, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x[i, j]) +@constraint(model, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) +@objective(model, Min, 0.1 * sum(x) - sum(y[1, :])) +optimize!(model) +solution_summary(model) +```` + +The optimal objective value is -5.1: + +````@example benders_decomposition +objective_value(model) +```` + +and the optimal flows are: + +````@example benders_decomposition +function optimal_flows(x) + return [(i, j) => x[i, j] for i in 1:n for j in 1:n if x[i, j] > 0] +end + +monolithic_solution = optimal_flows(value.(y)) +```` + +## [Iterative method](@id benders_iterative) + +!!! warning + This is a basic implementation for pedagogical purposes. We haven't + discussed any of the computational tricks that are required to build a + performant implementation for large-scale problems. See [In-place iterative method](@ref) + for one improvement that helps computation time. + +We start by formulating the first-stage subproblem. It includes the `x` +variables, and the constraints involving only `x`, and the terms in the +objective containing only `x`. We also need an initial lower bound on the +cost-to-go variable `θ`. One valid lower bound is to assume that we do not pay +for opening arcs, and there is flow all the arcs. + +````@example benders_decomposition +M = -sum(G) +model = Model(HiGHS.Optimizer) +set_silent(model) +@variable(model, x[1:n, 1:n], Bin) +@variable(model, θ >= M) +@constraint(model, sum(x) <= 11) +@objective(model, Min, 0.1 * sum(x) + θ) +model +```` + +For the next step, we need a function that takes a first-stage candidate +solution `x` and returns the optimal solution from the second-stage +subproblem: + +````@example benders_decomposition +function solve_subproblem(x_bar) + model = Model(HiGHS.Optimizer) + set_silent(model) + @variable(model, x[i in 1:n, j in 1:n] == x_bar[i, j]) + @variable(model, y[1:n, 1:n] >= 0) + @constraint(model, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x[i, j]) + @constraint(model, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) + @objective(model, Min, -sum(y[1, :])) + optimize!(model) + @assert is_solved_and_feasible(model; dual = true) + return (obj = objective_value(model), y = value.(y), π = reduced_cost.(x)) +end +```` + +Note that `solve_subproblem` returns a `NamedTuple` of the objective value, +the optimal primal solution for `y`, and the optimal dual solution for `π`, +which we obtained from the [`reduced_cost`](@ref) of the `x` variables. + +We're almost ready for our optimization loop, but first, here's a helpful +function for logging: + +````@example benders_decomposition +function print_iteration(k, args...) + f(x) = Printf.@sprintf("%12.4e", x) + println(lpad(k, 9), " ", join(f.(args), " ")) + return +end +```` + +We also need to put a limit on the number of iterations before termination: + +````@example benders_decomposition +MAXIMUM_ITERATIONS = 100 +```` + +And a way to check if the lower and upper bounds are close-enough to +terminate: + +````@example benders_decomposition +ABSOLUTE_OPTIMALITY_GAP = 1e-6 +```` + +Now we're ready to iterate Benders decomposition: + +````@example benders_decomposition +println("Iteration Lower Bound Upper Bound Gap") +for k in 1:MAXIMUM_ITERATIONS + optimize!(model) + @assert is_solved_and_feasible(model) + lower_bound = objective_value(model) + x_k = value.(x) + ret = solve_subproblem(x_k) + upper_bound = (objective_value(model) - value(θ)) + ret.obj + gap = abs(upper_bound - lower_bound) / abs(upper_bound) + print_iteration(k, lower_bound, upper_bound, gap) + if gap < ABSOLUTE_OPTIMALITY_GAP + println("Terminating with the optimal solution") + break + end + cut = @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) + @info "Adding the cut $(cut)" +end +```` + +Finally, we can obtain the optimal solution: + +````@example benders_decomposition +optimize!(model) +@assert is_solved_and_feasible(model) +x_optimal = value.(x) +optimal_ret = solve_subproblem(x_optimal) +iterative_solution = optimal_flows(optimal_ret.y) +```` + +which is the same as the monolithic solution: + +````@example benders_decomposition +iterative_solution == monolithic_solution +```` + +and it has the same objective value: + +````@example benders_decomposition +objective_value(model) +```` + +## Callback method + +The [Iterative method](@ref benders_iterative) section implemented Benders +decomposition using a loop. In each iteration, we re-solved the first-stage +subproblem to generate a candidate solution. However, modern MILP solvers such +as CPLEX, Gurobi, and GLPK provide lazy constraint callbacks which allow us to +add new cuts _while the solver is running_. This can be more efficient than an +iterative method because we can avoid repeating work such as solving the root +node of the first-stage MILP at each iteration. + +!!! tip + We use Gurobi for this model because HiGHS does not support lazy + constraints. For more information on callbacks, read the page + [Solver-independent callbacks](@ref callbacks_manual). + +As before, we construct the same first-stage subproblem: + +````@example benders_decomposition +lazy_model = Model(Gurobi.Optimizer) +set_silent(lazy_model) +@variable(lazy_model, x[1:n, 1:n], Bin) +@variable(lazy_model, θ >= M) +@constraint(lazy_model, sum(x) <= 11) +@objective(lazy_model, Min, 0.1 * sum(x) + θ) +lazy_model +```` + +What differs is that we write a callback function instead of a loop: + +````@example benders_decomposition +number_of_subproblem_solves = 0 +function my_callback(cb_data) + status = callback_node_status(cb_data, lazy_model) + if status != MOI.CALLBACK_NODE_STATUS_INTEGER + # Only add the constraint if `x` is an integer feasible solution + return + end + x_k = callback_value.(cb_data, x) + θ_k = callback_value(cb_data, θ) + global number_of_subproblem_solves += 1 + ret = solve_subproblem(x_k) + if θ_k < (ret.obj - 1e-6) + # Only add the constraint if θ_k violates the constraint + cut = @build_constraint(θ >= ret.obj + sum(ret.π .* (x .- x_k))) + MOI.submit(lazy_model, MOI.LazyConstraint(cb_data), cut) + end + return +end + +set_attribute(lazy_model, MOI.LazyConstraintCallback(), my_callback) +```` + +Now when we optimize!, our callback is run: + +````@example benders_decomposition +optimize!(lazy_model) +@assert is_solved_and_feasible(lazy_model) +```` + +For this model, the callback algorithm required more solves of the subproblem: + +````@example benders_decomposition +number_of_subproblem_solves +```` + +But for larger problems, you can expect the callback algorithm to be more +efficient than the iterative algorithm. + +Finally, we can obtain the optimal solution: + +````@example benders_decomposition +x_optimal = value.(x) +optimal_ret = solve_subproblem(x_optimal) +callback_solution = optimal_flows(optimal_ret.y) +```` + +which is the same as the monolithic solution: + +````@example benders_decomposition +callback_solution == monolithic_solution +```` + +## In-place iterative method + +Our implementation of the iterative method has a problem: every time we need +to solve the subproblem, we must rebuild it from scratch. This is expensive, +and it can be the bottleneck in the solution process. We can improve our +implementation by using re-using the subproblem between solves. + +First, we create our first-stage problem as usual: + +````@example benders_decomposition +model = Model(HiGHS.Optimizer) +set_silent(model) +@variable(model, x[1:n, 1:n], Bin) +@variable(model, θ >= M) +@constraint(model, sum(x) <= 11) +@objective(model, Min, 0.1 * sum(x) + θ) +model +```` + +Then, instead of building the subproblem in a function, we build it once here: + +````@example benders_decomposition +subproblem = Model(HiGHS.Optimizer) +set_silent(subproblem) +@variable(subproblem, x_copy[i in 1:n, j in 1:n]) +@variable(subproblem, y[1:n, 1:n] >= 0) +@constraint(subproblem, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x_copy[i, j]) +@constraint(subproblem, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) +@objective(subproblem, Min, -sum(y[1, :])) +subproblem +```` + +Our function to solve the subproblem is also slightly different because we +need to fix the value of the `x_copy` variables to the value of `x` from the +first-stage problem: + +````@example benders_decomposition +function solve_subproblem(model, x) + fix.(model[:x_copy], x) + optimize!(model) + @assert is_solved_and_feasible(model; dual = true) + return ( + obj = objective_value(model), + y = value.(model[:y]), + π = reduced_cost.(model[:x_copy]), + ) +end +```` + +Now we're ready to iterate our in-place Benders decomposition: + +````@example benders_decomposition +println("Iteration Lower Bound Upper Bound Gap") +for k in 1:MAXIMUM_ITERATIONS + optimize!(model) + @assert is_solved_and_feasible(model) + lower_bound = objective_value(model) + x_k = value.(x) + ret = solve_subproblem(subproblem, x_k) + upper_bound = (objective_value(model) - value(θ)) + ret.obj + gap = abs(upper_bound - lower_bound) / abs(upper_bound) + print_iteration(k, lower_bound, upper_bound, gap) + if gap < ABSOLUTE_OPTIMALITY_GAP + println("Terminating with the optimal solution") + break + end + cut = @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) + @info "Adding the cut $(cut)" +end +```` + +Finally, we can obtain the optimal solution: + +````@example benders_decomposition +optimize!(model) +@assert is_solved_and_feasible(model) +x_optimal = value.(x) +optimal_ret = solve_subproblem(subproblem, x_optimal) +inplace_solution = optimal_flows(optimal_ret.y) +```` + +which is the same as the monolithic solution: + +````@example benders_decomposition +inplace_solution == monolithic_solution +```` + +## Feasibility cuts + +So far, we have discussed only Benders optimality cuts. However, for some +first-stage values of `x`, the subproblem might be infeasible. The solution is +to add a Benders feasibility cut: +```math +v_k + u_k^\top (x - x_k) \le 0 +``` +where $u_k$ is a dual unbounded ray of the subproblem and $v_k$ is the +intercept of the unbounded ray. + +As a variation of our example which leads to infeasibilities, we add a +constraint that `sum(y) >= 1`. This means we need a choice of first-stage `x` +for which at least one unit can flow. + +The first-stage problem remains the same: + +````@example benders_decomposition +model = Model(HiGHS.Optimizer) +set_silent(model) +@variable(model, x[1:n, 1:n], Bin) +@variable(model, θ >= M) +@constraint(model, sum(x) <= 11) +@objective(model, Min, 0.1 * sum(x) + θ) +model +```` + +But the subproblem has a new constraint that `sum(y) >= 1`: + +````@example benders_decomposition +subproblem = Model(HiGHS.Optimizer) +set_silent(subproblem) +# We need to turn presolve off so that HiGHS will return an infeasibility +# certificate. +set_attribute(subproblem, "presolve", "off") +@variable(subproblem, x_copy[i in 1:n, j in 1:n]) +@variable(subproblem, y[1:n, 1:n] >= 0) +@constraint(subproblem, sum(y) >= 1) # <--- THIS IS NEW +@constraint(subproblem, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x_copy[i, j]) +@constraint(subproblem, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) +@objective(subproblem, Min, -sum(y[1, :])) +subproblem +```` + +The function to solve the subproblem now checks for feasibility, and returns +the dual objective value and an dual unbounded ray if the subproblem is +infeasible: + +````@example benders_decomposition +function solve_subproblem_with_feasibility(model, x) + fix.(model[:x_copy], x) + optimize!(model) + if is_solved_and_feasible(model; dual = true) + return ( + is_feasible = true, + obj = objective_value(model), + y = value.(model[:y]), + π = reduced_cost.(model[:x_copy]), + ) + end + return ( + is_feasible = false, + v = dual_objective_value(model), + u = reduced_cost.(model[:x_copy]), + ) +end +```` + +Now we're ready to iterate our in-place Benders decomposition: + +````@example benders_decomposition +println("Iteration Lower Bound Upper Bound Gap") +for k in 1:MAXIMUM_ITERATIONS + optimize!(model) + @assert is_solved_and_feasible(model) + lower_bound = objective_value(model) + x_k = value.(x) + ret = solve_subproblem_with_feasibility(subproblem, x_k) + if ret.is_feasible + # Benders Optimality Cuts + upper_bound = (objective_value(model) - value(θ)) + ret.obj + gap = abs(upper_bound - lower_bound) / abs(upper_bound) + print_iteration(k, lower_bound, upper_bound, gap) + if gap < ABSOLUTE_OPTIMALITY_GAP + println("Terminating with the optimal solution") + break + end + @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) + else + # Benders Feasibility Cuts + cut = @constraint(model, ret.v + sum(ret.u .* (x .- x_k)) <= 0) + @info "Adding the feasibility cut $(cut)" + end +end +```` + +Finally, we can obtain the optimal solution: + +````@example benders_decomposition +optimize!(model) +@assert is_solved_and_feasible(model) +x_optimal = value.(x) +optimal_ret = solve_subproblem(subproblem, x_optimal) +feasible_inplace_solution = optimal_flows(optimal_ret.y) +```` + +which is the same as the monolithic solution (because `sum(y) >= 1` in the +monolithic solution): + +````@example benders_decomposition +feasible_inplace_solution == monolithic_solution +```` + diff --git a/docs/src/tutorials/algorithms/tsp_lazy_constraints.md b/docs/src/tutorials/algorithms/tsp_lazy_constraints.md new file mode 100644 index 00000000000..ff87a9fc536 --- /dev/null +++ b/docs/src/tutorials/algorithms/tsp_lazy_constraints.md @@ -0,0 +1,296 @@ +```@meta +EditURL = "tsp_lazy_constraints.jl" +``` + +# [Traveling Salesperson Problem](@id tsp_lazy) + +_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ +[_Download the source as a `.jl` file_](tsp_lazy_constraints.jl). + +**This tutorial was originally contributed by Daniel Schermer.** + +This tutorial describes how to implement the +[Traveling Salesperson Problem](https://en.wikipedia.org/wiki/Travelling_salesman_problem) +in JuMP using solver-independent lazy constraints that dynamically separate +subtours. To be more precise, we use lazy constraints to cut off infeasible +subtours only when necessary and not before needed. + +It uses the following packages: + +````@example tsp_lazy_constraints +using JuMP +import Gurobi +import Plots +import Random +import Test +```` + +## [Mathematical Formulation](@id tsp_model) + +Assume that we are given a complete graph $\mathcal{G}(V,E)$ where $V$ is the +set of vertices (or cities) and $E$ is the set of edges (or roads). For each +pair of vertices $i, j \in V, i \neq j$ the edge $(i,j) \in E$ is associated +with a weight (or distance) $d_{ij} \in \mathbb{R}^+$. + +For this tutorial, we assume the problem to be symmetric, that is, +$d_{ij} = d_{ji} \, \forall i,j \in V$. + +In the Traveling Salesperson Problem, we are tasked with finding a tour with +minimal length that visits every vertex exactly once and then returns to the +point of origin, that is, a *Hamiltonian cycle* with minimal weight. + +To model the problem, we introduce a binary variable, +$x_{ij} \in \{0,1\} \; \forall i, j \in V$, that indicates if edge $(i,j)$ is +part of the tour or not. Using these variables, the Traveling Salesperson +Problem can be modeled as the following integer linear program. + +### [Objective Function](@id tsp_objective) + +The objective is to minimize the length of the tour (due to the assumed +symmetry, the second sum only contains $i 1 + pop!(unvisited, current) + end + neighbors = + [j for (i, j) in edges if i == current && j in unvisited] + end + if length(this_cycle) < length(shortest_subtour) + shortest_subtour = this_cycle + end + end + return shortest_subtour +end +```` + +Let us declare a helper function `selected_edges()` that will be repeatedly +used in what follows. + +````@example tsp_lazy_constraints +function selected_edges(x::Matrix{Float64}, n) + return Tuple{Int,Int}[(i, j) for i in 1:n, j in 1:n if x[i, j] > 0.5] +end +```` + +Other helper functions for computing subtours: + +````@example tsp_lazy_constraints +subtour(x::Matrix{Float64}) = subtour(selected_edges(x, size(x, 1)), size(x, 1)) +subtour(x::AbstractMatrix{VariableRef}) = subtour(value.(x)) +```` + +### Iterative method + +An iterative way of eliminating subtours is the following. + +However, it is reasonable to assume that this is not the most efficient way: +whenever a new subtour elimination constraint is added to the model, the +optimization has to start from the very beginning. + +That way, the solver will repeatedly discard useful information encountered +during previous solves (for example, all cuts, the incumbent solution, or lower +bounds). + +!!! info + Note that, in principle, it would also be feasible to add all subtours + (instead of just the shortest one) to the model. However, preventing just + the shortest cycle is often sufficient for breaking other subtours and + will keep the model size smaller. + +````@example tsp_lazy_constraints +iterative_model = build_tsp_model(d, n) +optimize!(iterative_model) +@assert is_solved_and_feasible(iterative_model) +time_iterated = solve_time(iterative_model) +cycle = subtour(iterative_model[:x]) +while 1 < length(cycle) < n + println("Found cycle of length $(length(cycle))") + S = [(i, j) for (i, j) in Iterators.product(cycle, cycle) if i < j] + @constraint( + iterative_model, + sum(iterative_model[:x][i, j] for (i, j) in S) <= length(cycle) - 1, + ) + optimize!(iterative_model) + @assert is_solved_and_feasible(iterative_model) + global time_iterated += solve_time(iterative_model) + global cycle = subtour(iterative_model[:x]) +end +```` + +````@example tsp_lazy_constraints +objective_value(iterative_model) +```` + +````@example tsp_lazy_constraints +time_iterated +```` + +As a quick sanity check, we visualize the optimal tour to verify that no +subtour is present: + +````@example tsp_lazy_constraints +function plot_tour(X, Y, x) + plot = Plots.plot() + for (i, j) in selected_edges(x, size(x, 1)) + Plots.plot!([X[i], X[j]], [Y[i], Y[j]]; legend = false) + end + return plot +end + +plot_tour(X, Y, value.(iterative_model[:x])) +```` + +### Lazy constraint method + +A more sophisticated approach makes use of _lazy constraints_. To be more +precise, we do this through the `subtour_elimination_callback()` below, which +is only run whenever we encounter a new integer-feasible solution. + +````@example tsp_lazy_constraints +lazy_model = build_tsp_model(d, n) +function subtour_elimination_callback(cb_data) + status = callback_node_status(cb_data, lazy_model) + if status != MOI.CALLBACK_NODE_STATUS_INTEGER + return # Only run at integer solutions + end + cycle = subtour(callback_value.(cb_data, lazy_model[:x])) + if !(1 < length(cycle) < n) + return # Only add a constraint if there is a cycle + end + S = [(i, j) for (i, j) in Iterators.product(cycle, cycle) if i < j] + con = @build_constraint( + sum(lazy_model[:x][i, j] for (i, j) in S) <= length(cycle) - 1, + ) + MOI.submit(lazy_model, MOI.LazyConstraint(cb_data), con) + return +end +set_attribute( + lazy_model, + MOI.LazyConstraintCallback(), + subtour_elimination_callback, +) +optimize!(lazy_model) +```` + +````@example tsp_lazy_constraints +@assert is_solved_and_feasible(lazy_model) +objective_value(lazy_model) +```` + +````@example tsp_lazy_constraints +time_lazy = solve_time(lazy_model) +```` + +This finds the same optimal tour: + +````@example tsp_lazy_constraints +plot_tour(X, Y, value.(lazy_model[:x])) +```` + +The solution time is faster than the iterative approach: + +````@example tsp_lazy_constraints +Test.@test time_lazy < time_iterated +```` + diff --git a/docs/src/tutorials/linear/callbacks.md b/docs/src/tutorials/linear/callbacks.md new file mode 100644 index 00000000000..a2818502c27 --- /dev/null +++ b/docs/src/tutorials/linear/callbacks.md @@ -0,0 +1,220 @@ +```@meta +EditURL = "callbacks.jl" +``` + +# [Callbacks](@id callbacks_tutorial) + +_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ +[_Download the source as a `.jl` file_](callbacks.jl). + +The purpose of the tutorial is to demonstrate the various solver-independent +and solver-dependent callbacks that are supported by JuMP. + +The tutorial uses the following packages: + +````@example callbacks +using JuMP +import Gurobi +import Random +import Test +```` + +!!! info + This tutorial uses the [MathOptInterface](@ref moi_documentation) API. + By default, JuMP exports the `MOI` symbol as an alias for the + MathOptInterface.jl package. We recommend making this more explicit in + your code by adding the following lines: + ```julia + import MathOptInterface as MOI + ``` + +## Lazy constraints + +An example using a lazy constraint callback. + +````@example callbacks +function example_lazy_constraint() + model = Model(Gurobi.Optimizer) + set_silent(model) + @variable(model, 0 <= x <= 2.5, Int) + @variable(model, 0 <= y <= 2.5, Int) + @objective(model, Max, y) + lazy_called = false + function my_callback_function(cb_data) + lazy_called = true + x_val = callback_value(cb_data, x) + y_val = callback_value(cb_data, y) + println("Called from (x, y) = ($x_val, $y_val)") + status = callback_node_status(cb_data, model) + if status == MOI.CALLBACK_NODE_STATUS_FRACTIONAL + println(" - Solution is integer infeasible!") + elseif status == MOI.CALLBACK_NODE_STATUS_INTEGER + println(" - Solution is integer feasible!") + else + @assert status == MOI.CALLBACK_NODE_STATUS_UNKNOWN + println(" - I don't know if the solution is integer feasible :(") + end + if y_val - x_val > 1 + 1e-6 + con = @build_constraint(y - x <= 1) + println("Adding $(con)") + MOI.submit(model, MOI.LazyConstraint(cb_data), con) + elseif y_val + x_val > 3 + 1e-6 + con = @build_constraint(y + x <= 3) + println("Adding $(con)") + MOI.submit(model, MOI.LazyConstraint(cb_data), con) + end + return + end + set_attribute(model, MOI.LazyConstraintCallback(), my_callback_function) + optimize!(model) + Test.@test is_solved_and_feasible(model) + Test.@test lazy_called + Test.@test value(x) == 1 + Test.@test value(y) == 2 + println("Optimal solution (x, y) = ($(value(x)), $(value(y)))") + return +end + +example_lazy_constraint() +```` + +## User-cuts + +An example using a user-cut callback. + +````@example callbacks +function example_user_cut_constraint() + Random.seed!(1) + N = 30 + item_weights, item_values = rand(N), rand(N) + model = Model(Gurobi.Optimizer) + set_silent(model) + # Turn off "Cuts" parameter so that our new one must be called. In real + # models, you should leave "Cuts" turned on. + set_attribute(model, "Cuts", 0) + @variable(model, x[1:N], Bin) + @constraint(model, sum(item_weights[i] * x[i] for i in 1:N) <= 10) + @objective(model, Max, sum(item_values[i] * x[i] for i in 1:N)) + callback_called = false + function my_callback_function(cb_data) + callback_called = true + x_vals = callback_value.(Ref(cb_data), x) + accumulated = sum(item_weights[i] for i in 1:N if x_vals[i] > 1e-4) + println("Called with accumulated = $(accumulated)") + n_terms = sum(1 for i in 1:N if x_vals[i] > 1e-4) + if accumulated > 10 + con = @build_constraint( + sum(x[i] for i in 1:N if x_vals[i] > 0.5) <= n_terms - 1 + ) + println("Adding $(con)") + MOI.submit(model, MOI.UserCut(cb_data), con) + end + end + set_attribute(model, MOI.UserCutCallback(), my_callback_function) + optimize!(model) + Test.@test is_solved_and_feasible(model) + Test.@test callback_called + @show callback_called + return +end + +example_user_cut_constraint() +```` + +## Heuristic solutions + +An example using a heuristic solution callback. + +````@example callbacks +function example_heuristic_solution() + Random.seed!(1) + N = 30 + item_weights, item_values = rand(N), rand(N) + model = Model(Gurobi.Optimizer) + set_silent(model) + # Turn off "Heuristics" parameter so that our new one must be called. In + # real models, you should leave "Heuristics" turned on. + set_attribute(model, "Heuristics", 0) + @variable(model, x[1:N], Bin) + @constraint(model, sum(item_weights[i] * x[i] for i in 1:N) <= 10) + @objective(model, Max, sum(item_values[i] * x[i] for i in 1:N)) + callback_called = false + function my_callback_function(cb_data) + callback_called = true + x_vals = callback_value.(Ref(cb_data), x) + ret = + MOI.submit(model, MOI.HeuristicSolution(cb_data), x, floor.(x_vals)) + println("Heuristic solution status = $(ret)") + Test.@test ret in ( + MOI.HEURISTIC_SOLUTION_ACCEPTED, + MOI.HEURISTIC_SOLUTION_REJECTED, + ) + end + set_attribute(model, MOI.HeuristicCallback(), my_callback_function) + optimize!(model) + Test.@test is_solved_and_feasible(model) + Test.@test callback_called + return +end + +example_heuristic_solution() +```` + +## Gurobi solver-dependent callback + +An example using Gurobi's solver-dependent callback. + +````@example callbacks +function example_solver_dependent_callback() + model = direct_model(Gurobi.Optimizer()) + @variable(model, 0 <= x <= 2.5, Int) + @variable(model, 0 <= y <= 2.5, Int) + @objective(model, Max, y) + cb_calls = Cint[] + function my_callback_function(cb_data, cb_where::Cint) + # You can reference variables outside the function as normal + push!(cb_calls, cb_where) + # You can select where the callback is run + if cb_where == Gurobi.GRB_CB_MIPNODE + # You can query a callback attribute using GRBcbget + resultP = Ref{Cint}() + Gurobi.GRBcbget( + cb_data, + cb_where, + Gurobi.GRB_CB_MIPNODE_STATUS, + resultP, + ) + if resultP[] != Gurobi.GRB_OPTIMAL + return # Solution is something other than optimal. + end + elseif cb_where != Gurobi.GRB_CB_MIPSOL + return + end + # Before querying `callback_value`, you must call: + Gurobi.load_callback_variable_primal(cb_data, cb_where) + x_val = callback_value(cb_data, x) + y_val = callback_value(cb_data, y) + # You can submit solver-independent MathOptInterface attributes such as + # lazy constraints, user-cuts, and heuristic solutions. + if y_val - x_val > 1 + 1e-6 + con = @build_constraint(y - x <= 1) + MOI.submit(model, MOI.LazyConstraint(cb_data), con) + elseif y_val + x_val > 3 + 1e-6 + con = @build_constraint(y + x <= 3) + MOI.submit(model, MOI.LazyConstraint(cb_data), con) + end + # You can terminate the callback as follows: + Gurobi.GRBterminate(backend(model)) + return + end + # You _must_ set this parameter if using lazy constraints. + set_attribute(model, "LazyConstraints", 1) + set_attribute(model, Gurobi.CallbackFunction(), my_callback_function) + optimize!(model) + Test.@test termination_status(model) == MOI.INTERRUPTED + return +end + +example_solver_dependent_callback() +```` + diff --git a/docs/src/tutorials/linear/multiple_solutions.md b/docs/src/tutorials/linear/multiple_solutions.md new file mode 100644 index 00000000000..0584a699329 --- /dev/null +++ b/docs/src/tutorials/linear/multiple_solutions.md @@ -0,0 +1,175 @@ +```@meta +EditURL = "multiple_solutions.jl" +``` + +# Finding multiple feasible solutions + +_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ +[_Download the source as a `.jl` file_](multiple_solutions.jl). + +_Author: James Foster (@jd-foster)_ + +This tutorial demonstrates how to formulate and solve a combinatorial problem +with multiple feasible solutions. In fact, we will see how to find _all_ +feasible solutions to our problem. We will also see how to enforce an +"all-different" constraint on a set of integer variables. + +## Required packages + +This tutorial uses the following packages: + +````@example multiple_solutions +using JuMP +import Gurobi +import Test +```` + +!!! warning + This tutorial uses [Gurobi.jl](@ref) as the solver because it supports + returning multiple feasible solutions, something that open-source MIP + solvers such as HiGHS do not currently support. Gurobi is a commercial + solver and requires a paid license. However, there are free licenses + available for academic and student users. See [Gurobi.jl](@ref) for more + details. + +## Symmetric number squares + +Symmetric [number squares](https://www.futilitycloset.com/2012/12/05/number-squares/) +and their sums often arise in recreational mathematics. Here are a few +examples: +``` + 1 5 2 9 2 3 1 8 5 2 1 9 + 5 8 3 7 3 7 9 0 2 3 8 4 ++ 2 3 4 0 + 1 9 5 6 + 1 8 6 7 += 9 7 0 6 = 8 0 6 4 = 9 4 7 0 +``` + +Notice how all the digits 0 to 9 are used at least once, the first three rows +sum to the last row, the columns in each are the same as the corresponding +rows (forming a symmetric matrix), and `0` does not appear in the first +column. + +We will answer the question: how many such squares are there? + +## JuMP model + +We now encode the symmetric number square as a JuMP model. First, we need a +symmetric matrix of decision variables between `0` and `9` to represent each +number: + +````@example multiple_solutions +n = 4 +model = Model() +set_silent(model) +@variable(model, 0 <= x_digits[row in 1:n, col in 1:n] <= 9, Int, Symmetric) +```` + +We modify the lower bound to ensure that the first column cannot contain `0`: + +````@example multiple_solutions +set_lower_bound.(x_digits[:, 1], 1) +```` + +Then, we need a constraint that the sum of the first three rows equals the +last row: + +````@example multiple_solutions +@expression(model, x_base_10, x_digits * [1_000, 100, 10, 1]); +@constraint(model, sum(x_base_10[i] for i in 1:n-1) == x_base_10[n]) +```` + +And we use [`MOI.AllDifferent`](@ref) to ensure that each digit is used +exactly once in the upper triangle matrix of `x_digits`: + +````@example multiple_solutions +x_digits_upper = [x_digits[i, j] for j in 1:n for i in 1:j] +@constraint(model, x_digits_upper in MOI.AllDifferent(length(x_digits_upper))); +nothing #hide +```` + +If we optimize this model, we find that Gurobi has returned one solution: + +````@example multiple_solutions +set_optimizer(model, Gurobi.Optimizer) +optimize!(model) +Test.@test is_solved_and_feasible(model) +Test.@test result_count(model) == 1 +solution_summary(model) +```` + +To return multiple solutions, we need to set Gurobi-specific parameters to +enable the [solution pool](https://docs.gurobi.com/projects/optimizer/en/current/features/solutionpool.html). +Moreover, there is a bug in Gurobi that means the solution pool is not +activated if we have already solved the model once. To work around the bug, we +need to reset the optimizer. If you turn the solution pool options on before +the first solve you do not need to reset the optimizer. + +````@example multiple_solutions +set_optimizer(model, Gurobi.Optimizer) +```` + +The first option turns on the exhaustive search mode for multiple solutions: + +````@example multiple_solutions +set_attribute(model, "PoolSearchMode", 2) +```` + +The second option sets a limit for the number of solutions found: + +````@example multiple_solutions +set_attribute(model, "PoolSolutions", 100) +```` + +Here the value 100 is an "arbitrary but large enough" whole number +for our particular model (and in general will depend on the application). + +We can then call `optimize!` and view the results. + +````@example multiple_solutions +optimize!(model) +Test.@test is_solved_and_feasible(model) +solution_summary(model) +```` + +Now Gurobi has found 20 solutions: + +````@example multiple_solutions +Test.@test result_count(model) == 20 +```` + +## Viewing the Results + +Access the various feasible solutions by using the [`value`](@ref) function +with the `result` keyword: + +````@example multiple_solutions +solutions = + [round.(Int, value.(x_digits; result = i)) for i in 1:result_count(model)]; +nothing #hide +```` + +Here we have converted the solution to an integer after rounding off very +small numerical tolerances. + +An example of one feasible solution is: + +````@example multiple_solutions +solutions[1] +```` + +and we can nicely print out all the feasible solutions with + +````@example multiple_solutions +function solution_string(x::Matrix) + header = [" ", " ", "+", "="] + return join([join(vcat(header[i], x[i, :]), " ") for i in 1:4], "\n") +end + +for i in 1:result_count(model) + println("Solution $i: \n", solution_string(solutions[i]), "\n") +end +```` + +The result is the full list of feasible solutions. So the answer to "how many +such squares are there?" turns out to be 20. + From 16e9022f296f5a30fef730340517f3340be97add Mon Sep 17 00:00:00 2001 From: Oscar Dowson Date: Wed, 22 Jan 2025 08:54:44 +1300 Subject: [PATCH 3/5] Update make.jl --- docs/make.jl | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/make.jl b/docs/make.jl index bd378cad048..0141500c0b5 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -324,9 +324,7 @@ jump_api_reference = DocumenterReference.automatic_reference_documentation(; # This constant dictates the layout of the documentation. It is manually # constructed so that we can have control over the order in which pages are # shown. If you add a new page to the documentation, make sure to add it here! - -filter_empty(x...) = filter(!isempty, collect(x)) - +# # !!! warning # If you move any of the top-level chapters around, make sure to update the # index of the "release_notes.md" in the section which builds the PDF. @@ -347,7 +345,7 @@ const _PAGES = [ ], "Transitioning" => ["tutorials/transitioning/transitioning_from_matlab.md"], - "Linear programs" => filter_empty( + "Linear programs" => [ "tutorials/linear/introduction.md", "tutorials/linear/knapsack.md", "tutorials/linear/diet.md", @@ -367,12 +365,12 @@ const _PAGES = [ "tutorials/linear/sudoku.md", "tutorials/linear/n-queens.md", "tutorials/linear/constraint_programming.md", - _HAS_GUROBI ? "tutorials/linear/callbacks.md" : "", + "tutorials/linear/callbacks.md", "tutorials/linear/lp_sensitivity.md", "tutorials/linear/basis.md", "tutorials/linear/mip_duality.md", - _HAS_GUROBI ? "tutorials/linear/multiple_solutions.md" : "", - ), + "tutorials/linear/multiple_solutions.md", + ], "Nonlinear programs" => [ "tutorials/nonlinear/introduction.md", "tutorials/nonlinear/simple_examples.md", @@ -401,14 +399,14 @@ const _PAGES = [ "tutorials/conic/ellipse_fitting.md", "tutorials/conic/quantum_discrimination.md", ], - "Algorithms" => filter_empty( - _HAS_GUROBI ? "tutorials/algorithms/benders_decomposition.md" : "", + "Algorithms" => [ + "tutorials/algorithms/benders_decomposition.md", "tutorials/algorithms/cutting_stock_column_generation.md", - _HAS_GUROBI ? "tutorials/algorithms/tsp_lazy_constraints.md" : "", + "tutorials/algorithms/tsp_lazy_constraints.md", "tutorials/algorithms/rolling_horizon.md", "tutorials/algorithms/parallelism.md", "tutorials/algorithms/pdhg.md", - ), + ], "Applications" => [ "tutorials/applications/power_systems.md", "tutorials/applications/optimal_power_flow.md", From 6b0ad8c54d085fd45ea84868e400a48712a9f42b Mon Sep 17 00:00:00 2001 From: odow Date: Wed, 22 Jan 2025 10:09:44 +1300 Subject: [PATCH 4/5] Update --- docs/make.jl | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/docs/make.jl b/docs/make.jl index 0141500c0b5..582f7f8a446 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -43,10 +43,10 @@ end const _GUROBI_EXCLUDES = String[] if !_HAS_GUROBI - push!(_GUROBI_EXCLUDES, "benders_decomposition.jl") - push!(_GUROBI_EXCLUDES, "tsp_lazy_constraints.jl") - push!(_GUROBI_EXCLUDES, "callbacks.jl") - push!(_GUROBI_EXCLUDES, "multiple_solutions.jl") + push!(_GUROBI_EXCLUDES, "benders_decomposition") + push!(_GUROBI_EXCLUDES, "tsp_lazy_constraints") + push!(_GUROBI_EXCLUDES, "callbacks") + push!(_GUROBI_EXCLUDES, "multiple_solutions") end # ============================================================================== @@ -75,8 +75,12 @@ end function _file_list(full_dir, relative_dir, extension) function filter_fn(filename) - return endswith(filename, extension) && - all(f -> _HAS_GUROBI || !endswith(filename, f), _GUROBI_EXCLUDES) + if !endswith(filename, extension) + return false + elseif _HAS_GUROBI + return true + end + return all(f -> !endswith(filename, f * extension), _GUROBI_EXCLUDES) end return map(filter!(filter_fn, sort!(readdir(full_dir)))) do file return joinpath(relative_dir, file) From 8c5685ec95bd6c7ad4d6db8003dddf3e15f248c0 Mon Sep 17 00:00:00 2001 From: odow Date: Thu, 23 Jan 2025 15:02:26 +1300 Subject: [PATCH 5/5] Update --- .../algorithms/benders_decomposition.md | 562 +----------------- .../algorithms/tsp_lazy_constraints.md | 292 +-------- docs/src/tutorials/linear/callbacks.md | 216 +------ .../tutorials/linear/multiple_solutions.md | 171 +----- 4 files changed, 8 insertions(+), 1233 deletions(-) diff --git a/docs/src/tutorials/algorithms/benders_decomposition.md b/docs/src/tutorials/algorithms/benders_decomposition.md index 1d77cac9746..12e986598ec 100644 --- a/docs/src/tutorials/algorithms/benders_decomposition.md +++ b/docs/src/tutorials/algorithms/benders_decomposition.md @@ -4,563 +4,5 @@ EditURL = "benders_decomposition.jl" # [Benders decomposition](@id benders_decomposition_classical) -_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ -[_Download the source as a `.jl` file_](benders_decomposition.jl). - -**This tutorial was originally contributed by Shuvomoy Das Gupta.** - -This tutorial describes how to implement [Benders decomposition](https://en.wikipedia.org/wiki/Benders_decomposition) -in JuMP. It uses the following packages: - -````@example benders_decomposition -using JuMP -import Gurobi -import HiGHS -import Printf -```` - -## Theory - -Benders decomposition is a useful algorithm for solving convex optimization -problems with a large number of variables. It works best when a larger problem -can be decomposed into two (or more) smaller problems that are individually -much easier to solve. - -This tutorial demonstrates Benders decomposition on the following -mixed-integer linear program: -```math -\begin{aligned} -\text{min} \ & c_1(x) + c_2(y) \\ -\text{subject to} \ & f_1(x) \in S_1 \\ - & f_2(y) \in S_2 \\ - & f_3(x, y) \in S_3 \\ - & x \in \mathbb{Z}^m \\ - & y \in \mathbb{R}^n \\ -\end{aligned} -``` -where the functions $f$ and $c$ are linear, and the sets $S$ are inequality -sets like $\ge l$, $\le u$, or $= b$. - -Any mixed integer programming problem can be written in the form above. - -If there are relatively few integer variables, and many more continuous -variables, then it may be beneficial to decompose the problem into a small -problem containing only integer variables and a linear program containing -only continuous variables. Hopefully, the linear program will be much easier -to solve in isolation than in the full mixed-integer linear program. - -For example, if we knew a feasible solution for ``\bar{x}``, we could obtain a -solution for ``y`` by solving: -```math -\begin{aligned} -V_2(\bar{x}) = & \text{min} \ & c_2(y)\\ - & \text{subject to} \ & f_2(y) \in S_2 \\ - & & f_3(x, y) \in S_3 \\ - & & x = \bar{x} & \ [\pi] \\ - & & y \in \mathbb{R}^n \\ -\end{aligned} -``` -Note that we have included a "copy" of the `x` variable to simplify computing -$\pi$, which is the dual of $V_2$ with respect to $\bar{x}$. - -Because this model is a linear program, it is easy to solve. - -Replacing the ``c_2(y)`` component of the objective in our original problem -with ``V_2`` yields: -```math -\begin{aligned} -V_1 = & \text{min} \ & c_1(x) + V_2(x) \\ -& \text{subject to} \ & f_1(x) \in S_1 \\ -& & x \in \mathbb{Z}^m. -\end{aligned} -``` -This problem looks a lot simpler to solve because it involves only $x$ and a -subset of the constraints, but we need to do something else with ``V_2`` first. - -Because ``\bar{x}`` is a constant that appears on the right-hand side of the -constraints, ``V_2`` is a convex function with respect to ``\bar{x}``, and the -dual variable ``\pi`` is a subgradient of ``V_2(x)`` with respect to ``x``. -Therefore, if we have a candidate solution ``x_k``, then we can solve -``V_2(x_k)`` and obtain a feasible dual vector ``\pi_k``. Using these values, -we can construct a first-order Taylor-series approximation of ``V_2`` about -the point ``x_k``: -```math -V_2(x) \ge V_2(x_k) + \pi_k^\top (x - x_k). -``` -By convexity, we know that this inequality holds for all ``x``, and we call -these inequalities _cuts_. - -Benders decomposition is an iterative technique that replaces ``V_2(x)`` with -a new decision variable ``\theta``, and approximates it from below using cuts: -```math -\begin{aligned} -V_1^K = & \text{min} \ & c_1(x) + \theta \\ - & \text{subject to} \ & f_1(x) \in S_1 \\ - & & x \in \mathbb{Z}^m \\ - & & \theta \ge M \\ - & & \theta \ge V_2(x_k) + \pi_k^\top(x - x_k) & \quad \forall k = 1,\ldots,K. -\end{aligned} -``` -This integer program is called the _first-stage_ subproblem. - -To generate cuts, we solve ``V_1^K`` to obtain a candidate first-stage -solution ``x_k``, then we use that solution to solve ``V_2(x_k)``. Then, using -the optimal objective value and dual solution from ``V_2``, we add a new cut -to form ``V_1^{K+1}`` and repeat. - -### Bounds - -Due to convexity, we know that ``V_2(x) \ge \theta`` for all ``x``. Therefore, -the optimal objective value of ``V_1^K`` provides a valid _lower_ bound on the -objective value of the full problem. In addition, if we take a feasible -solution for ``x`` from the first-stage problem, then ``c_1(x) + V_2(x)`` -is a valid _upper_ bound on the objective value of the full problem. - -Benders decomposition uses the lower and upper bounds to determine when it has -found the global optimal solution. - -## Monolithic problem - -As an example problem, we consider the following variant of -[The max-flow problem](@ref), in which there is a binary variable to decide -whether to open each arc for a cost of 0.1 unit, and we can open at most 11 -arcs: - -````@example benders_decomposition -G = [ - 0 3 2 2 0 0 0 0 - 0 0 0 0 5 1 0 0 - 0 0 0 0 1 3 1 0 - 0 0 0 0 0 1 0 0 - 0 0 0 0 0 0 0 4 - 0 0 0 0 0 0 0 2 - 0 0 0 0 0 0 0 4 - 0 0 0 0 0 0 0 0 -] -n = size(G, 1) -model = Model(HiGHS.Optimizer) -set_silent(model) -@variable(model, x[1:n, 1:n], Bin) -@variable(model, y[1:n, 1:n] >= 0) -@constraint(model, sum(x) <= 11) -@constraint(model, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x[i, j]) -@constraint(model, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) -@objective(model, Min, 0.1 * sum(x) - sum(y[1, :])) -optimize!(model) -solution_summary(model) -```` - -The optimal objective value is -5.1: - -````@example benders_decomposition -objective_value(model) -```` - -and the optimal flows are: - -````@example benders_decomposition -function optimal_flows(x) - return [(i, j) => x[i, j] for i in 1:n for j in 1:n if x[i, j] > 0] -end - -monolithic_solution = optimal_flows(value.(y)) -```` - -## [Iterative method](@id benders_iterative) - -!!! warning - This is a basic implementation for pedagogical purposes. We haven't - discussed any of the computational tricks that are required to build a - performant implementation for large-scale problems. See [In-place iterative method](@ref) - for one improvement that helps computation time. - -We start by formulating the first-stage subproblem. It includes the `x` -variables, and the constraints involving only `x`, and the terms in the -objective containing only `x`. We also need an initial lower bound on the -cost-to-go variable `θ`. One valid lower bound is to assume that we do not pay -for opening arcs, and there is flow all the arcs. - -````@example benders_decomposition -M = -sum(G) -model = Model(HiGHS.Optimizer) -set_silent(model) -@variable(model, x[1:n, 1:n], Bin) -@variable(model, θ >= M) -@constraint(model, sum(x) <= 11) -@objective(model, Min, 0.1 * sum(x) + θ) -model -```` - -For the next step, we need a function that takes a first-stage candidate -solution `x` and returns the optimal solution from the second-stage -subproblem: - -````@example benders_decomposition -function solve_subproblem(x_bar) - model = Model(HiGHS.Optimizer) - set_silent(model) - @variable(model, x[i in 1:n, j in 1:n] == x_bar[i, j]) - @variable(model, y[1:n, 1:n] >= 0) - @constraint(model, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x[i, j]) - @constraint(model, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) - @objective(model, Min, -sum(y[1, :])) - optimize!(model) - @assert is_solved_and_feasible(model; dual = true) - return (obj = objective_value(model), y = value.(y), π = reduced_cost.(x)) -end -```` - -Note that `solve_subproblem` returns a `NamedTuple` of the objective value, -the optimal primal solution for `y`, and the optimal dual solution for `π`, -which we obtained from the [`reduced_cost`](@ref) of the `x` variables. - -We're almost ready for our optimization loop, but first, here's a helpful -function for logging: - -````@example benders_decomposition -function print_iteration(k, args...) - f(x) = Printf.@sprintf("%12.4e", x) - println(lpad(k, 9), " ", join(f.(args), " ")) - return -end -```` - -We also need to put a limit on the number of iterations before termination: - -````@example benders_decomposition -MAXIMUM_ITERATIONS = 100 -```` - -And a way to check if the lower and upper bounds are close-enough to -terminate: - -````@example benders_decomposition -ABSOLUTE_OPTIMALITY_GAP = 1e-6 -```` - -Now we're ready to iterate Benders decomposition: - -````@example benders_decomposition -println("Iteration Lower Bound Upper Bound Gap") -for k in 1:MAXIMUM_ITERATIONS - optimize!(model) - @assert is_solved_and_feasible(model) - lower_bound = objective_value(model) - x_k = value.(x) - ret = solve_subproblem(x_k) - upper_bound = (objective_value(model) - value(θ)) + ret.obj - gap = abs(upper_bound - lower_bound) / abs(upper_bound) - print_iteration(k, lower_bound, upper_bound, gap) - if gap < ABSOLUTE_OPTIMALITY_GAP - println("Terminating with the optimal solution") - break - end - cut = @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) - @info "Adding the cut $(cut)" -end -```` - -Finally, we can obtain the optimal solution: - -````@example benders_decomposition -optimize!(model) -@assert is_solved_and_feasible(model) -x_optimal = value.(x) -optimal_ret = solve_subproblem(x_optimal) -iterative_solution = optimal_flows(optimal_ret.y) -```` - -which is the same as the monolithic solution: - -````@example benders_decomposition -iterative_solution == monolithic_solution -```` - -and it has the same objective value: - -````@example benders_decomposition -objective_value(model) -```` - -## Callback method - -The [Iterative method](@ref benders_iterative) section implemented Benders -decomposition using a loop. In each iteration, we re-solved the first-stage -subproblem to generate a candidate solution. However, modern MILP solvers such -as CPLEX, Gurobi, and GLPK provide lazy constraint callbacks which allow us to -add new cuts _while the solver is running_. This can be more efficient than an -iterative method because we can avoid repeating work such as solving the root -node of the first-stage MILP at each iteration. - -!!! tip - We use Gurobi for this model because HiGHS does not support lazy - constraints. For more information on callbacks, read the page - [Solver-independent callbacks](@ref callbacks_manual). - -As before, we construct the same first-stage subproblem: - -````@example benders_decomposition -lazy_model = Model(Gurobi.Optimizer) -set_silent(lazy_model) -@variable(lazy_model, x[1:n, 1:n], Bin) -@variable(lazy_model, θ >= M) -@constraint(lazy_model, sum(x) <= 11) -@objective(lazy_model, Min, 0.1 * sum(x) + θ) -lazy_model -```` - -What differs is that we write a callback function instead of a loop: - -````@example benders_decomposition -number_of_subproblem_solves = 0 -function my_callback(cb_data) - status = callback_node_status(cb_data, lazy_model) - if status != MOI.CALLBACK_NODE_STATUS_INTEGER - # Only add the constraint if `x` is an integer feasible solution - return - end - x_k = callback_value.(cb_data, x) - θ_k = callback_value(cb_data, θ) - global number_of_subproblem_solves += 1 - ret = solve_subproblem(x_k) - if θ_k < (ret.obj - 1e-6) - # Only add the constraint if θ_k violates the constraint - cut = @build_constraint(θ >= ret.obj + sum(ret.π .* (x .- x_k))) - MOI.submit(lazy_model, MOI.LazyConstraint(cb_data), cut) - end - return -end - -set_attribute(lazy_model, MOI.LazyConstraintCallback(), my_callback) -```` - -Now when we optimize!, our callback is run: - -````@example benders_decomposition -optimize!(lazy_model) -@assert is_solved_and_feasible(lazy_model) -```` - -For this model, the callback algorithm required more solves of the subproblem: - -````@example benders_decomposition -number_of_subproblem_solves -```` - -But for larger problems, you can expect the callback algorithm to be more -efficient than the iterative algorithm. - -Finally, we can obtain the optimal solution: - -````@example benders_decomposition -x_optimal = value.(x) -optimal_ret = solve_subproblem(x_optimal) -callback_solution = optimal_flows(optimal_ret.y) -```` - -which is the same as the monolithic solution: - -````@example benders_decomposition -callback_solution == monolithic_solution -```` - -## In-place iterative method - -Our implementation of the iterative method has a problem: every time we need -to solve the subproblem, we must rebuild it from scratch. This is expensive, -and it can be the bottleneck in the solution process. We can improve our -implementation by using re-using the subproblem between solves. - -First, we create our first-stage problem as usual: - -````@example benders_decomposition -model = Model(HiGHS.Optimizer) -set_silent(model) -@variable(model, x[1:n, 1:n], Bin) -@variable(model, θ >= M) -@constraint(model, sum(x) <= 11) -@objective(model, Min, 0.1 * sum(x) + θ) -model -```` - -Then, instead of building the subproblem in a function, we build it once here: - -````@example benders_decomposition -subproblem = Model(HiGHS.Optimizer) -set_silent(subproblem) -@variable(subproblem, x_copy[i in 1:n, j in 1:n]) -@variable(subproblem, y[1:n, 1:n] >= 0) -@constraint(subproblem, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x_copy[i, j]) -@constraint(subproblem, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) -@objective(subproblem, Min, -sum(y[1, :])) -subproblem -```` - -Our function to solve the subproblem is also slightly different because we -need to fix the value of the `x_copy` variables to the value of `x` from the -first-stage problem: - -````@example benders_decomposition -function solve_subproblem(model, x) - fix.(model[:x_copy], x) - optimize!(model) - @assert is_solved_and_feasible(model; dual = true) - return ( - obj = objective_value(model), - y = value.(model[:y]), - π = reduced_cost.(model[:x_copy]), - ) -end -```` - -Now we're ready to iterate our in-place Benders decomposition: - -````@example benders_decomposition -println("Iteration Lower Bound Upper Bound Gap") -for k in 1:MAXIMUM_ITERATIONS - optimize!(model) - @assert is_solved_and_feasible(model) - lower_bound = objective_value(model) - x_k = value.(x) - ret = solve_subproblem(subproblem, x_k) - upper_bound = (objective_value(model) - value(θ)) + ret.obj - gap = abs(upper_bound - lower_bound) / abs(upper_bound) - print_iteration(k, lower_bound, upper_bound, gap) - if gap < ABSOLUTE_OPTIMALITY_GAP - println("Terminating with the optimal solution") - break - end - cut = @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) - @info "Adding the cut $(cut)" -end -```` - -Finally, we can obtain the optimal solution: - -````@example benders_decomposition -optimize!(model) -@assert is_solved_and_feasible(model) -x_optimal = value.(x) -optimal_ret = solve_subproblem(subproblem, x_optimal) -inplace_solution = optimal_flows(optimal_ret.y) -```` - -which is the same as the monolithic solution: - -````@example benders_decomposition -inplace_solution == monolithic_solution -```` - -## Feasibility cuts - -So far, we have discussed only Benders optimality cuts. However, for some -first-stage values of `x`, the subproblem might be infeasible. The solution is -to add a Benders feasibility cut: -```math -v_k + u_k^\top (x - x_k) \le 0 -``` -where $u_k$ is a dual unbounded ray of the subproblem and $v_k$ is the -intercept of the unbounded ray. - -As a variation of our example which leads to infeasibilities, we add a -constraint that `sum(y) >= 1`. This means we need a choice of first-stage `x` -for which at least one unit can flow. - -The first-stage problem remains the same: - -````@example benders_decomposition -model = Model(HiGHS.Optimizer) -set_silent(model) -@variable(model, x[1:n, 1:n], Bin) -@variable(model, θ >= M) -@constraint(model, sum(x) <= 11) -@objective(model, Min, 0.1 * sum(x) + θ) -model -```` - -But the subproblem has a new constraint that `sum(y) >= 1`: - -````@example benders_decomposition -subproblem = Model(HiGHS.Optimizer) -set_silent(subproblem) -# We need to turn presolve off so that HiGHS will return an infeasibility -# certificate. -set_attribute(subproblem, "presolve", "off") -@variable(subproblem, x_copy[i in 1:n, j in 1:n]) -@variable(subproblem, y[1:n, 1:n] >= 0) -@constraint(subproblem, sum(y) >= 1) # <--- THIS IS NEW -@constraint(subproblem, [i = 1:n, j = 1:n], y[i, j] <= G[i, j] * x_copy[i, j]) -@constraint(subproblem, [i = 2:n-1], sum(y[i, :]) == sum(y[:, i])) -@objective(subproblem, Min, -sum(y[1, :])) -subproblem -```` - -The function to solve the subproblem now checks for feasibility, and returns -the dual objective value and an dual unbounded ray if the subproblem is -infeasible: - -````@example benders_decomposition -function solve_subproblem_with_feasibility(model, x) - fix.(model[:x_copy], x) - optimize!(model) - if is_solved_and_feasible(model; dual = true) - return ( - is_feasible = true, - obj = objective_value(model), - y = value.(model[:y]), - π = reduced_cost.(model[:x_copy]), - ) - end - return ( - is_feasible = false, - v = dual_objective_value(model), - u = reduced_cost.(model[:x_copy]), - ) -end -```` - -Now we're ready to iterate our in-place Benders decomposition: - -````@example benders_decomposition -println("Iteration Lower Bound Upper Bound Gap") -for k in 1:MAXIMUM_ITERATIONS - optimize!(model) - @assert is_solved_and_feasible(model) - lower_bound = objective_value(model) - x_k = value.(x) - ret = solve_subproblem_with_feasibility(subproblem, x_k) - if ret.is_feasible - # Benders Optimality Cuts - upper_bound = (objective_value(model) - value(θ)) + ret.obj - gap = abs(upper_bound - lower_bound) / abs(upper_bound) - print_iteration(k, lower_bound, upper_bound, gap) - if gap < ABSOLUTE_OPTIMALITY_GAP - println("Terminating with the optimal solution") - break - end - @constraint(model, θ >= ret.obj + sum(ret.π .* (x .- x_k))) - else - # Benders Feasibility Cuts - cut = @constraint(model, ret.v + sum(ret.u .* (x .- x_k)) <= 0) - @info "Adding the feasibility cut $(cut)" - end -end -```` - -Finally, we can obtain the optimal solution: - -````@example benders_decomposition -optimize!(model) -@assert is_solved_and_feasible(model) -x_optimal = value.(x) -optimal_ret = solve_subproblem(subproblem, x_optimal) -feasible_inplace_solution = optimal_flows(optimal_ret.y) -```` - -which is the same as the monolithic solution (because `sum(y) >= 1` in the -monolithic solution): - -````@example benders_decomposition -feasible_inplace_solution == monolithic_solution -```` - +This page is a placeholder that appears only if the documentation is built from +a fork. diff --git a/docs/src/tutorials/algorithms/tsp_lazy_constraints.md b/docs/src/tutorials/algorithms/tsp_lazy_constraints.md index ff87a9fc536..0b4a932ce7b 100644 --- a/docs/src/tutorials/algorithms/tsp_lazy_constraints.md +++ b/docs/src/tutorials/algorithms/tsp_lazy_constraints.md @@ -4,293 +4,5 @@ EditURL = "tsp_lazy_constraints.jl" # [Traveling Salesperson Problem](@id tsp_lazy) -_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ -[_Download the source as a `.jl` file_](tsp_lazy_constraints.jl). - -**This tutorial was originally contributed by Daniel Schermer.** - -This tutorial describes how to implement the -[Traveling Salesperson Problem](https://en.wikipedia.org/wiki/Travelling_salesman_problem) -in JuMP using solver-independent lazy constraints that dynamically separate -subtours. To be more precise, we use lazy constraints to cut off infeasible -subtours only when necessary and not before needed. - -It uses the following packages: - -````@example tsp_lazy_constraints -using JuMP -import Gurobi -import Plots -import Random -import Test -```` - -## [Mathematical Formulation](@id tsp_model) - -Assume that we are given a complete graph $\mathcal{G}(V,E)$ where $V$ is the -set of vertices (or cities) and $E$ is the set of edges (or roads). For each -pair of vertices $i, j \in V, i \neq j$ the edge $(i,j) \in E$ is associated -with a weight (or distance) $d_{ij} \in \mathbb{R}^+$. - -For this tutorial, we assume the problem to be symmetric, that is, -$d_{ij} = d_{ji} \, \forall i,j \in V$. - -In the Traveling Salesperson Problem, we are tasked with finding a tour with -minimal length that visits every vertex exactly once and then returns to the -point of origin, that is, a *Hamiltonian cycle* with minimal weight. - -To model the problem, we introduce a binary variable, -$x_{ij} \in \{0,1\} \; \forall i, j \in V$, that indicates if edge $(i,j)$ is -part of the tour or not. Using these variables, the Traveling Salesperson -Problem can be modeled as the following integer linear program. - -### [Objective Function](@id tsp_objective) - -The objective is to minimize the length of the tour (due to the assumed -symmetry, the second sum only contains $i 1 - pop!(unvisited, current) - end - neighbors = - [j for (i, j) in edges if i == current && j in unvisited] - end - if length(this_cycle) < length(shortest_subtour) - shortest_subtour = this_cycle - end - end - return shortest_subtour -end -```` - -Let us declare a helper function `selected_edges()` that will be repeatedly -used in what follows. - -````@example tsp_lazy_constraints -function selected_edges(x::Matrix{Float64}, n) - return Tuple{Int,Int}[(i, j) for i in 1:n, j in 1:n if x[i, j] > 0.5] -end -```` - -Other helper functions for computing subtours: - -````@example tsp_lazy_constraints -subtour(x::Matrix{Float64}) = subtour(selected_edges(x, size(x, 1)), size(x, 1)) -subtour(x::AbstractMatrix{VariableRef}) = subtour(value.(x)) -```` - -### Iterative method - -An iterative way of eliminating subtours is the following. - -However, it is reasonable to assume that this is not the most efficient way: -whenever a new subtour elimination constraint is added to the model, the -optimization has to start from the very beginning. - -That way, the solver will repeatedly discard useful information encountered -during previous solves (for example, all cuts, the incumbent solution, or lower -bounds). - -!!! info - Note that, in principle, it would also be feasible to add all subtours - (instead of just the shortest one) to the model. However, preventing just - the shortest cycle is often sufficient for breaking other subtours and - will keep the model size smaller. - -````@example tsp_lazy_constraints -iterative_model = build_tsp_model(d, n) -optimize!(iterative_model) -@assert is_solved_and_feasible(iterative_model) -time_iterated = solve_time(iterative_model) -cycle = subtour(iterative_model[:x]) -while 1 < length(cycle) < n - println("Found cycle of length $(length(cycle))") - S = [(i, j) for (i, j) in Iterators.product(cycle, cycle) if i < j] - @constraint( - iterative_model, - sum(iterative_model[:x][i, j] for (i, j) in S) <= length(cycle) - 1, - ) - optimize!(iterative_model) - @assert is_solved_and_feasible(iterative_model) - global time_iterated += solve_time(iterative_model) - global cycle = subtour(iterative_model[:x]) -end -```` - -````@example tsp_lazy_constraints -objective_value(iterative_model) -```` - -````@example tsp_lazy_constraints -time_iterated -```` - -As a quick sanity check, we visualize the optimal tour to verify that no -subtour is present: - -````@example tsp_lazy_constraints -function plot_tour(X, Y, x) - plot = Plots.plot() - for (i, j) in selected_edges(x, size(x, 1)) - Plots.plot!([X[i], X[j]], [Y[i], Y[j]]; legend = false) - end - return plot -end - -plot_tour(X, Y, value.(iterative_model[:x])) -```` - -### Lazy constraint method - -A more sophisticated approach makes use of _lazy constraints_. To be more -precise, we do this through the `subtour_elimination_callback()` below, which -is only run whenever we encounter a new integer-feasible solution. - -````@example tsp_lazy_constraints -lazy_model = build_tsp_model(d, n) -function subtour_elimination_callback(cb_data) - status = callback_node_status(cb_data, lazy_model) - if status != MOI.CALLBACK_NODE_STATUS_INTEGER - return # Only run at integer solutions - end - cycle = subtour(callback_value.(cb_data, lazy_model[:x])) - if !(1 < length(cycle) < n) - return # Only add a constraint if there is a cycle - end - S = [(i, j) for (i, j) in Iterators.product(cycle, cycle) if i < j] - con = @build_constraint( - sum(lazy_model[:x][i, j] for (i, j) in S) <= length(cycle) - 1, - ) - MOI.submit(lazy_model, MOI.LazyConstraint(cb_data), con) - return -end -set_attribute( - lazy_model, - MOI.LazyConstraintCallback(), - subtour_elimination_callback, -) -optimize!(lazy_model) -```` - -````@example tsp_lazy_constraints -@assert is_solved_and_feasible(lazy_model) -objective_value(lazy_model) -```` - -````@example tsp_lazy_constraints -time_lazy = solve_time(lazy_model) -```` - -This finds the same optimal tour: - -````@example tsp_lazy_constraints -plot_tour(X, Y, value.(lazy_model[:x])) -```` - -The solution time is faster than the iterative approach: - -````@example tsp_lazy_constraints -Test.@test time_lazy < time_iterated -```` - +This page is a placeholder that appears only if the documentation is built from +a fork. diff --git a/docs/src/tutorials/linear/callbacks.md b/docs/src/tutorials/linear/callbacks.md index a2818502c27..7e5cbdd625d 100644 --- a/docs/src/tutorials/linear/callbacks.md +++ b/docs/src/tutorials/linear/callbacks.md @@ -4,217 +4,5 @@ EditURL = "callbacks.jl" # [Callbacks](@id callbacks_tutorial) -_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ -[_Download the source as a `.jl` file_](callbacks.jl). - -The purpose of the tutorial is to demonstrate the various solver-independent -and solver-dependent callbacks that are supported by JuMP. - -The tutorial uses the following packages: - -````@example callbacks -using JuMP -import Gurobi -import Random -import Test -```` - -!!! info - This tutorial uses the [MathOptInterface](@ref moi_documentation) API. - By default, JuMP exports the `MOI` symbol as an alias for the - MathOptInterface.jl package. We recommend making this more explicit in - your code by adding the following lines: - ```julia - import MathOptInterface as MOI - ``` - -## Lazy constraints - -An example using a lazy constraint callback. - -````@example callbacks -function example_lazy_constraint() - model = Model(Gurobi.Optimizer) - set_silent(model) - @variable(model, 0 <= x <= 2.5, Int) - @variable(model, 0 <= y <= 2.5, Int) - @objective(model, Max, y) - lazy_called = false - function my_callback_function(cb_data) - lazy_called = true - x_val = callback_value(cb_data, x) - y_val = callback_value(cb_data, y) - println("Called from (x, y) = ($x_val, $y_val)") - status = callback_node_status(cb_data, model) - if status == MOI.CALLBACK_NODE_STATUS_FRACTIONAL - println(" - Solution is integer infeasible!") - elseif status == MOI.CALLBACK_NODE_STATUS_INTEGER - println(" - Solution is integer feasible!") - else - @assert status == MOI.CALLBACK_NODE_STATUS_UNKNOWN - println(" - I don't know if the solution is integer feasible :(") - end - if y_val - x_val > 1 + 1e-6 - con = @build_constraint(y - x <= 1) - println("Adding $(con)") - MOI.submit(model, MOI.LazyConstraint(cb_data), con) - elseif y_val + x_val > 3 + 1e-6 - con = @build_constraint(y + x <= 3) - println("Adding $(con)") - MOI.submit(model, MOI.LazyConstraint(cb_data), con) - end - return - end - set_attribute(model, MOI.LazyConstraintCallback(), my_callback_function) - optimize!(model) - Test.@test is_solved_and_feasible(model) - Test.@test lazy_called - Test.@test value(x) == 1 - Test.@test value(y) == 2 - println("Optimal solution (x, y) = ($(value(x)), $(value(y)))") - return -end - -example_lazy_constraint() -```` - -## User-cuts - -An example using a user-cut callback. - -````@example callbacks -function example_user_cut_constraint() - Random.seed!(1) - N = 30 - item_weights, item_values = rand(N), rand(N) - model = Model(Gurobi.Optimizer) - set_silent(model) - # Turn off "Cuts" parameter so that our new one must be called. In real - # models, you should leave "Cuts" turned on. - set_attribute(model, "Cuts", 0) - @variable(model, x[1:N], Bin) - @constraint(model, sum(item_weights[i] * x[i] for i in 1:N) <= 10) - @objective(model, Max, sum(item_values[i] * x[i] for i in 1:N)) - callback_called = false - function my_callback_function(cb_data) - callback_called = true - x_vals = callback_value.(Ref(cb_data), x) - accumulated = sum(item_weights[i] for i in 1:N if x_vals[i] > 1e-4) - println("Called with accumulated = $(accumulated)") - n_terms = sum(1 for i in 1:N if x_vals[i] > 1e-4) - if accumulated > 10 - con = @build_constraint( - sum(x[i] for i in 1:N if x_vals[i] > 0.5) <= n_terms - 1 - ) - println("Adding $(con)") - MOI.submit(model, MOI.UserCut(cb_data), con) - end - end - set_attribute(model, MOI.UserCutCallback(), my_callback_function) - optimize!(model) - Test.@test is_solved_and_feasible(model) - Test.@test callback_called - @show callback_called - return -end - -example_user_cut_constraint() -```` - -## Heuristic solutions - -An example using a heuristic solution callback. - -````@example callbacks -function example_heuristic_solution() - Random.seed!(1) - N = 30 - item_weights, item_values = rand(N), rand(N) - model = Model(Gurobi.Optimizer) - set_silent(model) - # Turn off "Heuristics" parameter so that our new one must be called. In - # real models, you should leave "Heuristics" turned on. - set_attribute(model, "Heuristics", 0) - @variable(model, x[1:N], Bin) - @constraint(model, sum(item_weights[i] * x[i] for i in 1:N) <= 10) - @objective(model, Max, sum(item_values[i] * x[i] for i in 1:N)) - callback_called = false - function my_callback_function(cb_data) - callback_called = true - x_vals = callback_value.(Ref(cb_data), x) - ret = - MOI.submit(model, MOI.HeuristicSolution(cb_data), x, floor.(x_vals)) - println("Heuristic solution status = $(ret)") - Test.@test ret in ( - MOI.HEURISTIC_SOLUTION_ACCEPTED, - MOI.HEURISTIC_SOLUTION_REJECTED, - ) - end - set_attribute(model, MOI.HeuristicCallback(), my_callback_function) - optimize!(model) - Test.@test is_solved_and_feasible(model) - Test.@test callback_called - return -end - -example_heuristic_solution() -```` - -## Gurobi solver-dependent callback - -An example using Gurobi's solver-dependent callback. - -````@example callbacks -function example_solver_dependent_callback() - model = direct_model(Gurobi.Optimizer()) - @variable(model, 0 <= x <= 2.5, Int) - @variable(model, 0 <= y <= 2.5, Int) - @objective(model, Max, y) - cb_calls = Cint[] - function my_callback_function(cb_data, cb_where::Cint) - # You can reference variables outside the function as normal - push!(cb_calls, cb_where) - # You can select where the callback is run - if cb_where == Gurobi.GRB_CB_MIPNODE - # You can query a callback attribute using GRBcbget - resultP = Ref{Cint}() - Gurobi.GRBcbget( - cb_data, - cb_where, - Gurobi.GRB_CB_MIPNODE_STATUS, - resultP, - ) - if resultP[] != Gurobi.GRB_OPTIMAL - return # Solution is something other than optimal. - end - elseif cb_where != Gurobi.GRB_CB_MIPSOL - return - end - # Before querying `callback_value`, you must call: - Gurobi.load_callback_variable_primal(cb_data, cb_where) - x_val = callback_value(cb_data, x) - y_val = callback_value(cb_data, y) - # You can submit solver-independent MathOptInterface attributes such as - # lazy constraints, user-cuts, and heuristic solutions. - if y_val - x_val > 1 + 1e-6 - con = @build_constraint(y - x <= 1) - MOI.submit(model, MOI.LazyConstraint(cb_data), con) - elseif y_val + x_val > 3 + 1e-6 - con = @build_constraint(y + x <= 3) - MOI.submit(model, MOI.LazyConstraint(cb_data), con) - end - # You can terminate the callback as follows: - Gurobi.GRBterminate(backend(model)) - return - end - # You _must_ set this parameter if using lazy constraints. - set_attribute(model, "LazyConstraints", 1) - set_attribute(model, Gurobi.CallbackFunction(), my_callback_function) - optimize!(model) - Test.@test termination_status(model) == MOI.INTERRUPTED - return -end - -example_solver_dependent_callback() -```` - +This page is a placeholder that appears only if the documentation is built from +a fork. diff --git a/docs/src/tutorials/linear/multiple_solutions.md b/docs/src/tutorials/linear/multiple_solutions.md index 0584a699329..47b05af9e70 100644 --- a/docs/src/tutorials/linear/multiple_solutions.md +++ b/docs/src/tutorials/linear/multiple_solutions.md @@ -4,172 +4,5 @@ EditURL = "multiple_solutions.jl" # Finding multiple feasible solutions -_This tutorial was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl)._ -[_Download the source as a `.jl` file_](multiple_solutions.jl). - -_Author: James Foster (@jd-foster)_ - -This tutorial demonstrates how to formulate and solve a combinatorial problem -with multiple feasible solutions. In fact, we will see how to find _all_ -feasible solutions to our problem. We will also see how to enforce an -"all-different" constraint on a set of integer variables. - -## Required packages - -This tutorial uses the following packages: - -````@example multiple_solutions -using JuMP -import Gurobi -import Test -```` - -!!! warning - This tutorial uses [Gurobi.jl](@ref) as the solver because it supports - returning multiple feasible solutions, something that open-source MIP - solvers such as HiGHS do not currently support. Gurobi is a commercial - solver and requires a paid license. However, there are free licenses - available for academic and student users. See [Gurobi.jl](@ref) for more - details. - -## Symmetric number squares - -Symmetric [number squares](https://www.futilitycloset.com/2012/12/05/number-squares/) -and their sums often arise in recreational mathematics. Here are a few -examples: -``` - 1 5 2 9 2 3 1 8 5 2 1 9 - 5 8 3 7 3 7 9 0 2 3 8 4 -+ 2 3 4 0 + 1 9 5 6 + 1 8 6 7 -= 9 7 0 6 = 8 0 6 4 = 9 4 7 0 -``` - -Notice how all the digits 0 to 9 are used at least once, the first three rows -sum to the last row, the columns in each are the same as the corresponding -rows (forming a symmetric matrix), and `0` does not appear in the first -column. - -We will answer the question: how many such squares are there? - -## JuMP model - -We now encode the symmetric number square as a JuMP model. First, we need a -symmetric matrix of decision variables between `0` and `9` to represent each -number: - -````@example multiple_solutions -n = 4 -model = Model() -set_silent(model) -@variable(model, 0 <= x_digits[row in 1:n, col in 1:n] <= 9, Int, Symmetric) -```` - -We modify the lower bound to ensure that the first column cannot contain `0`: - -````@example multiple_solutions -set_lower_bound.(x_digits[:, 1], 1) -```` - -Then, we need a constraint that the sum of the first three rows equals the -last row: - -````@example multiple_solutions -@expression(model, x_base_10, x_digits * [1_000, 100, 10, 1]); -@constraint(model, sum(x_base_10[i] for i in 1:n-1) == x_base_10[n]) -```` - -And we use [`MOI.AllDifferent`](@ref) to ensure that each digit is used -exactly once in the upper triangle matrix of `x_digits`: - -````@example multiple_solutions -x_digits_upper = [x_digits[i, j] for j in 1:n for i in 1:j] -@constraint(model, x_digits_upper in MOI.AllDifferent(length(x_digits_upper))); -nothing #hide -```` - -If we optimize this model, we find that Gurobi has returned one solution: - -````@example multiple_solutions -set_optimizer(model, Gurobi.Optimizer) -optimize!(model) -Test.@test is_solved_and_feasible(model) -Test.@test result_count(model) == 1 -solution_summary(model) -```` - -To return multiple solutions, we need to set Gurobi-specific parameters to -enable the [solution pool](https://docs.gurobi.com/projects/optimizer/en/current/features/solutionpool.html). -Moreover, there is a bug in Gurobi that means the solution pool is not -activated if we have already solved the model once. To work around the bug, we -need to reset the optimizer. If you turn the solution pool options on before -the first solve you do not need to reset the optimizer. - -````@example multiple_solutions -set_optimizer(model, Gurobi.Optimizer) -```` - -The first option turns on the exhaustive search mode for multiple solutions: - -````@example multiple_solutions -set_attribute(model, "PoolSearchMode", 2) -```` - -The second option sets a limit for the number of solutions found: - -````@example multiple_solutions -set_attribute(model, "PoolSolutions", 100) -```` - -Here the value 100 is an "arbitrary but large enough" whole number -for our particular model (and in general will depend on the application). - -We can then call `optimize!` and view the results. - -````@example multiple_solutions -optimize!(model) -Test.@test is_solved_and_feasible(model) -solution_summary(model) -```` - -Now Gurobi has found 20 solutions: - -````@example multiple_solutions -Test.@test result_count(model) == 20 -```` - -## Viewing the Results - -Access the various feasible solutions by using the [`value`](@ref) function -with the `result` keyword: - -````@example multiple_solutions -solutions = - [round.(Int, value.(x_digits; result = i)) for i in 1:result_count(model)]; -nothing #hide -```` - -Here we have converted the solution to an integer after rounding off very -small numerical tolerances. - -An example of one feasible solution is: - -````@example multiple_solutions -solutions[1] -```` - -and we can nicely print out all the feasible solutions with - -````@example multiple_solutions -function solution_string(x::Matrix) - header = [" ", " ", "+", "="] - return join([join(vcat(header[i], x[i, :]), " ") for i in 1:4], "\n") -end - -for i in 1:result_count(model) - println("Solution $i: \n", solution_string(solutions[i]), "\n") -end -```` - -The result is the full list of feasible solutions. So the answer to "how many -such squares are there?" turns out to be 20. - +This page is a placeholder that appears only if the documentation is built from +a fork.