Add remove_bridge, print_active_bridges, and document

docs/src/manual/models.md

@@ -493,12 +493,12 @@ By default [`Model`](@ref) will create a `CachingOptimizer` in `AUTOMATIC` mode.
 
 ### LazyBridgeOptimizer
 
-The second layer that JuMP applies automatically is a `LazyBridgeOptimizer`. A
-`LazyBridgeOptimizer` is an MOI layer that attempts to transform the problem
-from the formulation provided by the user into an equivalent problem supported
-by the solver. This may involve adding new variables and constraints to the
-optimizer. The transformations are selected from a set of known recipes called
-_bridges_.
+The second layer that JuMP applies automatically is a [`MOI.Bridges.LazyBridgeOptimizer`](@ref).
+A [`MOI.Bridges.LazyBridgeOptimizer`](@ref) is an MOI layer that attempts to
+transform the problem from the formulation provided by the user into an
+equivalent problem supported by the solver. This may involve adding new
+variables and constraints to the optimizer. The transformations are selected
+from a set of known recipes called _bridges_.
 
 A common example of a bridge is one that splits an interval constraint like
 `@constraint(model, 1 <= x + y <= 2)` into two constraints,
@@ -523,6 +523,11 @@ with model cache MOIU.UniversalFallback{MOIU.Model{Float64}}
 with optimizer A HiGHS model with 0 columns and 0 rows.
 ```
 
+Bridges can be added and removed from a [`MOI.Bridges.LazyBridgeOptimizer`](@ref)
+using [`add_bridge`](@ref) and [`remove_bridge`](@ref). Use
+[`print_active_bridges`](@ref) to see which bridges are used to reformulate the
+model. Read the [Ellipsoid approximation](@ref) tutorial for more details.
+
 ### Unsafe backend
 
 In some advanced use-cases, it is necessary to work with the inner optimization

docs/src/reference/models.md

@@ -82,7 +82,10 @@ MOIU.attach_optimizer(::JuMP.Model)
 ## Bridge tools
 
 ```@docs
+add_bridge
+remove_bridge
 bridge_constraints
+print_active_bridges
 print_bridge_graph
 ```
 

docs/src/tutorials/conic/ellipse_approx.jl

@@ -5,58 +5,51 @@
 
 # # Ellipsoid approximation
 
-# This example considers the problem of computing _extremal ellipsoids_:
-# finding ellipsoids that best approximate a given set.
-# Our example will focus on outer approximations of finite sets
-# of points.
+# This tutorial considers the problem of computing _extremal ellipsoids_:
+# finding ellipsoids that best approximate a given set. As an extension, we show
+# how to use JuMP to inspect the bridges that were used, and how to explore
+# alternative formulations.
 
-# This example comes from section 4.9 "Applications VII: Extremal Ellipsoids"
+# The model comes from Section 4.9 "Applications VII: Extremal Ellipsoids"
 # of the book *Lectures on Modern Convex Optimization* by
 # [Ben-Tal and Nemirovski (2001)](http://epubs.siam.org/doi/book/10.1137/1.9780898718829).
 
 # For a related example, see also the [Minimal ellipses](@ref) tutorial.
 
 # ## Problem formulation
 
-# Suppose that we are given a set ``S`` consisting of ``m`` points in ``n``-dimensional space:
+# Suppose that we are given a set ``\mathcal{S}`` consisting of ``m`` points in
+# ``n``-dimensional space:
 # ```math
 # \mathcal{S} = \{ x_1, \ldots, x_m \} \subset \mathbb{R}^n
 # ```
 # Our goal is to determine an optimal vector ``c \in  \mathbb{R}^n`` and
-# an optimal ``n \times n`` real symmetric matrix ``D`` such that the ellipse
+# an optimal ``n \times n`` real symmetric matrix ``D`` such that the ellipse:
 # ```math
 # E(D, c) = \{ x : (x - c)^\top D ( x - c) \leq 1 \},
 # ```
-# contains ``\mathcal{S}`` and such that this ellipse has
-# the smallest possible volume.
+# contains ``\mathcal{S}`` and has the smallest possible volume.
 
-# The optimal ``D`` and ``c`` are given by the convex semidefinite program
+# The optimal ``D`` and ``c`` are given by the optimization problem:
 # ```math
 # \begin{aligned}
-# \text{maximize }   && \quad (\det(Z))^{\frac{1}{n}}  & \\
-# \text{subject to } && \quad Z \; & \succeq \; 0 & \text{ (PSD) }, & \\
-# && \quad\begin{bmatrix}
-#     s  &  z^\top   \\
-#     z  &  Z        \\
-# \end{bmatrix}
-#  \; & \succeq \; 0 & \text{ (PSD) }, & \\
-# && x_i^\top Z x_i - 2x_i^\top z + s \; & \leq \; 0 &  i=1, \ldots, m &
+# \max        \quad & t                                                                    \\
+# \text{s.t.} \quad & Z \succeq 0 \\
+#                   & \begin{bmatrix} s & z^\top \\ z & Z \\ \end{bmatrix} \succeq 0 \\
+#                   & x_i^\top Z x_i - 2x_i^\top z + s \leq 1    & \quad i=1, \ldots, m \\
+#                   & t \le \sqrt[n]{\det(Z)},
 # \end{aligned}
 # ```
-# with matrix variable ``Z``, vector variable ``z`` and real variables ``t, s``.
-# The optimal solution ``(t_*, Z_*, z_*, s_*)`` gives the optimal ellipse data as
-# ```math
-# D = Z_*, \quad c = Z_*^{-1} z_*.
-# ```
+# where ``D = Z_*`` and ``c = Z_*^{-1} z_*``.
 
 # ## Required packages
 
 # This tutorial uses the following packages:
 
 using JuMP
 import LinearAlgebra
-import Random
 import Plots
+import Random
 import SCS
 import Test  #src
 
@@ -65,10 +58,10 @@ import Test  #src
 # We first need to generate some points to work with.
 
 function generate_point_cloud(
-    m;       # number of 2-dimensional points
-    a = 10,  # scaling in x direction
-    b = 2,   # scaling in y direction
-    rho = deg2rad(30),  # rotation of points around origin
+    m;            # number of 2-dimensional points
+    a = 10,       # scaling in x direction
+    b = 2,        # scaling in y direction
+    rho = π / 6,  # rotation of points around origin
     random_seed = 1,
 )
     rng = Random.MersenneTwister(random_seed)
@@ -79,32 +72,26 @@ function generate_point_cloud(
 end
 
 # For the sake of this example, let's take ``m = 600``:
-S = generate_point_cloud(600)
+S = generate_point_cloud(600);
 
 # We will visualise the points (and ellipse) using the Plots package:
 
-function plot_point_cloud(plot, S; r = 1.1 * maximum(abs.(S)), colour = :green)
-    Plots.scatter!(
-        plot,
-        S[:, 1],
-        S[:, 2];
-        xlim = (-r, r),
-        ylim = (-r, r),
-        label = nothing,
-        c = colour,
-        shape = :x,
-    )
-    return
-end
-
-plot = Plots.plot(; size = (600, 600))
-plot_point_cloud(plot, S)
-plot
+r = 1.1 * maximum(abs.(S))
+plot = Plots.scatter(
+    S[:, 1],
+    S[:, 2];
+    xlim = (-r, r),
+    ylim = (-r, r),
+    label = nothing,
+    c = :green,
+    shape = :x,
+    size = (600, 600),
+)
 
 # ## JuMP formulation
 
-# Now let's build the JuMP model. We'll be able to compute
-# ``D`` and ``c`` after the solve.
+# Now let's build and the JuMP model. We'll compute ``D`` and ``c`` after the
+# solve.
 
 model = Model(SCS.Optimizer)
 ## We need to use a tighter tolerance for this example, otherwise the bounding
@@ -114,29 +101,16 @@ set_silent(model)
 m, n = size(S)
 @variable(model, z[1:n])
 @variable(model, Z[1:n, 1:n], PSD)
-@variable(model, t)
 @variable(model, s)
-
-X = [
-    s z'
-    z Z
-]
-@constraint(model, LinearAlgebra.Symmetric(X) >= 0, PSDCone())
-
-for i in 1:m
-    x = S[i, :]
-    @constraint(model, (x' * Z * x) - (2 * x' * z) + s <= 1)
-end
-
-# We cannot directly represent the objective ``(\det(Z))^{\frac{1}{n}}``, so we introduce
-# the conic reformulation:
-
-@variable(model, nth_root_det_Z)
-@constraint(model, [nth_root_det_Z; vec(Z)] in MOI.RootDetConeSquare(n))
-@objective(model, Max, nth_root_det_Z)
-
-# Now, solve the program:
-
+@variable(model, t)
+@constraint(model, [s z'; z Z] >= 0, PSDCone())
+@constraint(
+    model,
+    [i in 1:m],
+    S[i, :]' * Z * S[i, :] - 2 * S[i, :]' * z + s <= 1,
+)
+@constraint(model, [t; vec(Z)] in MOI.RootDetConeSquare(n))
+@objective(model, Max, t)
 optimize!(model)
 Test.@test termination_status(model) == OPTIMAL    #src
 Test.@test primal_status(model) == FEASIBLE_POINT  #src
@@ -148,6 +122,9 @@ solution_summary(model)
 # ``D`` and ``c``:
 
 D = value.(Z)
+
+#-
+
 c = D \ value.(z)
 
 # Finally, overlaying the solution in the plot we see the minimal volume approximating
@@ -158,10 +135,129 @@ Test.@test isapprox(c, [-3.24802, -1.842825]; atol = 1e-2)                    #s
 
 P = sqrt(D)
 q = -P * c
+data = [tuple(P \ [cos(θ) - q[1], sin(θ) - q[2]]...) for θ in 0:0.05:(2pi+0.05)]
+Plots.plot!(plot, data; c = :crimson, label = nothing)
 
-Plots.plot!(
-    plot,
-    [tuple(P \ [cos(θ) - q[1], sin(θ) - q[2]]...) for θ in 0:0.05:(2pi+0.05)];
-    c = :crimson,
-    label = nothing,
-)
+# ## Alternative formulations
+
+# The formulation of `model` uses [`MOI.RootDetConeSquare`](@ref). However,
+# because SCS does not natively support this cone, JuMP automatically
+# reformulates the problem into an equivalent problem that SCS _does_ support.
+# You can see the reformulation that JuMP chose using [`print_active_bridges`](@ref):
+
+print_active_bridges(model)
+
+# There's a lot going on here, but the first bullet is:
+# ```raw
+# * Unsupported objective: MOI.VariableIndex
+# |  bridged by:
+# |   MOIB.Objective.FunctionizeBridge{Float64}
+# |  introduces:
+# |   * Supported objective: MOI.ScalarAffineFunction{Float64}
+# ```
+# This says that SCS does not support a `MOI.VariableIndex` objective function,
+# and that JuMP used a [`MOI.Bridges.Objective.FunctionizeBridge`](@ref) to
+# convert it into a `MOI.ScalarAffineFunction{Float64}` objective function.
+
+# We can leave JuMP to do the reformulation, or we can rewrite our model to
+# have an objective function that SCS natively supports:
+
+@objective(model, Max, 1.0 * t + 0.0);
+
+# Re-printing the active bridges:
+
+print_active_bridges(model)
+
+# we get `* Supported objective: MOI.ScalarAffineFunction{Float64}`.
+
+# We can manually implement some other reformulations to change our model to
+# something that SCS more closely supports by:
+#
+#  * Replacing the [`MOI.VectorOfVariables`](@ref) in [`MOI.PositiveSemidefiniteConeTriangle`](@ref)
+#    constraint `@variable(model, Z[1:n, 1:n], PSD)` with the
+#    [`MOI.VectorAffineFunction`](@ref) in [`MOI.PositiveSemidefiniteConeTriangle`](@ref)
+#    `@constraint(model, Z >= 0, PSDCone())`.
+#
+#  * Replacing the [`MOI.VectorOfVariables`](@ref) in [`MOI.PositiveSemidefiniteConeSquare`](@ref)
+#    constraint `[s z'; z Z] >= 0, PSDCone()` with the
+#    [`MOI.VectorAffineFunction`](@ref) in [`MOI.PositiveSemidefiniteConeTriangle`](@ref)
+#    `@constraint(model, LinearAlgebra.Symmetric([s z'; z Z]) >= 0, PSDCone())`.
+#
+#  * Replacing the [`MOI.ScalarAffineFunction`](@ref) in [`MOI.GreaterThan`](@ref)
+#    constraints with the vectorized equivalent of
+#    [`MOI.VectorAffineFunction`](@ref) in [`MOI.Nonnegatives`](@ref)
+#
+#  * Replacing the [`MOI.VectorOfVariables`](@ref) in [`MOI.RootDetConeSquare`](@ref)
+#    constraint with [`MOI.VectorAffineFunction`](@ref) in
+#    [`MOI.RootDetConeTriangle`](@ref).
+
+# Note that we still need to bridge [`MOI.PositiveSemidefiniteConeTriangle`](@ref)
+# constraints because SCS uses an internal `SCS.ScaledPSDCone` set instead.
+
+model = Model(SCS.Optimizer)
+set_attribute(model, "eps_rel", 1e-6)
+set_silent(model)
+@variable(model, z[1:n])
+@variable(model, s)
+@variable(model, t)
+## The former @variable(model, Z[1:n, 1:n], PSD)
+@variable(model, Z[1:n, 1:n], Symmetric)
+@constraint(model, Z >= 0, PSDCone())
+## The former [s z'; z Z] >= 0, PSDCone()
+@constraint(model, LinearAlgebra.Symmetric([s z'; z Z]) >= 0, PSDCone())
+## The former constraint S[i, :]' * Z * S[i, :] - 2 * S[i, :]' * z + s <= 1
+f = [1 - S[i, :]' * Z * S[i, :] + 2 * S[i, :]' * z - s for i in 1:m]
+@constraint(model, f in MOI.Nonnegatives(m))
+## The former constraint [t; vec(Z)] in MOI.RootDetConeSquare(n)
+Z_upper = [Z[i, j] for j in 1:n for i in 1:j]
+@constraint(model, 1 * vcat(t, Z_upper) .+ 0 in MOI.RootDetConeTriangle(n))
+## The former @objective(model, Max, t)
+@objective(model, Max, 1 * t + 0)
+optimize!(model)
+Test.@test isapprox(D, value.(Z); atol = 1e-6)  #src
+solve_time_1 = solve_time(model)
+
+# This formulation gives the much smaller graph:
+
+print_active_bridges(model)
+
+# The last bullet shows how JuMP reformulated the [`MOI.RootDetConeTriangle`](@ref)
+# constraint by adding a mix of [`MOI.PositiveSemidefiniteConeTriangle`](@ref)
+# and [`MOI.GeometricMeanCone`](@ref) constraints.
+
+# Because SCS doesn't natively support the [`MOI.GeometricMeanCone`](@ref),
+# these constraints were further bridged using a [`MOI.Bridges.Constraint.GeoMeanToPowerBridge`](@ref)
+# to a series of [`MOI.PowerCone`](@ref) constraints.
+
+# However, there are many other ways that a [`MOI.GeometricMeanCone`](@ref) can
+# be reformulated into something that SCS supports. Let's see what happens if we
+# use [`remove_bridge`](@ref) to remove the
+# [`MOI.Bridges.Constraint.GeoMeanToPowerBridge`](@ref):
+
+remove_bridge(model, MOI.Bridges.Constraint.GeoMeanToPowerBridge)
+optimize!(model)
+
+# This time, the solve took:
+
+solve_time_2 = solve_time(model)
+
+# where previously it took
+
+solve_time_1
+
+# Why was the solve time different?
+
+print_active_bridges(model)
+
+# This time, JuMP used a [`MOI.Bridges.Constraint.GeoMeanBridge`](@ref) to
+# reformulate the constraint into a set of [`MOI.RotatedSecondOrderCone`](@ref)
+# constraints, which were further reformulated into a set of supported
+# [`MOI.SecondOrderCone`](@ref) constraints.
+
+# Since the two models are equivalent, we can conclude that for this particular
+# model, the [`MOI.SecondOrderCone`](@ref) formulation is more efficient.
+
+# In general though, the performance of a particular reformulation is problem-
+# and solver-specific. Therefore, JuMP chooses to minimize the number of bridges in
+# the default reformulation, leaving you to explore alternative formulations
+# using the tools and techniques shown in this tutorial.