From 6fde90d4b5ec36226331cb48c82589978accbc97 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Fri, 23 Feb 2024 05:44:00 +0000 Subject: [PATCH] build based on 5126334 --- previews/PR663/.documenter-siteinfo.json | 1 + previews/PR663/FAQ.html | 2 + .../PR663/ad_author/call_back_into_ad.html | 7 + previews/PR663/ad_author/opt_out.html | 7 + .../PR663/ad_author/use_in_ad_system.html | 2 + previews/PR663/api.html | 109 +++ previews/PR663/assets/documenter.js | 893 ++++++++++++++++++ previews/PR663/assets/indigo.css | 88 ++ previews/PR663/assets/logo.png | Bin 0 -> 31296 bytes previews/PR663/assets/logo.svg | 27 + previews/PR663/assets/make_logo.jl | 84 ++ .../PR663/assets/themes/documenter-dark.css | 7 + .../PR663/assets/themes/documenter-light.css | 9 + previews/PR663/assets/themeswap.js | 84 ++ previews/PR663/assets/warner.js | 52 + .../PR663/design/changing_the_primal.html | 104 ++ previews/PR663/design/many_tangents.html | 33 + previews/PR663/index.html | 65 ++ previews/PR663/maths/arrays.html | 328 +++++++ previews/PR663/maths/complex.html | 52 + .../PR663/maths/nondiff_points-359f64e4.svg | 43 + .../PR663/maths/nondiff_points-a4828821.svg | 37 + .../PR663/maths/nondiff_points-bd317c35.svg | 39 + .../PR663/maths/nondiff_points-c6064fd4.svg | 39 + .../PR663/maths/nondiff_points-d1529e0d.svg | 41 + .../PR663/maths/nondiff_points-f10bd981.svg | 39 + .../PR663/maths/nondiff_points-fea88105.svg | 43 + previews/PR663/maths/nondiff_points.html | 2 + previews/PR663/maths/propagators.html | 31 + .../rule_author/converting_zygoterules.html | 13 + previews/PR663/rule_author/debug_mode.html | 2 + previews/PR663/rule_author/example.html | 26 + previews/PR663/rule_author/intro.html | 2 + .../rule_author/rule_definition_tools.html | 9 + .../superpowers/gradient_accumulation.html | 18 + .../superpowers/mutation_support.html | 17 + .../rule_author/superpowers/opt_out.html | 10 + .../rule_author/superpowers/projectto.html | 18 + .../rule_author/superpowers/ruleconfig.html | 18 + previews/PR663/rule_author/tangents.html | 2 + previews/PR663/rule_author/testing.html | 2 + .../PR663/rule_author/tips_for_packages.html | 17 + .../which_functions_need_rules.html | 64 ++ .../PR663/rule_author/writing_good_rules.html | 112 +++ previews/PR663/search_index.js | 3 + previews/PR663/siteinfo.js | 1 + previews/PR663/videos.html | 8 + 47 files changed, 2610 insertions(+) create mode 100644 previews/PR663/.documenter-siteinfo.json create mode 100644 previews/PR663/FAQ.html create mode 100644 previews/PR663/ad_author/call_back_into_ad.html create mode 100644 previews/PR663/ad_author/opt_out.html create mode 100644 previews/PR663/ad_author/use_in_ad_system.html create mode 100644 previews/PR663/api.html create mode 100644 previews/PR663/assets/documenter.js create mode 100644 previews/PR663/assets/indigo.css create mode 100644 previews/PR663/assets/logo.png create mode 100644 previews/PR663/assets/logo.svg create mode 100644 previews/PR663/assets/make_logo.jl create mode 100644 previews/PR663/assets/themes/documenter-dark.css create mode 100644 previews/PR663/assets/themes/documenter-light.css create mode 100644 previews/PR663/assets/themeswap.js create mode 100644 previews/PR663/assets/warner.js create mode 100644 previews/PR663/design/changing_the_primal.html create mode 100644 previews/PR663/design/many_tangents.html create mode 100644 previews/PR663/index.html create mode 100644 previews/PR663/maths/arrays.html create mode 100644 previews/PR663/maths/complex.html create mode 100644 previews/PR663/maths/nondiff_points-359f64e4.svg create mode 100644 previews/PR663/maths/nondiff_points-a4828821.svg create mode 100644 previews/PR663/maths/nondiff_points-bd317c35.svg create mode 100644 previews/PR663/maths/nondiff_points-c6064fd4.svg create mode 100644 previews/PR663/maths/nondiff_points-d1529e0d.svg create mode 100644 previews/PR663/maths/nondiff_points-f10bd981.svg create mode 100644 previews/PR663/maths/nondiff_points-fea88105.svg create mode 100644 previews/PR663/maths/nondiff_points.html create mode 100644 previews/PR663/maths/propagators.html create mode 100644 previews/PR663/rule_author/converting_zygoterules.html create mode 100644 previews/PR663/rule_author/debug_mode.html create mode 100644 previews/PR663/rule_author/example.html create mode 100644 previews/PR663/rule_author/intro.html create mode 100644 previews/PR663/rule_author/rule_definition_tools.html create mode 100644 previews/PR663/rule_author/superpowers/gradient_accumulation.html create mode 100644 previews/PR663/rule_author/superpowers/mutation_support.html create mode 100644 previews/PR663/rule_author/superpowers/opt_out.html create mode 100644 previews/PR663/rule_author/superpowers/projectto.html create mode 100644 previews/PR663/rule_author/superpowers/ruleconfig.html create mode 100644 previews/PR663/rule_author/tangents.html create mode 100644 previews/PR663/rule_author/testing.html create mode 100644 previews/PR663/rule_author/tips_for_packages.html create mode 100644 previews/PR663/rule_author/which_functions_need_rules.html create mode 100644 previews/PR663/rule_author/writing_good_rules.html create mode 100644 previews/PR663/search_index.js create mode 100644 previews/PR663/siteinfo.js create mode 100644 previews/PR663/videos.html diff --git a/previews/PR663/.documenter-siteinfo.json b/previews/PR663/.documenter-siteinfo.json new file mode 100644 index 000000000..ddd5c1401 --- /dev/null +++ b/previews/PR663/.documenter-siteinfo.json @@ -0,0 +1 @@ +{"documenter":{"julia_version":"1.6.7","generation_timestamp":"2024-02-23T05:43:52","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/previews/PR663/FAQ.html b/previews/PR663/FAQ.html new file mode 100644 index 000000000..279ce4d9f --- /dev/null +++ b/previews/PR663/FAQ.html @@ -0,0 +1,2 @@ + +FAQ · ChainRules
GitHub

FAQ

What is up with the different symbols?

Δx, ∂x, dx

ChainRules uses these perhaps atypically. As a notation that is the same across propagators, regardless of direction (in contrast see ẋ and x̄ below).

  • Δx is the input to a propagator, (i.e a seed for a pullback; or a perturbation for a pushforward).
  • ∂x is the output of a propagator.
  • dx could be either input or output.

dots and bars: $\dot{y} = \dfrac{∂y}{∂x} = \overline{x}$

  • v̇ is a derivative of the input moving forward: $v̇ = \frac{∂v}{∂x}$ for input $x$, intermediate value $v$.
  • v̄ is a derivative of the output moving backward: $v̄ = \frac{∂y}{∂v}$ for output $y$, intermediate value $v$.

others

  • Ω is often used as the return value of the function. Especially, but not exclusively, for scalar functions.
    • ΔΩ is thus a seed for the pullback.
    • ∂Ω is thus the output of a pushforward.

Why does rrule return the primal function evaluation?

You might wonder why frule(f, x) returns f(x) and the derivative of f at x, and similarly for rrule returning f(x) and the pullback for f at x. Why not just return the pushforward/pullback, and let the user call f(x) to get the answer separately?

There are three reasons the rules also calculate the f(x).

  1. For some rules an alternative way of calculating f(x) can give the same answer while also generating intermediate values that can be used in the calculations required to propagate the derivative.
  2. For many rrules the output value is used in the definition of the pullback. For example tan, sigmoid etc.
  3. For some frules there exists a single, non-separable operation that will compute both derivative and primal result. For example, this is the case for many of the methods for differential equation sensitivity analysis.

For more information and examples see the design notes on changing the primal.

Where are the derivatives for keyword arguments?

Pullbacks do not return a sensitivity for keyword arguments; similarly, pushforwards do not accept a perturbation for keyword arguments. This is because in practice functions are very rarely differentiable with respect to keyword arguments.

As a rule, keyword arguments tend to control side-effects, like logging verbosity, or to be functionality-changing to perform a different operation, e.g. dims=3, and thus not differentiable.

To the best of our knowledge no Julia AD system, with support for the definition of custom primitives, supports differentiating with respect to keyword arguments. At some point in the future ChainRules may support these. Maybe.

What is the difference between ZeroTangent and NoTangent ?

ZeroTangent and NoTangent act almost exactly the same in practice: they result in no change whenever added to anything. Odds are if you write a rule that returns the wrong one everything will just work fine. We provide both to allow for clearer writing of rules, and easier debugging.

ZeroTangent() represents the fact that if one perturbs (adds a small change to) the matching primal, there will be no change in the behaviour of the primal function. For example, in fst(x, y) = x, the derivative of fst with respect to y is ZeroTangent(). fst(10, 5) == 10 and if we add 0.1 to 5 we still get fst(10, 5.1) == 10.

NoTangent() represents the fact that if one perturbs the matching primal, the primal function will now error. For example, in access(xs, n) = xs[n], the derivative of access with respect to n is NoTangent(). access([10, 20, 30], 2) == 20, but if we add 0.1 to 2 we get access([10, 20, 30], 2.1) which errors as indexing can't be applied at fractional indexes.

Why do I get an error involving nothing?

When no custom frule or rrule exists, if you try to call one of those, it will return nothing by default. As a result, you may encounter errors like

MethodError: no method matching iterate(::Nothing)

Sometimes you think you have implemented the right rule, but it is called with a slightly different set of arguments than you expected. You can use Cthulhu.jl to dive into the call stack and figure out which method you are missing.

An alternative is to call back into AD: read the documentation on rule configuration to know more.

When to use ChainRules vs ChainRulesCore?

ChainRulesCore.jl is a light-weight dependency for defining rules for functions in your packages, without you needing to depend on ChainRules.jl itself. It has almost no dependencies of its own. If you only want to define rules, not use them, then you probably only want to load ChainRulesCore.jl.

ChainRules.jl provides the full functionality for AD systems. In particular, it has all the rules for Base Julia and the standard libraries. It is thus a much heavier package to load. AD systems making use of frules and rrules should load ChainRules.jl.

Where should I put my rules?

We recommend adding custom rules to your own packages with ChainRulesCore.jl. It is good to have them in the same package that defines the original function. This avoids type-piracy, and makes it easy to keep in-sync. ChainRulesCore is a very light-weight dependency.

How do I test my rules?

You can use ChainRulesTestUtils.jl to test your custom rules. ChainRulesTestUtils.jl has some dependencies, so it is a separate package from ChainRulesCore.jl. This means your package can depend on the light-weight ChainRulesCore.jl, and make ChainRulesTestUtils.jl a test-only dependency.

Remember to read the section On writing good rrule / frule methods.

Is removing a thunk a breaking change?

Removing thunks is not considered a breaking change. This is because (in principle) removing them changes the implementation of the values returned by an rrule, not the value that they represent. This is morally the same as similar issues discussed in ColPrac, such as details of floating point arithmetic changing.

On a practical level, it's important that this is the case because thunks are a bit of a hack, and over time it is hoped that the need for them will reduce, as they increase code-complexity and place additional stress on the compiler.

Where can I learn more about AD ?

There are not so many truly excellent learning resources for autodiff out there in the world, which is a bit sad. The list here is incomplete, but is vetted for quality.

diff --git a/previews/PR663/ad_author/call_back_into_ad.html b/previews/PR663/ad_author/call_back_into_ad.html new file mode 100644 index 000000000..988e80340 --- /dev/null +++ b/previews/PR663/ad_author/call_back_into_ad.html @@ -0,0 +1,7 @@ + +Support calling back into ADs · ChainRules
GitHub

Declaring support for calling back into ADs

To declare support or lack of support for forward and reverse-mode, use the two pairs of complementary types. For reverse mode: HasReverseMode, NoReverseMode. For forwards mode: HasForwardsMode, NoForwardsMode. AD systems that support any calling back into AD should have one from each set.

If an AD HasReverseMode, then it must define rrule_via_ad for that RuleConfig subtype. Similarly, if an AD HasForwardsMode then it must define frule_via_ad for that RuleConfig subtype.

For example:

struct MyReverseOnlyADRuleConfig <: RuleConfig{Union{HasReverseMode, NoForwardsMode}} end
+
+function ChainRulesCore.rrule_via_ad(::MyReverseOnlyADRuleConfig, f, args...)
+    ...
+    return y, pullback
+end

Note that it is not actually required that the same AD is used for forward and reverse. For example Nabla.jl is a reverse mode AD. It might declare that it HasForwardsMode, and then define a wrapper around ForwardDiff.jl in order to provide that capacity.

diff --git a/previews/PR663/ad_author/opt_out.html b/previews/PR663/ad_author/opt_out.html new file mode 100644 index 000000000..8dd7fbb25 --- /dev/null +++ b/previews/PR663/ad_author/opt_out.html @@ -0,0 +1,7 @@ + +Support opting out of rules · ChainRules
GitHub

Support opting out of rules

We provide two ways to know that a rule has been opted out of.

rrule / frule returns nothing

@opt_out defines a frule or rrule matching the signature that returns nothing.

If you are in a position to generate code, in response to values returned by function calls then you can do something like:

res = rrule(f, xs)
+if res === nothing
+    y, pullback = perform_ad_via_decomposition(r, xs)  # do AD without hitting the rrule
+else
+    y, pullback = res
+end

The Julia compiler will specialize based on inferring the return type of rrule, and so can remove that branch.

no_rrule / no_frule has a method

@opt_out also defines a method for ChainRulesCore.no_frule or ChainRulesCore.no_rrule. The body of this method doesn't matter, what matters is that it is a method-table. A simple thing you can do with this is not support opting out. To do this, filter all methods from the rrule/frule method table that also occur in the no_frule/no_rrule table. This will thus avoid ever hitting an rrule/frule that returns nothing (and thus prevents your library from erroring). This is easily done, though it does mean ignoring the user's stated desire to opt out of the rule.

More complex you can use this to generate code that triggers your AD. If for a given signature there is a more specific method in the no_rrule/no_frule method-table, than the one that would be hit from the rrule/frule table (Excluding the one that exactly matches which will return nothing) then you know that the rule should not be used. You can, likely by looking at the primal method table, workout which method you would have it if the rule had not been defined, and then invoke it.

diff --git a/previews/PR663/ad_author/use_in_ad_system.html b/previews/PR663/ad_author/use_in_ad_system.html new file mode 100644 index 000000000..1138094a7 --- /dev/null +++ b/previews/PR663/ad_author/use_in_ad_system.html @@ -0,0 +1,2 @@ + +Usage in AD · ChainRules
GitHub

Using ChainRules in your AD system

This section is for authors of AD systems. It assumes a pretty solid understanding of both Julia and automatic differentiation. It explains how to make use of ChainRule's "rulesets" (frules, rrules,) to avoid having to code all your own AD primitives / custom sensitives.

There are 3 main ways to access ChainRules rule sets in your AutoDiff system.

  1. Operator Overloading Generation
    • Use ChainRulesOverloadGeneration.jl.
    • This is primarily intended for operator overloading based AD systems which will generate overloads for primal functions based for their overloaded types based on the existence of an rrule/frule.
    • A source code generation based AD can also use this by overloading their transform generating function directly so as not to recursively generate a transform but to just return the rule.
    • This does not play nice with Revise.jl, adding or modifying rules in loaded files will not be reflected until a manual refresh, and deleting rules will not be reflected at all.
  2. Source code transform based on inserting branches that check of rrule/frule return nothing
    • If the rrule/frule returns a rule result then use it, if it returns nothing then do normal AD path.
    • In theory type inference optimizes these branches out; in practice it may not.
    • This is a fairly simple Cassette overdub (or similar) of all calls, and is suitable for overloading based AD or source code transformation.
  3. Source code transform based on rrule/frule method-table
    • If an applicable rrule/frule exists in the method table then use it, else generate normal AD path.
    • This avoids having branches in your generated code.
    • This requires maintaining your own back-edges.
    • This is pretty hardcore even by the standard of source code transformations.
diff --git a/previews/PR663/api.html b/previews/PR663/api.html new file mode 100644 index 000000000..957b28f6e --- /dev/null +++ b/previews/PR663/api.html @@ -0,0 +1,109 @@ + +API · ChainRules
GitHub

API Documentation

Rules

ChainRulesCore.frule — Method
frule([::RuleConfig,] (Δf, Δx...), f, x...)

Expressing the output of f(x...) as Ω, return the tuple:

(Ω, ΔΩ)

The second return value is the tangent w.r.t. the output.

If no method matching frule((Δf, Δx...), f, x...) has been defined, then return nothing.

Examples:

unary input, unary output scalar function:

julia> dself = NoTangent();
+
+julia> x = rand()
+0.8236475079774124
+
+julia> sinx, Δsinx = frule((dself, 1), sin, x)
+(0.7336293678134624, 0.6795498147167869)
+
+julia> sinx == sin(x)
+true
+
+julia> Δsinx == cos(x)
+true

Unary input, binary output scalar function:

julia> sincosx, Δsincosx = frule((dself, 1), sincos, x);
+
+julia> sincosx == sincos(x)
+true
+
+julia> Δsincosx[1] == cos(x)
+true
+
+julia> Δsincosx[2] == -sin(x)
+true

Note that techically speaking julia does not have multiple output functions, just functions that return a single output that is iterable, like a Tuple. So this is actually a Tangent:

julia> Δsincosx
+Tangent{Tuple{Float64, Float64}}(0.6795498147167869, -0.7336293678134624)

The optional RuleConfig option allows specifying frules only for AD systems that support given features. If not needed, then it can be omitted and the frule without it will be hit as a fallback. This is the case for most rules.

See also: rrule, @scalar_rule, RuleConfig

source
ChainRulesCore.rrule — Method
rrule([::RuleConfig,] f, x...)

Expressing x as the tuple (x₁, x₂, ...) and the output tuple of f(x...) as Ω, return the tuple:

(Ω, (Ω̄₁, Ω̄₂, ...) -> (s̄elf, x̄₁, x̄₂, ...))

Where the second return value is the the propagation rule or pullback. It takes in cotangents corresponding to the outputs (x̄₁, x̄₂, ...), and s̄elf, the internal values of the function itself (for closures)

If no method matching rrule(f, xs...) has been defined, then return nothing.

Examples:

unary input, unary output scalar function:

julia> x = rand();
+
+julia> sinx, sin_pullback = rrule(sin, x);
+
+julia> sinx == sin(x)
+true
+
+julia> sin_pullback(1) == (NoTangent(), cos(x))
+true

binary input, unary output scalar function:

julia> x, y = rand(2);
+
+julia> hypotxy, hypot_pullback = rrule(hypot, x, y);
+
+julia> hypotxy == hypot(x, y)
+true
+
+julia> hypot_pullback(1) == (NoTangent(), (x / hypot(x, y)), (y / hypot(x, y)))
+true

The optional RuleConfig option allows specifying rrules only for AD systems that support given features. If not needed, then it can be omitted and the rrule without it will be hit as a fallback. This is the case for most rules.

See also: frule, @scalar_rule, RuleConfig

source

Rule Definition Tools

ChainRulesCore.@non_differentiable — Macro
@non_differentiable(signature_expression)

A helper to make it easier to declare that a method is not differentiable. This is a short-hand for defining an frule and rrule that return NoTangent() for all partials (even for the function s̄elf-partial itself)

Keyword arguments should not be included.

julia> @non_differentiable Base.:(==)(a, b)
+
+julia> _, pullback = rrule(==, 2.0, 3.0);
+
+julia> pullback(1.0)
+(NoTangent(), NoTangent(), NoTangent())

You can place type-constraints in the signature:

julia> @non_differentiable Base.length(xs::Union{Number, Array})
+
+julia> frule((ZeroTangent(), 1), length, [2.0, 3.0])
+(2, NoTangent())
Warning

This helper macro covers only the simple common cases. It does not support where-clauses. For these you can declare the rrule and frule directly

source
ChainRulesCore.@opt_out — Macro
@opt_out frule([config], _, f, args...)
+@opt_out rrule([config], f, args...)

This allows you to opt-out of an frule or an rrule by providing a more specific method, that says to use the AD system to differentiate it.

For example, consider some function foo(x::AbtractArray). In general, you know an efficient and generic way to implement its rrule. You do so, (likely making use of ProjectTo). But it actually turns out that for some FancyArray type it is better to let the AD do its thing.

Then you would write something like:

function rrule(::typeof(foo), x::AbstractArray)
+    foo_pullback(ȳ) = ...
+    return foo(x), foo_pullback
+end
+
+@opt_out rrule(::typeof(foo), ::FancyArray)

This will generate an rrule that returns nothing, and will also add a similar entry to ChainRulesCore.no_rrule.

Similar applies for frule and ChainRulesCore.no_frule

For more information see the documentation on opting out of rules.

source
ChainRulesCore.@scalar_rule — Macro
@scalar_rule(f(x₁, x₂, ...),
+             @setup(statement₁, statement₂, ...),
+             (∂f₁_∂x₁, ∂f₁_∂x₂, ...),
+             (∂f₂_∂x₁, ∂f₂_∂x₂, ...),
+             ...)

A convenience macro that generates simple scalar forward or reverse rules using the provided partial derivatives. Specifically, generates the corresponding methods for frule and rrule:

function ChainRulesCore.frule((NoTangent(), Δx₁, Δx₂, ...), ::typeof(f), x₁::Number, x₂::Number, ...)
+    Ω = f(x₁, x₂, ...)
+    $(statement₁, statement₂, ...)
+    return Ω, (
+            (∂f₁_∂x₁ * Δx₁ + ∂f₁_∂x₂ * Δx₂ + ...),
+            (∂f₂_∂x₁ * Δx₁ + ∂f₂_∂x₂ * Δx₂ + ...),
+            ...
+        )
+end
+
+function ChainRulesCore.rrule(::typeof(f), x₁::Number, x₂::Number, ...)
+    Ω = f(x₁, x₂, ...)
+    $(statement₁, statement₂, ...)
+    return Ω, ((ΔΩ₁, ΔΩ₂, ...)) -> (
+            NoTangent(),
+            ∂f₁_∂x₁ * ΔΩ₁ + ∂f₂_∂x₁ * ΔΩ₂ + ...),
+            ∂f₁_∂x₂ * ΔΩ₁ + ∂f₂_∂x₂ * ΔΩ₂ + ...),
+            ...
+        )
+end

If no type constraints in f(x₁, x₂, ...) within the call to @scalar_rule are provided, each parameter in the resulting frule/rrule definition is given a type constraint of Number. Constraints may also be explicitly be provided to override the Number constraint, e.g. f(x₁::Complex, x₂), which will constrain x₁ to Complex and x₂ to Number.

At present this does not support defining for closures/functors. Thus in reverse-mode, the first returned partial, representing the derivative with respect to the function itself, is always NoTangent(). And in forward-mode, the first input to the returned propagator is always ignored.

The result of f(x₁, x₂, ...) is automatically bound to Ω. This allows the primal result to be conveniently referenced (as Ω) within the derivative/setup expressions.

This macro assumes complex functions are holomorphic. In general, for non-holomorphic functions, the frule and rrule must be defined manually.

If the derivative is one, (e.g. for identity functions) true can be used as the most general multiplicative identity.

The @setup argument can be elided if no setup code is need. In other words:

@scalar_rule(f(x₁, x₂, ...),
+             (∂f₁_∂x₁, ∂f₁_∂x₂, ...),
+             (∂f₂_∂x₁, ∂f₂_∂x₂, ...),
+             ...)

is equivalent to:

@scalar_rule(f(x₁, x₂, ...),
+             @setup(nothing),
+             (∂f₁_∂x₁, ∂f₁_∂x₂, ...),
+             (∂f₂_∂x₁, ∂f₂_∂x₂, ...),
+             ...)

For examples, see ChainRules' rulesets directory.

See also: frule, rrule.

source

Tangent Types

ChainRulesCore.AbstractZero — Type
AbstractZero <: AbstractTangent

Supertype for zero-like tangents—i.e., tangents that act like zero when added or multiplied to other values. If an AD system encounters a propagator that takes as input only subtypes of AbstractZero, then it can stop performing AD operations. All propagators are linear functions, and thus the final result will be zero.

All AbstractZero subtypes are singleton types. There are two of them: ZeroTangent() and NoTangent().

source
ChainRulesCore.NoTangent — Type
NoTangent() <: AbstractZero

This tangent indicates that the derivative does not exist. It is the tangent type for primal types that are not differentiable, such as integers or booleans (when they are not being used to represent floating-point values). The only valid way to perturb such values is to not change them at all. As a consequence, NoTangent is functionally identical to ZeroTangent(), but it provides additional semantic information.

Adding NoTangent() to a primal is generally wrong: gradient-based methods cannot be used to optimize over discrete variables. An optimization package making use of this might want to check for such a case.

Note

This does not indicate that the derivative is not implemented, but rather that mathematically it is not defined.

This mostly shows up as the derivative with respect to dimension, index, or size arguments.

    function rrule(fill, x, len::Int)
+        y = fill(x, len)
+        fill_pullback(ȳ) = (NoTangent(), @thunk(sum(Ȳ)), NoTangent())
+        return y, fill_pullback
+    end
source
ChainRulesCore.ZeroTangent — Type
ZeroTangent() <: AbstractZero

The additive identity for tangents. This is basically the same as 0. A derivative of ZeroTangent() does not propagate through the primal function.

source
ChainRulesCore.zero_tangent — Function
zero_tangent(primal)

This returns an appropriate zero tangent suitable for accumulating tangents of the primal. For mutable composites types this is a structural MutableTangent For Arrays, it is applied recursively for each element. For other types, in particular immutable types, we do not make promises beyond that it will be iszero and suitable for accumulating against. For types without a tangent space (e.g. singleton structs) this returns NoTangent(). In general, it is more likely to produce a structural tangent.

!!! warning Exprimental zero_tangentis an experimental feature, and is part of the mutation support featureset. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore. Exactly how it should be used (e.g. is it forward-mode only?)

source
ChainRulesCore.MutableTangent — Type
MutableTangent{P}(fields) <: StructuralTangent{P} <: AbstractTangent

This type represents the tangent to a mutable struct. It itself is also mutable.

!!! warning Exprimental MutableTangent is an experimental feature, and is part of the mutation support featureset. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore. Exactly how it should be used (e.g. is it forward-mode only?)

!!! warning Do not directly mess with the tangent backing data It is relatively straight forward for a forwards-mode AD to work correctly in the presence of mutation and aliasing of primal values. However, this requires that the tangent is aliased in turn and conversely that it is copied when the primal is). If you seperately alias the backing data, etc by using the internal ChainRulesCore.backing function you can break this.

source
ChainRulesCore.StructuralTangent — Type
StructuralTangent{P} <: AbstractTangent

Representing the type of the tangent of a struct P (or a Tuple/NamedTuple). as an object with mirroring fields.

!!!!!! warning Exprimental StructuralTangent is an experimental feature, and is part of the mutation support featureset. The StructuralTangent constructor returns a MutableTangent for mutable structs. MutableTangent is an experimental feature. Thus use of StructuralTangent (rather than Tangent directly) is also experimental. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore.

source
ChainRulesCore.Tangent — Type
Tangent{P, T} <: StructuralTangent{P} <: AbstractTangent

This type represents the tangent for a struct/NamedTuple, or Tuple. P is the the corresponding primal type that this is a tangent for.

Tangent{P} should have fields (technically properties), that match to a subset of the fields of the primal type; and each should be a tangent type matching to the primal type of that field. Fields of the P that are not present in the Tangent are treated as Zero.

T is an implementation detail representing the backing data structure. For Tuple it will be a Tuple, and for everything else it will be a NamedTuple. It should not be passed in by user.

For Tangents of Tuples, iterate and getindex are overloaded to behave similarly to for a tuple. For Tangents of structs, getproperty is overloaded to allow for accessing values via tangent.fieldname. Any fields not explictly present in the Tangent are treated as being set to ZeroTangent(). To make a Tangent have all the fields of the primal the canonicalize function is provided.

source
ChainRulesCore.canonicalize — Method
canonicalize(tangent::Tangent{P}) -> Tangent{P}

Return the canonical Tangent for the primal type P. The property names of the returned Tangent match the field names of the primal, and all fields of P not present in the input tangent are explictly set to ZeroTangent().

source
ChainRulesCore.InplaceableThunk — Type
InplaceableThunk(add!::Function, val::Thunk)

A wrapper for a Thunk, that allows it to define an inplace add! function.

add! should be defined such that: ithunk.add!(Δ) = Δ .+= ithunk.val but it should do this more efficently than simply doing this directly. (Otherwise one can just use a normal Thunk).

Most operations on an InplaceableThunk treat it just like a normal Thunk; and destroy its inplacability.

source
ChainRulesCore.Thunk — Type
Thunk(()->v)

A thunk is a deferred computation. It wraps a zero argument closure that when invoked returns a tangent. @thunk(v) is a macro that expands into Thunk(()->v).

To evaluate the wrapped closure, call unthunk which is a no-op when the argument is not a Thunk.

julia> t = @thunk(3)
+Thunk(var"#4#5"())
+
+julia> unthunk(t)
+3

When to @thunk?

When writing rrules (and to a lesser exent frules), it is important to @thunk appropriately. Propagation rules that return multiple derivatives may not have all deriviatives used. By @thunking the work required for each derivative, they then compute only what is needed.

How do thunks prevent work?

If we have res = pullback(...) = @thunk(f(x)), @thunk(g(x)) then if we did dx + res[1] then only f(x) would be evaluated, not g(x). Also if we did ZeroTangent() * res[1] then the result would be ZeroTangent() and f(x) would not be evaluated.

So why not thunk everything?

@thunk creates a closure over the expression, which (effectively) creates a struct with a field for each variable used in the expression, and call overloaded.

Do not use @thunk if this would be equal or more work than actually evaluating the expression itself. This is commonly the case for scalar operators.

For more details see the manual section on using thunks effectively.

source
ChainRulesCore.unthunk — Method
unthunk(x)

On AbstractThunks this removes 1 layer of thunking. On any other type, it is the identity operation.

source
ChainRulesCore.@not_implemented — Macro
@not_implemented(info)

Create a tangent that indicates that the derivative is not implemented.

The info should be useful information about the missing tangent for debugging.

Note

This macro should be used only if the automatic differentiation would error otherwise. It is mostly useful if the function has multiple inputs or outputs, and one has worked out analytically and implemented some but not all tangents.

Note

It is good practice to include a link to a GitHub issue about the missing tangent in the debugging information.

source

Accumulation

ChainRulesCore.add!! — Function
add!!(x, y)

Returns x+y, potentially mutating x in-place to hold this value. This avoids allocations when x can be mutated in this way.

source
add!!(x, t::InplacableThunk)

The specialization of add!! for InplaceableThunk promises to only call t.add! on x if x is suitably mutable; otherwise it will be out of place.

source
ChainRulesCore.is_inplaceable_destination — Function
is_inplaceable_destination(x) -> Bool

Returns true if x is suitable for for storing inplace accumulation of gradients. For arrays this means x .= y will mutate x, if y is an appropriate tangent.

Here "appropriate" means that y cannot be complex unless x is too, and that for structured matrices like x isa Diagonal, y shares this structure.

history

Wrapper array types should overload this function if they can be written into. Before ChainRulesCore 1.16, it would guess true for most wrappers based on parent, but this is not safe, e.g. it will lead to an error with ReadOnlyArrays.jl.

There must always be a correct non-mutating path, so in uncertain cases, this function returns false.

source

RuleConfig

ChainRulesCore.RuleConfig — Type
RuleConfig{T}

The configuration for what rules to use. T: traits. This should be a Union of all special traits needed for rules to be allowed to be defined for your AD. If nothing special this should be set to Union{}.

AD authors should define a subtype of RuleConfig to use when calling frule/rrule.

Rule authors can dispatch on this config when defining rules. For example:

# only define rrule for `pop!` on AD systems where mutation is supported.
+rrule(::RuleConfig{>:SupportsMutation}, typeof(pop!), ::Vector) = ...
+
+# this definition of map is for any AD that defines a forwards mode
+rrule(conf::RuleConfig{>:HasForwardsMode}, typeof(map), ::Vector) = ...
+
+# this definition of map is for any AD that only defines a reverse mode.
+# It is not as good as the rrule that can be used if the AD defines a forward-mode as well.
+rrule(conf::RuleConfig{>:Union{NoForwardsMode, HasReverseMode}}, typeof(map), ::Vector) = ...

For more details see rule configurations and calling back into AD.

source

ProjectTo

ChainRulesCore.ProjectTo — Type
(p::ProjectTo{T})(dx)

Projects the tangent dx onto a specific tangent space.

The type T is meant to encode the largest acceptable space, so usually this enforces p(dx)::T. But some subspaces which aren't subtypes of T may be allowed, and in particular dx::AbstractZero always passes through.

Usually T is the "outermost" part of the type, and p stores additional properties such as projectors for each constituent field. Arrays have either one projector p.element expressing the element type for an array of numbers, or else an array of projectors p.elements. These properties can be supplied as keyword arguments on construction, p = ProjectTo{T}(; field=data, element=Projector(x)). For each T in use, corresponding methods should be written for ProjectTo{T}(dx) with nonzero dx.

When called on dx::Thunk, the projection is inserted into the thunk.

source

Ignoring gradients

ChainRulesCore.ignore_derivatives — Function
ignore_derivatives(f::Function)

Tells the AD system to ignore the gradients of the wrapped closure. The primal computation (forward pass) is executed normally.

ignore_derivatives() do
+    value = rand()
+    push!(collection, value)
+end

Using this incorrectly could lead to incorrect gradients. For example, the following function will have zero gradients with respect to its argument:

function wrong_grads(x)
+    y = ones(3)
+    ignore_derivatives() do
+        push!(y, x)
+    end
+    return sum(y)
+end
source
ignore_derivatives(x)

Tells the AD system to ignore the gradients of the argument. Can be used to avoid unnecessary computation of gradients.

ignore_derivatives(x) * w
source

Internal

ChainRulesCore.AbstractTangent — Type

The subtypes of AbstractTangent define a custom "algebra" for chain rule evaluation that attempts to factor various features like complex derivative support, broadcast fusion, zero-elision, etc. into nicely separated parts.

In general a tangent type is the type of a derivative of a value. The type of the value is for contrast called the primal type. Differential types correspond to primal types, although the relation is not one-to-one. Subtypes of AbstractTangent are not the only tangent types. In fact for the most common primal types, such as Real or AbstractArray{Real} the the tangent type is the same as the primal type.

In a circular definition: the most important property of a tangent is that it should be able to be added (by defining +) to another tangent of the same primal type. That allows for gradients to be accumulated.

It generally also should be able to be added to a primal to give back another primal, as this facilitates gradient descent.

All subtypes of AbstractTangent implement the following operations:

  • +(a, b): linearly combine tangent a and tangent b
  • *(a, b): multiply the tangent b by the scaling factor a
  • Base.zero(x) = ZeroTangent(): a zero.

Further, they often implement other linear operators, such as conj, adjoint, dot. Pullbacks/pushforwards are linear operators, and their inputs are often AbstractTangent subtypes. Pullbacks/pushforwards in-turn call other linear operators on those inputs. Thus it is desirable to have all common linear operators work on AbstractTangents.

source
ChainRulesCore.debug_mode — Function
debug_mode() -> Bool

Determines if ChainRulesCore is in debug_mode. Defaults to false, but if the user redefines it to return true then extra information will be shown when errors occur.

Enable via:

ChainRulesCore.debug_mode() = true
source
ChainRulesCore.no_rrule — Function
no_rrule

This is an piece of infastructure supporting opting out of rrule. It follows the signature for rrule exactly. A collection of type-tuples is stored in its method-table. If something has this defined, it means that it must having a must also have a rrule, defined that returns nothing.

Do not overload no_rrule directly

It is fine and intended to query the method table of no_rrule. It is not safe to add to that directly, as corresponding changes also need to be made to rrule. The @opt_out macro does both these things, and so should almost always be used rather than defining a method of no_rrule directly.

Mechanics

note: when the text below says methods == it actually means: parameters(m.sig)[2:end] (i.e. the signature type tuple) rather than the method object m itself.

To decide if should opt-out using this mechanism.

  • find the most specific method of rrule and no_rule e.g with Base.which
  • if the method of no_rrule == the method of rrule, then should opt-out

To just ignore the fact that rules can be opted-out from, and that some rules thus return nothing, then filter the list of methods of rrule to remove those that are == to ones that occur in the method table of no_rrule.

Note also when doing this you must still also handle falling back from rule with config, to rule without config.

On the other-hand if your AD can work with rrules that return nothing, then it is simpler to just use that mechanism for opting out; and you don't need to worry about this at all.

For more information see the documentation on opting out of rules

See also ChainRulesCore.no_frule.

source
ChainRulesCore.no_frule — Function
no_frule

This is an piece of infastructure supporting opting out of frule. It follows the signature for frule exactly. A collection of type-tuples is stored in its method-table. If something has this defined, it means that it must having a must also have a frule, defined that returns nothing.

Do not overload no_frule directly

It is fine and intended to query the method table of no_frule. It is not safe to add to that directly, as corresponding changes also need to be made to frule. The @opt_out macro does both these things, and so should almost always be used rather than defining a method of no_frule directly.

Mechanics

note: when the text below says methods == it actually means: parameters(m.sig)[2:end] (i.e. the signature type tuple) rather than the method object m itself.

To decide if should opt-out using this mechanism.

  • find the most specific method of frule and no_rule e.g with Base.which
  • if the method of no_frule == the method of frule, then should opt-out

To just ignore the fact that rules can be opted-out from, and that some rules thus return nothing, then filter the list of methods of frule to remove those that are == to ones that occur in the method table of no_frule.

Note also when doing this you must still also handle falling back from rule with config, to rule without config.

On the other-hand if your AD can work with frules that return nothing, then it is simpler to just use that mechanism for opting out; and you don't need to worry about this at all.

For more information see the documentation on opting out of rules

See also ChainRulesCore.no_rrule.

source
diff --git a/previews/PR663/assets/documenter.js b/previews/PR663/assets/documenter.js new file mode 100644 index 000000000..58deef09b --- /dev/null +++ b/previews/PR663/assets/documenter.js @@ -0,0 +1,893 @@ +// Generated by Documenter.jl +requirejs.config({ + paths: { + 'highlight-julia': 'https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/languages/julia.min', + 'headroom': 'https://cdnjs.cloudflare.com/ajax/libs/headroom/0.12.0/headroom.min', + 'jqueryui': 'https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.13.2/jquery-ui.min', + 'minisearch': 'https://cdn.jsdelivr.net/npm/minisearch@6.1.0/dist/umd/index.min', + 'jquery': 'https://cdnjs.cloudflare.com/ajax/libs/jquery/3.7.0/jquery.min', + 'headroom-jquery': 'https://cdnjs.cloudflare.com/ajax/libs/headroom/0.12.0/jQuery.headroom.min', + 'highlight': 'https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/highlight.min', + 'highlight-julia-repl': 'https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/languages/julia-repl.min', + }, + shim: { + "highlight-julia": { + "deps": [ + "highlight" + ] + }, + "headroom-jquery": { + "deps": [ + "jquery", + "headroom" + ] + }, + "highlight-julia-repl": { + "deps": [ + "highlight" + ] + } +} +}); +//////////////////////////////////////////////////////////////////////////////// +require([], function() { +window.MathJax = { + "tex": { + "inlineMath": [ + [ + "$", + "$" + ], + [ + "\\(", + "\\)" + ] + ], + "macros": { + "Re": "{\\operatorname{Re}}", + "ip": [ + "{\\left\\langle #1, #2 \\right\\rangle}", + 2 + ], + "tr": "{\\operatorname{tr}}", + "Im": "{\\operatorname{Im}}" + }, + "tags": "ams" + }, + "options": { + "ignoreHtmlClass": "tex2jax_ignore", + "processHtmlClass": "tex2jax_process" + } +} +; + +(function () { + var script = document.createElement('script'); + script.src = 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/3.2.2/es5/tex-svg.js'; + script.async = true; + document.head.appendChild(script); +})(); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery', 'highlight', 'highlight-julia', 'highlight-julia-repl'], function($) { +$(document).ready(function() { + hljs.highlightAll(); +}) + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +let timer = 0; +var isExpanded = true; + +$(document).on("click", ".docstring header", function () { + let articleToggleTitle = "Expand docstring"; + + debounce(() => { + if ($(this).siblings("section").is(":visible")) { + $(this) + .find(".docstring-article-toggle-button") + .removeClass("fa-chevron-down") + .addClass("fa-chevron-right"); + } else { + $(this) + .find(".docstring-article-toggle-button") + .removeClass("fa-chevron-right") + .addClass("fa-chevron-down"); + + articleToggleTitle = "Collapse docstring"; + } + + $(this) + .find(".docstring-article-toggle-button") + .prop("title", articleToggleTitle); + $(this).siblings("section").slideToggle(); + }); +}); + +$(document).on("click", ".docs-article-toggle-button", function () { + let articleToggleTitle = "Expand docstring"; + let navArticleToggleTitle = "Expand all docstrings"; + + debounce(() => { + if (isExpanded) { + $(this).removeClass("fa-chevron-up").addClass("fa-chevron-down"); + $(".docstring-article-toggle-button") + .removeClass("fa-chevron-down") + .addClass("fa-chevron-right"); + + isExpanded = false; + + $(".docstring section").slideUp(); + } else { + $(this).removeClass("fa-chevron-down").addClass("fa-chevron-up"); + $(".docstring-article-toggle-button") + .removeClass("fa-chevron-right") + .addClass("fa-chevron-down"); + + isExpanded = true; + articleToggleTitle = "Collapse docstring"; + navArticleToggleTitle = "Collapse all docstrings"; + + $(".docstring section").slideDown(); + } + + $(this).prop("title", navArticleToggleTitle); + $(".docstring-article-toggle-button").prop("title", articleToggleTitle); + }); +}); + +function debounce(callback, timeout = 300) { + if (Date.now() - timer > timeout) { + callback(); + } + + clearTimeout(timer); + + timer = Date.now(); +} + +}) +//////////////////////////////////////////////////////////////////////////////// +require([], function() { +function addCopyButtonCallbacks() { + for (const el of document.getElementsByTagName("pre")) { + const button = document.createElement("button"); + button.classList.add("copy-button", "fa-solid", "fa-copy"); + button.setAttribute("aria-label", "Copy this code block"); + button.setAttribute("title", "Copy"); + + el.appendChild(button); + + const success = function () { + button.classList.add("success", "fa-check"); + button.classList.remove("fa-copy"); + }; + + const failure = function () { + button.classList.add("error", "fa-xmark"); + button.classList.remove("fa-copy"); + }; + + button.addEventListener("click", function () { + copyToClipboard(el.innerText).then(success, failure); + + setTimeout(function () { + button.classList.add("fa-copy"); + button.classList.remove("success", "fa-check", "fa-xmark"); + }, 5000); + }); + } +} + +function copyToClipboard(text) { + // clipboard API is only available in secure contexts + if (window.navigator && window.navigator.clipboard) { + return window.navigator.clipboard.writeText(text); + } else { + return new Promise(function (resolve, reject) { + try { + const el = document.createElement("textarea"); + el.textContent = text; + el.style.position = "fixed"; + el.style.opacity = 0; + document.body.appendChild(el); + el.select(); + document.execCommand("copy"); + + resolve(); + } catch (err) { + reject(err); + } finally { + document.body.removeChild(el); + } + }); + } +} + +if (document.readyState === "loading") { + document.addEventListener("DOMContentLoaded", addCopyButtonCallbacks); +} else { + addCopyButtonCallbacks(); +} + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery', 'headroom', 'headroom-jquery'], function($, Headroom) { + +// Manages the top navigation bar (hides it when the user starts scrolling down on the +// mobile). +window.Headroom = Headroom; // work around buggy module loading? +$(document).ready(function () { + $("#documenter .docs-navbar").headroom({ + tolerance: { up: 10, down: 10 }, + }); +}); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery', 'minisearch'], function($, minisearch) { + +// In general, most search related things will have "search" as a prefix. +// To get an in-depth about the thought process you can refer: https://hetarth02.hashnode.dev/series/gsoc + +let results = []; +let timer = undefined; + +let data = documenterSearchIndex["docs"].map((x, key) => { + x["id"] = key; // minisearch requires a unique for each object + return x; +}); + +// list below is the lunr 2.1.3 list minus the intersect with names(Base) +// (all, any, get, in, is, only, which) and (do, else, for, let, where, while, with) +// ideally we'd just filter the original list but it's not available as a variable +const stopWords = new Set([ + "a", + "able", + "about", + "across", + "after", + "almost", + "also", + "am", + "among", + "an", + "and", + "are", + "as", + "at", + "be", + "because", + "been", + "but", + "by", + "can", + "cannot", + "could", + "dear", + "did", + "does", + "either", + "ever", + "every", + "from", + "got", + "had", + "has", + "have", + "he", + "her", + "hers", + "him", + "his", + "how", + "however", + "i", + "if", + "into", + "it", + "its", + "just", + "least", + "like", + "likely", + "may", + "me", + "might", + "most", + "must", + "my", + "neither", + "no", + "nor", + "not", + "of", + "off", + "often", + "on", + "or", + "other", + "our", + "own", + "rather", + "said", + "say", + "says", + "she", + "should", + "since", + "so", + "some", + "than", + "that", + "the", + "their", + "them", + "then", + "there", + "these", + "they", + "this", + "tis", + "to", + "too", + "twas", + "us", + "wants", + "was", + "we", + "were", + "what", + "when", + "who", + "whom", + "why", + "will", + "would", + "yet", + "you", + "your", +]); + +let index = new minisearch({ + fields: ["title", "text"], // fields to index for full-text search + storeFields: ["location", "title", "text", "category", "page"], // fields to return with search results + processTerm: (term) => { + let word = stopWords.has(term) ? null : term; + if (word) { + // custom trimmer that doesn't strip @ and !, which are used in julia macro and function names + word = word + .replace(/^[^a-zA-Z0-9@!]+/, "") + .replace(/[^a-zA-Z0-9@!]+$/, ""); + } + + return word ?? null; + }, + // add . as a separator, because otherwise "title": "Documenter.Anchors.add!", would not find anything if searching for "add!", only for the entire qualification + tokenize: (string) => string.split(/[\s\-\.]+/), + // options which will be applied during the search + searchOptions: { + boost: { title: 100 }, + fuzzy: 2, + processTerm: (term) => { + let word = stopWords.has(term) ? null : term; + if (word) { + word = word + .replace(/^[^a-zA-Z0-9@!]+/, "") + .replace(/[^a-zA-Z0-9@!]+$/, ""); + } + + return word ?? null; + }, + tokenize: (string) => string.split(/[\s\-\.]+/), + }, +}); + +index.addAll(data); + +let filters = [...new Set(data.map((x) => x.category))]; +var modal_filters = make_modal_body_filters(filters); +var filter_results = []; + +$(document).on("keyup", ".documenter-search-input", function (event) { + // Adding a debounce to prevent disruptions from super-speed typing! + debounce(() => update_search(filter_results), 300); +}); + +$(document).on("click", ".search-filter", function () { + if ($(this).hasClass("search-filter-selected")) { + $(this).removeClass("search-filter-selected"); + } else { + $(this).addClass("search-filter-selected"); + } + + // Adding a debounce to prevent disruptions from crazy clicking! + debounce(() => get_filters(), 300); +}); + +/** + * A debounce function, takes a function and an optional timeout in milliseconds + * + * @function callback + * @param {number} timeout + */ +function debounce(callback, timeout = 300) { + clearTimeout(timer); + timer = setTimeout(callback, timeout); +} + +/** + * Make/Update the search component + * + * @param {string[]} selected_filters + */ +function update_search(selected_filters = []) { + let initial_search_body = ` +
Type something to get started!
+ `; + + let querystring = $(".documenter-search-input").val(); + + if (querystring.trim()) { + results = index.search(querystring, { + filter: (result) => { + // Filtering results + if (selected_filters.length === 0) { + return result.score >= 1; + } else { + return ( + result.score >= 1 && selected_filters.includes(result.category) + ); + } + }, + }); + + let search_result_container = ``; + let search_divider = `
`; + + if (results.length) { + let links = []; + let count = 0; + let search_results = ""; + + results.forEach(function (result) { + if (result.location) { + // Checking for duplication of results for the same page + if (!links.includes(result.location)) { + search_results += make_search_result(result, querystring); + count++; + } + + links.push(result.location); + } + }); + + let result_count = `
${count} result(s)
`; + + search_result_container = ` +
+ ${modal_filters} + ${search_divider} + ${result_count} +
+ ${search_results} +
+
+ `; + } else { + search_result_container = ` +
+ ${modal_filters} + ${search_divider} +
0 result(s)
+
+
No result found!
+ `; + } + + if ($(".search-modal-card-body").hasClass("is-justify-content-center")) { + $(".search-modal-card-body").removeClass("is-justify-content-center"); + } + + $(".search-modal-card-body").html(search_result_container); + } else { + filter_results = []; + modal_filters = make_modal_body_filters(filters, filter_results); + + if (!$(".search-modal-card-body").hasClass("is-justify-content-center")) { + $(".search-modal-card-body").addClass("is-justify-content-center"); + } + + $(".search-modal-card-body").html(initial_search_body); + } +} + +/** + * Make the modal filter html + * + * @param {string[]} filters + * @param {string[]} selected_filters + * @returns string + */ +function make_modal_body_filters(filters, selected_filters = []) { + let str = ``; + + filters.forEach((val) => { + if (selected_filters.includes(val)) { + str += `${val}`; + } else { + str += `${val}`; + } + }); + + let filter_html = ` +
+ Filters: + ${str} +
+ `; + + return filter_html; +} + +/** + * Make the result component given a minisearch result data object and the value of the search input as queryString. + * To view the result object structure, refer: https://lucaong.github.io/minisearch/modules/_minisearch_.html#searchresult + * + * @param {object} result + * @param {string} querystring + * @returns string + */ +function make_search_result(result, querystring) { + let search_divider = `
`; + let display_link = + result.location.slice(Math.max(0), Math.min(50, result.location.length)) + + (result.location.length > 30 ? "..." : ""); // To cut-off the link because it messes with the overflow of the whole div + + if (result.page !== "") { + display_link += ` (${result.page})`; + } + + let textindex = new RegExp(`\\b${querystring}\\b`, "i").exec(result.text); + let text = + textindex !== null + ? result.text.slice( + Math.max(textindex.index - 100, 0), + Math.min( + textindex.index + querystring.length + 100, + result.text.length + ) + ) + : ""; // cut-off text before and after from the match + + let display_result = text.length + ? "..." + + text.replace( + new RegExp(`\\b${querystring}\\b`, "i"), // For first occurrence + '$&' + ) + + "..." + : ""; // highlights the match + + let in_code = false; + if (!["page", "section"].includes(result.category.toLowerCase())) { + in_code = true; + } + + // We encode the full url to escape some special characters which can lead to broken links + let result_div = ` + +
+
${result.title}
+
${result.category}
+
+

+ ${display_result} +

+
+ ${display_link} +
+ + ${search_divider} + `; + + return result_div; +} + +/** + * Get selected filters, remake the filter html and lastly update the search modal + */ +function get_filters() { + let ele = $(".search-filters .search-filter-selected").get(); + filter_results = ele.map((x) => $(x).text().toLowerCase()); + modal_filters = make_modal_body_filters(filters, filter_results); + update_search(filter_results); +} + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +// Modal settings dialog +$(document).ready(function () { + var settings = $("#documenter-settings"); + $("#documenter-settings-button").click(function () { + settings.toggleClass("is-active"); + }); + // Close the dialog if X is clicked + $("#documenter-settings button.delete").click(function () { + settings.removeClass("is-active"); + }); + // Close dialog if ESC is pressed + $(document).keyup(function (e) { + if (e.keyCode == 27) settings.removeClass("is-active"); + }); +}); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +let search_modal_header = ` +
+
+

+ + + + +

+
+
+ +
+
+`; + +let initial_search_body = ` +
Type something to get started!
+`; + +let search_modal_footer = ` + +`; + +$(document.body).append( + ` + + ` +); + +document.querySelector(".docs-search-query").addEventListener("click", () => { + openModal(); +}); + +document.querySelector(".close-search-modal").addEventListener("click", () => { + closeModal(); +}); + +$(document).on("click", ".search-result-link", function () { + closeModal(); +}); + +document.addEventListener("keydown", (event) => { + if ((event.ctrlKey || event.metaKey) && event.key === "/") { + openModal(); + } else if (event.key === "Escape") { + closeModal(); + } + + return false; +}); + +// Functions to open and close a modal +function openModal() { + let searchModal = document.querySelector("#search-modal"); + + searchModal.classList.add("is-active"); + document.querySelector(".documenter-search-input").focus(); +} + +function closeModal() { + let searchModal = document.querySelector("#search-modal"); + let initial_search_body = ` +
Type something to get started!
+ `; + + searchModal.classList.remove("is-active"); + document.querySelector(".documenter-search-input").blur(); + + if (!$(".search-modal-card-body").hasClass("is-justify-content-center")) { + $(".search-modal-card-body").addClass("is-justify-content-center"); + } + + $(".documenter-search-input").val(""); + $(".search-modal-card-body").html(initial_search_body); +} + +document + .querySelector("#search-modal .modal-background") + .addEventListener("click", () => { + closeModal(); + }); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +// Manages the showing and hiding of the sidebar. +$(document).ready(function () { + var sidebar = $("#documenter > .docs-sidebar"); + var sidebar_button = $("#documenter-sidebar-button"); + sidebar_button.click(function (ev) { + ev.preventDefault(); + sidebar.toggleClass("visible"); + if (sidebar.hasClass("visible")) { + // Makes sure that the current menu item is visible in the sidebar. + $("#documenter .docs-menu a.is-active").focus(); + } + }); + $("#documenter > .docs-main").bind("click", function (ev) { + if ($(ev.target).is(sidebar_button)) { + return; + } + if (sidebar.hasClass("visible")) { + sidebar.removeClass("visible"); + } + }); +}); + +// Resizes the package name / sitename in the sidebar if it is too wide. +// Inspired by: https://github.com/davatron5000/FitText.js +$(document).ready(function () { + e = $("#documenter .docs-autofit"); + function resize() { + var L = parseInt(e.css("max-width"), 10); + var L0 = e.width(); + if (L0 > L) { + var h0 = parseInt(e.css("font-size"), 10); + e.css("font-size", (L * h0) / L0); + // TODO: make sure it survives resizes? + } + } + // call once and then register events + resize(); + $(window).resize(resize); + $(window).on("orientationchange", resize); +}); + +// Scroll the navigation bar to the currently selected menu item +$(document).ready(function () { + var sidebar = $("#documenter .docs-menu").get(0); + var active = $("#documenter .docs-menu .is-active").get(0); + if (typeof active !== "undefined") { + sidebar.scrollTop = active.offsetTop - sidebar.offsetTop - 15; + } +}); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +// Theme picker setup +$(document).ready(function () { + // onchange callback + $("#documenter-themepicker").change(function themepick_callback(ev) { + var themename = $("#documenter-themepicker option:selected").attr("value"); + if (themename === "auto") { + // set_theme(window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light'); + window.localStorage.removeItem("documenter-theme"); + } else { + // set_theme(themename); + window.localStorage.setItem("documenter-theme", themename); + } + // We re-use the global function from themeswap.js to actually do the swapping. + set_theme_from_local_storage(); + }); + + // Make sure that the themepicker displays the correct theme when the theme is retrieved + // from localStorage + if (typeof window.localStorage !== "undefined") { + var theme = window.localStorage.getItem("documenter-theme"); + if (theme !== null) { + $("#documenter-themepicker option").each(function (i, e) { + e.selected = e.value === theme; + }); + } + } +}); + +}) +//////////////////////////////////////////////////////////////////////////////// +require(['jquery'], function($) { + +// update the version selector with info from the siteinfo.js and ../versions.js files +$(document).ready(function () { + // If the version selector is disabled with DOCUMENTER_VERSION_SELECTOR_DISABLED in the + // siteinfo.js file, we just return immediately and not display the version selector. + if ( + typeof DOCUMENTER_VERSION_SELECTOR_DISABLED === "boolean" && + DOCUMENTER_VERSION_SELECTOR_DISABLED + ) { + return; + } + + var version_selector = $("#documenter .docs-version-selector"); + var version_selector_select = $("#documenter .docs-version-selector select"); + + version_selector_select.change(function (x) { + target_href = version_selector_select + .children("option:selected") + .get(0).value; + window.location.href = target_href; + }); + + // add the current version to the selector based on siteinfo.js, but only if the selector is empty + if ( + typeof DOCUMENTER_CURRENT_VERSION !== "undefined" && + $("#version-selector > option").length == 0 + ) { + var option = $( + "" + ); + version_selector_select.append(option); + } + + if (typeof DOC_VERSIONS !== "undefined") { + var existing_versions = version_selector_select.children("option"); + var existing_versions_texts = existing_versions.map(function (i, x) { + return x.text; + }); + DOC_VERSIONS.forEach(function (each) { + var version_url = documenterBaseURL + "/../" + each + "/"; + var existing_id = $.inArray(each, existing_versions_texts); + // if not already in the version selector, add it as a new option, + // otherwise update the old option with the URL and enable it + if (existing_id == -1) { + var option = $( + "" + ); + version_selector_select.append(option); + } else { + var option = existing_versions[existing_id]; + option.value = version_url; + option.disabled = false; + } + }); + } + + // only show the version selector if the selector has been populated + if (version_selector_select.children("option").length > 0) { + version_selector.toggleClass("visible"); + } +}); + +}) diff --git a/previews/PR663/assets/indigo.css b/previews/PR663/assets/indigo.css new file mode 100644 index 000000000..7dafd68aa --- /dev/null +++ b/previews/PR663/assets/indigo.css @@ -0,0 +1,88 @@ +@font-face { + font-family: JuliaMono; + src: local("JuliaMono"), url("https://cdn.jsdelivr.net/gh/cormullion/juliamono/webfonts/JuliaMono-Regular.woff2"); } + +html.theme--documenter-dark pre, +html.theme--documenter-dark code { + font-family: "JuliaMono"; } + +html:not(.theme--documenter-dark) body #documenter a { + color: #4595D1; } + +html:not(.theme--documenter-dark) body #documenter a:hover, html:not(.theme--documenter-dark) body #documenter a:focus { + color: #194E82; } + +html:not(.theme--documenter-dark) body #documenter .docs-sidebar { + box-shadow: none; + color: #FFFFFF; + background-color: #194E82; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar .docs-package-name a, html:not(.theme--documenter-dark) body #documenter .docs-sidebar .docs-package-name a:hover { + color: #FFFFFF; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu { + border-top: none; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active ul.internal .tocitem:hover, html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active .tocitem:hover, html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active .tocitem, html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu .tocitem:hover { + color: #FFFFFF; + background-color: #4595D1; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu .tocitem { + color: #FFFFFF; + background: none; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active { + border-top: none; + border-bottom: none; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active ul.internal { + margin: 0; + border-top: none; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active ul.internal li { + margin-top: 0; } + html:not(.theme--documenter-dark) body #documenter .docs-sidebar ul.docs-menu li.is-active ul.internal .tocitem { + color: #194E82; + background-color: #FFFFFF; + padding: 0.5em; + padding-left: 1em; } + +html:not(.theme--documenter-dark) body #documenter article#documenter-page a.docs-heading-anchor { + color: #194E82; } + +html:not(.theme--documenter-dark) body #documenter pre, html:not(.theme--documenter-dark) body #documenter code { + color: inherit; + font-family: "JuliaMono"; } + html:not(.theme--documenter-dark) body #documenter pre .hljs-meta, html:not(.theme--documenter-dark) body #documenter code .hljs-meta { + color: #4595D1; } + html:not(.theme--documenter-dark) body #documenter pre .hljs-keyword, html:not(.theme--documenter-dark) body #documenter code .hljs-keyword { + color: #194E82; } + +.admonition.is-category-terminology { + background-color: #FFFEDD; + border-color: #FFEC8B; } + .admonition.is-category-terminology > .admonition-header { + background-color: #FFEC8B; + color: black; } + .admonition.is-category-terminology > .admonition-header:before { + content: "Terminology: "; + font-family: inherit; + font-weight: bold; } + +html.theme--documenter-dark .admonition.is-category-terminology { + border-color: #FFEC8B; } + +details { + padding-left: 1rem; + padding-right: 1rem; + padding-bottom: 1rem; + background: aliceblue; } + +html.theme--documenter-dark details { + background: #282f2f; } + +.video-container { + position: relative; + width: 100%; + padding-bottom: 56.25%; } + +.video-container > iframe { + position: absolute; + top: 0; + left: 0; + width: 100%; + height: 100%; + border: 0; } diff --git a/previews/PR663/assets/logo.png b/previews/PR663/assets/logo.png new file mode 100644 index 0000000000000000000000000000000000000000..e6ebdcd11defeb333376b696cb7b59f0f28fdd15 GIT binary patch literal 31296 zcmYhj1yoes_dh%^bPCcb-QA4{BAwDmcZYOJw{*9_5K4#S&>`K;0MaQXCA^pK^ZT#$ zE`~d6&D?YDKKtz6pEFUa%CeYfBxoQI2vhEZv^od`*Yom^f&_fxM>H%0ydjw?$V!8r zU;YX@%925#Hy}Cbx0+r#$7`M$1{O=Vf6g-3f8FFMouL+n;YCY=n5boN@K_6Y5&OT~ zGvC*@-z-SIekBqa^ybZ%52+;Zs9!^`%L{83>A?ieR(?G{%j)I?!7v#_fCm zvo~KC?_4$lJ6eMrpjyTc;|a~6;hd)hlv>IplrTm|OehYtJvVS1egJNs>;9jMG5I-r z4Vn%rE8_CN{$F&eDAh=6Q}|sF1Bd}WMvbOp&04Z#?R{>qa_)@s&oTL^Ir;K2`AJ(U z<3{5-UQq%b%h~+iaCMucOpLEoDT-WQ6}d8im;4mD$|Ob13`N2WLEH?1M)hc(*&14V zw0!hL5$^9#otlxoG{BR3w2;~<1xa5pjM383>^U`i7xGBQp$-Koad=*wk(1OzPN|4~1 zOyyrl@3?4w1kJxsyLV!0#Ud@afvCd4pE$ z*BAa0#ujR3S+)6iH*~}RE0C=&^# zu)1RO>%_yrbkm1e*_Af0Zyhn~eIIO0S)9?GSvU*yoqHx}anMZs;9xP(J&*(xuQ^8H zh*{_-Z{!%Q)ArQX0k4YtjkA#*GHGw(rpn)f_mN%j_n*;~iIUnwLOIz5nD6|h2Mk~kUma|*XK5{6-bjM z+3R!H>lQsr-!-3s)Ml+KY*#Bc7N=wRNH(ZIkVKJIUNkj*lq!muF3Ktf87k+>(e9y3 zBQWTY^kcBjl*AB6c2m7DFE^0^mLJ^5bjIiufOoNl@VBV8d=M>%W|x1r!pcnOr+X`X z-6Npq!{g2QQi5g8!T_XOv2%+8{lUplWnk@vT?s`5t|?1O^z7d&e{X;OC^!Ha)2)pz znR&GF73o@8e{V2H?NLNP_fKwPs81(ZnlUusFla^1UsCtg)t|X2Oz)L{bVd#_eYf4E z-&v1Hy#sDlV6NuKu|~VzYE4GMZRQE%6!DY~Y4&ldnDcU3wrT$*C6i zvZ!?^$^--R_AR;=mcEt1be56}s#dZs80FVmd5-cwh-J7Pzo-9nL|WeQ%;rq4ZmagP zXIvhZMS1~aHQ9=O0e<9!TIomR$+sNDu%~$}M@L)YQ?pjIxkzTRmW6B&u_F9!;xPX1lj^(z>_KOZ56wz3 zR}j51YK_`LL7xf0L}`yP@DpOn#8vBE7jZIUt=?6C?=RN=tMYz9xJh3 zA#5nVADS!)OR0M!k;4zyc7?{Kqm?|6bb$*H1DFMJi1{t`WR;us)+>PKt#dXhr3jx1 zMvl!kjlo|dlhnrZyvppDlejUpTx7Lydtep7;s7%z0dk8qfdkbDIy)Xw@8tW5Q%~Dn zRA@uOkcO~G5>=-ghruD8i#=T>E2c6y#3!S81!n?72(ntki6fv6HVU{9>0 zrE?+dQrKXzmwCUw%&Wv^8!SE6Q;<)onEz&`P-gU-O?x%#P+N#eldPE|Oc5q-{9$jh z5AJK(ofMgVCY0|La7Rr8JJm@FOE_t!;_E(`xvgeo$fXACbwEs*(L|nnm|>Px%$m@c zB>|GZ=+cq`Yi?e+t-(e6xJi`%S3o|4+fZ&ZWoB-dV&xS7j#Q;D)&p{Zk&w+G;yVOI zh1$XlJGOCs0ecV$<{`fx9lS}E!_3hmm=&Qgac;yC@kov;@duOG{u$i}X3y=#^?l>E zAa$Hi+DKKocXffw4oNcH=@0(d@FQvFmjXx>3Wof{2XU&4pSi zrxh6uZho`B_w!WzK4}>34a%d&Sw`O}pl`KKgzy`l8H%g?zGyrAmFs^ZMa}qZMz6iS zs*1DS!_9)4gFgEV(s{fn+xi?R0sMnQ8DeEJS?wuGt&+M_)iP@N>4Y5A7_mti{MNr1 zx7WOwm+D3l9Dts}j&}dEpw%lM{f0iNiWsAw4Yyc~qKqc3K#ZV_jlhaA)n{TspEoY( z#qCO!&&fk=3+GL&01t`RjINUCsC1o}h4$&>WN&UgSRJON2^dbVCaX0Hr;uB4h>2 z9^7>L*y4#TF^!c1{wTRKo=^sN8MG8+c14)rCM?+5HvuTxrC;6`Cxa(deFlleMU09R zQ>4jqC+VV^2|+5XG0LD{@#wcL-t*=iX2Y4|USE(s+|RvavB_(1_XBNZ9Jn8u&xI?9 z9y{C$A_l(5N6W&O!C4hJih>|@E4RiUiTAY9)U*nU&oP`&Dy~_N(BW1w4GNv4rGUVo z&6n~#cFlbHRKykOb9_&BlRiG08{pU`+^>^H&BGYuTnnJs`?6C4W zo}qG`gy?r#DZx4gk#1FnBwc5Au(mt8Jt3wbaSNItVGTUl$zN9oGq(D6M=`Tst@_z| zxBqVVgG!+Av&L%sk}uR6U_F*Dn{0Q}SedeXHKc2jF8y`%ltq zjMIDw)07_$VxG*LyD$%@lLe!FKx<_>69wU%cPHk&baOJH@BKkEgC7LHrTUOugKOmP z-?mpvuRu+p;I>`G9h7|8GcPP%(cIme@84Hvro|tjHpI3go+WuM7E_6kZ-Uxj_KI@Y z^1yOC1$pQ6%EWc0z?ZDLJW%LW9OM>IZIl)q!R^JMgkP6l!HRCQBXgDf=Ta(^lDM~6 z6mN@b z!9=?B=549s=+j)F^R4^pEt zPgdj;Lj5K;6$KH9xiew~YY|aQ9S@WTiUb3*AR&$Y0qh!BJhxX^5L5X@QKqs#+y}=& zaH@&GNhmU059F%jOL{Rq&ZwGee2E+krD>}Me;a^*-(=7rUtEC280z5keAL;f98EC^7) z^T89uj(j%pB~57ZJRqNdtIt-NkLwe zA)08BlXUov3d}Z$(%#-uCh$pim{!vr}C?Bsl1?SWA5O+3MzFt{x+pM9Aa}y3fq7s9YJ0+UEFzX%xD&);g z68ml1@6TG?6PHVqDkPnypgs->5pa#5BqUZn7^#J>N)q3 zKW^QasO)t{(G~jVb%|Q~h z)0yr2!u5iJ+-@#u0ENN;g*w*I@}SLZu4=3>Z?)@eGK7EvH^H!dQpwwcAD?evAFdx`|9KyXQjncXXqWg}0?r4plsY_u-+<-wIxU1xF&q-h zQoZP9KRoojZBnAeF5=n<{-1ET`aV){n45QlR3dQg4i~r`3qAaNCV!imL!JC9cD|F zUCo~pHCbey<9*`kTq_Z8eYul23k_*pdTrc=&h3e(IZpPXSCv>7m$3V~f`vZ`Z7Cz(X6~4y>Z-CUASRcbz)Y+C`#mknPzn@#Z?To+1 z>yQC;h4#)Vywvm{2ZvfgJEUHp!8&aXu_P2XIY6C|>Ou4^vHJXfOecm~Ga{fhA<0rH za1=;hKPpxBRBs$Be9h&Ja80m)pts)`3F-xwpd?F0p#&H~efC+DE7Dleu9;SWFhD8J(Uo&<-J8Pe*71Vrh?{$YwX>{rT;@nk$%Kp3nZdi<|c?Zn4yhC zRaXXAuWu(!=kHrJ&}YCP$4I_5;hKEs|&?{Cv{+d!IZsLmRgB6(P+k$n)KReGhqq zUbsZn3omE^UbMOzdld$6R-q?UsT&kpk--P?1(|sM;8^>Nr9eH(4Jg4RB1n|L?1!M! z+UF_5>X$nlKqnQw37bQaoSuM6N$nw@XVj;o)GytXFmQ2`E~oWVvS7u@hq{>2OLbY$ z;-u{Zq6-KLn&B&GdOuEKT4O$fSX$4v?g)V>Lm?>63!kEH*hn#T+KjK!z#Q(bo6UIx-gX6*Q{yil(Uq#m|N#5Lfk$5qZ zR1^#83=3e#sCd90uZ?*u+@yGXrG^vZZ3j+n4MdX)Zo+ws)HI5@|6APGpT^DTk%hF} znVT_=rh5weEQnu4p`wEG2kI-^5!9ry-eIq(MKq~-i7$B*O5c~lVs*O>{nSARU_}?R z+_?8~nk8JmX~dUkRMH+TpkpL*oB7NQH1+-ErG~W2Wf=#^DfBi(FVpGftqy zy;Uk9z4l8aEz9Bh-^Umowwkzwn?rFLiQNC)w}WNGZa?rwKv`xaL)IYj_^XJY3uml< zCveL+(PbohB6)acZ-N{(FUdQG9vvU^+SnWpUCU7#c4-eHAN+z?6wJX%0cF?D-|<`t zEw7eL_C@@35v>L9T(@cW6aXq;W>DsIv?Bi5E0E*&(u-`Q;#aJuae%beraqp&%Jo0CeEk@-WlVHGAXhr= zHuyifIo+*@%h7qr-falVt=XLZNBY1gERLPm@6Gmc3R6`6e-vht@R6A@Na?aB^9+b? zNts6&EcSJ)@}RhGYr6kO9PsVW8sfOZe5PM<8vi#NHy<-r_^Kd2lNp5zRbB6-=WQL@ zq%(&9%S!+6mb*Gz%@=mk^iW#>JnGy0PoM(IUg=^vVGJmTsyqJoNHXzvDsFV##Oj$C zk}o1OF}W8Ti}K%Edeb>P|GKL2V410u0QyAy@;Iz~`*IqraLgJVUd(#S_VO!?SIOm5 zz^;CjitM!l)>j4MC3~k5LqBcfk;kR>ro@gDaVSI95cu-ghW%%o013dD;hu_B?p-y) zt@E8CIJ^Jd zq{42$W&HaWkA;wtEA+ql_siLJ`D_Kl^WH|5@bprS+$DhrL=UCN#LYG~6<*{9c<^14 z{(qnKD#ZbzOb^pXc>0~!;btwD7Q9UPLu@hYa43m1%0q@$nO?$UH`cN9);Uc&IX4;i z^%9x1DT;BA#7ZTyMtwKGZDC+Z!pV-yIV5|ytoQ$j$A$wVi7by_q}@#(FnXH0Z8eR9 zTU8igORwTG~pEpFKE$2CP`P8S@aErkiEtib~HpV>XST!U%sO?zx zVh~VY=g_P*4&X7-=BVg;Flwko`|3JhvAi=`CnoyF^Rzxj#akas`{EU-D827LZh-{<+en zziKJ^nylsMq{a4F(9T|v#ld>F*KXY2BkO%YXR+y-QM#N(WW!)o!vOIz1EKHc&}XRe8cpFiSF0ULsArZa6{{aM7f=hx zNx0bdlNNdH-npzYsPK;#N&PZN0we+N2zErecgNI{P#0J0dvTmHb)4|}*m0i`{^iaE(91XR7Niv6v)HhE+7 zLh_dz!l(|>DTSqS5^=2ux6Pe^@j-{$Ouh*1|7PgNfiO)ffa)#h8O4?rP97AZUEvhj zy%_{E?sj~9Hc;!YgRzQoqcS0me9w|`xXnkTCdx!U5>iXwX`6v^Td@f7t*|p%&~PcR z2`bk~xW{y^_q*!O@!8PW_a34%zitNGDFFDt~vn)3cKO{i&?UN7`U$U!n zioA{q^tO_VBcjmURS?xOC{==XK1@9ck;h5S}Ml!6h#dLfu0mt(X+LH!x&y}^ASPLpXfXa+o$LAAvH@6!ac(|O5<vW@r(k(_wj&*_r_ ztzdr4g6_>pAhtVPX;gF91j1&Z*2qOb63hY4hbIFC6Fb_DGjS3jdLii0-B)LTmt^ z5wTJ*munbZuh$C2<5npSSu(OSUUT>!nq52-q#ZO9-0Y6!;1mz1_$emSMcHgUr%_s# z0+avfzkIIlKw6V9V@C1>-l%g@Wz6R7afy~c&1!0}^L?W;k|)ImFV7P|#c%QqBEuJeK#CZBgag z3pIZ{9o)N!(F%N0KY>IG-K?Ka6oZ65C9Jy~&ovp+^y~o1$PwLG%_>K@v@W z-8nh9igiNh7XJ}R&SbvjREE1ZpJjSfJmEtwbVF_UoKiS~f7v|<@ohdZiyC#Z3D6>a zTzK9mR(Q!$vC=SG@IG?z6VGbuTqnIrjro^xJdLDuT$E|tLzkN2Ko6*<9hgFzt-xDv zZxSSqq64pmKGWd{WS+%o*0JvNU|WeHN1()@lZ4VAOyU@FgIPfF4Lw#+k|egRCdZxm z-NU872TX0&zQ;{i)`Jh)Q}7(*8ynxaJM`hexca~J-82v$+6cXt6oRmG@>!**`lJw! zL9}0)u#&`fLB1$*zXOc3G7JYsd*y%n86KKcR}fEiyt3S#XqcYmj!_H)ETpf-mt~Z{ zZjdxURs%a2$uan^IRlhm4Q}zi6Qk5?|fDfdYl| z+unt%*y2fGj^{CSgAm>=QrZ891z1L(#IC^98#|yw*dw|WiXNgNQEaaQ%&FXS+q*fa zgj)8$n^+fGBV@tic7zzq(Y98M8lZtm{g@qJh$m{62PcKCtcmkTs5VoGVA)bGQieC- zWkx^}^Ozd_P50l714d&X=o2W7)1yOe5hcY3d$y(mL`|A-qu9;rE_|Ts>9k0T zQscVNV2zxF0K{h|E+S}NIJfANFvl96?Qr2zBclkA_-t>AtIG{ z_3uujLd5|MMsbLrev--uw@?bg?@x4B+k(bHEv2>dljyX1MYWq0#l*0`_wO}se;;a# zu0jo=T|s|NeMIUzm!U1_BXp=}YkUDMWGcnrS*f%Yw=26-p z=I_>Xl39#h1kpaW_=OEu*kJCZavj-WwvfjD#3^SK&+@i1QlBe2Gu6yB;?D(-E(9G7 zELh&jT5Yg&&_taR12q?dBoy@Q@MYpBeBR_T;AkFhHz;v*A#u5$Dtwk zH2ok{-TNs5*aV1BGo7c`?AT{fM|kYwDRFGyX`Uy|lxhM6J)N?V6Gp}h@i7VtzZ@xH zu#TtgW$GE@>KRir?U2Lu2dXG|f1YTgo@fvHPAfO~>ci)(1}GYguXSq>*%sjm3_g3T zVq#UTZ>@F|`voU-el%&_!0=YnxGQtLEdgVLI@PmN?FNVgd2fA;B6Wl@Um>l}3gr~N ziK^iolm5<|ZR(Ppc{R)oUmxpFnB!D*;sW^X*UNZRt7J6uZqALcOwDjFrz9^Y*Mw(B zARcvehGoLu2>uGGA4rRM?`cm=q@1To>!4NRi%;;zpAr z?{Yuw87>>jP}q7JNyc*QsqyD0+%2jxbM~28&mgk}^A#tZ6qE3$!?NvuaZ|Cj_f9|q zK<2H)AfmtG-xi3caNP}54AVYF+S}7P(#T;+S=;aLROy&gp=oV?bpO-lI4h#f+IP0A z9~dFaz31tYZ1PD({-yVRSW)_BY7>+!)bbBKA3m*Rl>Ri2Cr8Z8<=vTcN4)rGdX3ty!8Hc(bL84K7%Rw+>_1BiocS@LiuGtoIu+; z<@QeXv(O-dUu(Mwe>jc7)3_dh4hqUvow3j^EbTri@IWblHYV;IzH{l8P-RoOsW~|_ z1UVIt1gSqfVSxfL{B*$q-2T)IX-KX_CO5~C{X(8gBb%vjGT&3s9)0oE_;|QCpLZkM*D@ zBj>nICi<|IecWR{d}daX$!q_XWz8uZv|Eg8V$WgpLsz?rOCRBm(tptE7H zAT7<7bjN;PF;0b#Z`S^+zxx7C1;hXt0LxX?gnFN<_iTFWdp+)=?;g5$;WgArGbSMvOiN~$s&KNX5A3pTlDOz82$Tj};K zfwIUFYYlO+t#m&Vb5V^`G9k-tcc7iYJL`4`d_7MfsazPpjLyGghk`7Z1liP&7;T6x z1hCZ+?oM%UgHMg4BOpSKz;vZ|I6)hQrEUF0NjN3}wFPNO`Htv18FwW06Mp3HK zV0{rDn@Z9fsA6?CU>?(b@A+mOskeg}f;aEum1lYS zv|zsrPq_3aE?zy^oVK6nm~qtN0yc3z)w$q3bfQp8K?3YY1gmRh?&Pf%uTFXcIY~RU zu|fVQ8GCn?+M5j8>kv4dTP+jh&j@?=CrS_sBk9>!3$!05nFHE(GhF<{y+}ePJVY#@uLLX zVhhw9E-%|%G#B^%eo`7!oO^20-2EhR2Cz`#YpS`PgE+9k?5v1iW_ec6F~QP6ZGDtd z0^tY$ijrbiCzhwbNfp{?7*IRdjSyaXEb~ATkRARu3`zh|%Ki(qmzw)+IG_FjiM}l- z)^lgWcI6K{!(ciB(9Mu$%&hoe$({c;&fSRk+CmJHSO1!4c(m$NK&l|vsgC80B6cx^ z04cjjS_Ica^X>!GjO()4=Cx2;o#xp&)^LSt2G9eSTx-dR7ye`#D@Q$woev*SmC0B7 zNkiYc(R7J>7T4@(qt&{#IoH_@gZo?XV6Z}vQNfJ|Y1(_G+T2C}(}(zR$D2<~xT??6 zLMdq7YNhn^gKjkukWs{)dBLiKEFSyuQz+Q_FP_;NxO&ojMHcUy zl|Le9P$4}c-J&>?`krar@t;q1N^|S%HSqN3=CA3lTJoG#Gxj@j*R}_K#n6N^lB9!9 z#bs}--U&;rlGP79Trv=cXBSs>6{oIy_E)8`0~TYEY7m`z=e07l)k-moRNA!#%dgE)qv_2QP=xGlWkvt<@7OHiCO&mI zfC1LDs6vv|nD3km_B}UH0%z`Pw?^&aL}}HVl%(ut$gHUW^{PtV(eI2 ziaL?fg*N8HIu~XPJNz6@ELAnG_>T2{)4IP&*V=9c-^LcNjvQAf{5m43u^;-z?cPCA zbZsOiz5KpOavAQC^4tz!rD`tXW3L(iWplBWy6M=)LDST-naipDdbfmdtDM#Qnr?t2p5Q=frH*G}%$eCrJCyM$lTA4!q!CRnkrL_uU7BMCV}45HgQ5$#iepfRZ?_VK*#o! z40BrP8#-Ck(M!uOiD}f_{l|?DoTB&MjgDh|V(t!=yY{dXfAt*aY?*2V( zIR37PSl}AU<0O!T@0Ggf81!qb;hrf?-=BQ~0yze9%Hhuwg-J6rzR%-Tu7w22ik)?Z z;VdL*&+pA=R71-E;O)<{e1M+fGUX{&k{`Bn4J&FEy}Bm-Ky~>#vlgf>8y4s3TfM=T zOOE{}C!-FOlKWa;_q8R5>YGaInbMGwVOIh)mh(bwcIXuU{Gmorj>w*Of=hi} z20prq`+$)wZB14mY+=1v7Wf0}Iku;Z6V%s=d+gPe4kYOj`US!AegcIPTD%PK6S}-g z4?wi2INBS}6H&pFXKZr@ItxY8B}S|FW)${G95dm9`>c%2Bg!6+=5P__SiS_=FFRt+|yE9OE54VU(!z$`sj?M^E8J)zJ_Xb0mHf$us9nHRb$y z#PvNqe9&~jfq$Dn8{)bMX<8|-Td{T*$~g4c;wGHH7_=sap;roAxH?HdF7jMmMfy3g z)LlAzTR(ak z;^#Y5&52MX4xZYbz^t;HgN3&fz@Gp#$e#^XALKU>NxmHPrn}C*>;LkKPQ03YB zW+RY)E24+9BD5B;lk&%eC09uJEW-}d{-+3}uYmZ`I%5LktPU7Vk$ zm%GoJn=zlS-$6-*!)a~+V9w#5JxnuZMe*d}(oJJ(iu(ZLyUKOlzuMDH0$r2Y{j))6 z(2s8NJS`YJ>}OYl$09?UVNP;cHZ;1nw!taL$Z-j%7OkiyfI@?ZBULHsVQ>X?o9(WC z8W(L^YyfRQ{mi|CA4JRzAl~mF%AB?Sfu5fh8*vVeZkDAvr-uia+jYyGFsv!d`*mn@->N8&KC>e%K)Zhg@?jomPF*MiqHX^r7_j_ln&MRNy3b zTJpwsE1Hb+{Nwz&0aVkNq{48Tw*c$&g?3Vj&EDoWYmWLYNX-_cVH3Dk)tPWiIk_ka zCz=4|$OAoL?xQ=9GhJB6uO>vkvCkg4z2ks>sF|jgp~DybZ||YePj6L4ig@A4TaoCn zGH83iW{A7+gYiB}GD}Ud-=Vaj4Q_ zxl>SPC=aOcPG}ON&o%`Z37UyLL&!Ztz`zw~d^Aa(a*Y1zq&*5;8!^_vo4xExC~V?l zMsn1|o;54=HCPq*MLVTF$)Ha067GEgL)&bCB++ytXVG`@IO%vOeGzPL1 zT0i@CuIt+Liu6Z0MWa7!5^5QFa7=IR5*<0(QLU#V%)NakGgi_HjQaBtR1JE)cQ5*A zyZiggBA+2yyE}dFbJa=@;MWV5OLIeP@fj~7X?qNhA>TYH&f#YJu~cc=<&DVWN|J3D zGZoDa@NB``wHa5r1V^NYML#`XulVrSHd=wRg*WK3$Qo6?BjvqE1_BfLT`%W(vUT_b zAVRWpa-_mn0JjI}DJ}WyNbm;6fuF#<01opaCNsGvo0ga$nfLtrXML}_dSuP=G1=g* zeUcPG+WtHgHYhRV^{wKpDFH}+6<_gtPVel|2Eaux27dT3ZOfH-V z+TNnC7U_*=n>G3}jmhNFpzH zJRXP1=A1=KPJ7m#7?X!QW#w7*-)KkDYaa0=k4Z0aePUKYgZ*gk+ItAsGi`sU zPo%xop550?f}NJatH5Duh_MIoxz$zCv#BO(W>r@$_1W2)0GT*R3xMzM+hh)x*1<9W z8-!IH^;%pykG8zdlz1{9NL8HY6BMIWcUia59jVODw{%$17qNqNi*&yQ+ClR1=Rrur z-FPg4lcKAv4^B|oudR>{8_8?BT<-A$SUL}b+L6~;T-cBic$hZds3S48kP{*snhyW> zkTi8DTuM0ARXfhNBsQBb7$e@QtM~ z8vy0Kq#hYng9LQ4gkY1C86SNy$`r-Yhwx0)~H#@%6qPUG?!?mEe5)Hw#NEPmKztT=M{I8nA{gwK8 zyMI#ua=#)!$aBGV2HRG@t#wieVgR{8Z8uAY4> zqZanJgv41*2{GTUs@HO4q1nhp9vry7K2V*l4?w&N2qFwYqCf16srN4ausz7X(+o}VSnUn_lhdU~PdOgI ziN8Tk2P7Fp8h1%3Dw5aD_BB}!IU;-9%2}>hS!o=bgr&jO9D%wXq-PDRvyii`SG%B_(S@7gbzKz(qB)s zS?U@II17*kKTEzNWEY%jq4OIDYBCI(@cionoJ%9^r77D2*&cm+a7R}p3dh+nAuLGy z-NukM0lNq@>OM;$_;_b-@$2aEsWBf%NgA|{AdX_cAW+fnLStJP{<@(10CwoYmt#O6 zJrIG+0B&EL8O$9M(Gd{jeFWUu2GmY1M4AdC6@^c8>d+J9&>gCAH#D`d--+HX$P1{j z%V1!q>>dnDm92an5r}Ln+D#B0-^Z1HFCdsTP}dj+j+Sv-v{Kr@cJcyFZ5=JfejsEZ zQAn}vl1u`}K@{K1g<8V+VLbF>eL`;RI}Hbf`{c=jikn`M8=a44^G#hq>0dnbu#Orl z7QN+l3rY;x!Zu6@7p6rmA7^glBA%80#?=FSwYZ5BG z4|;<4o3w1QVmG?NchDN+@)@M9qs^6bnK)`bUrw`*;H+f~u(<5i3$>$+sw zaGnFNMc3wj0od3ob<&+88(8K(Z37p)E&jf=m>J&Py9w!i*~YyrO6SyzrR2_cjqXuQ z))Uo%vB=tw;jtBKt#EuJI1#(Nvqf2VHFgnYLgLtEDM*5yw5rZ>wG4@{%pc-UYfHT= zg>R=t(A`*-U+;tXrrM|D zV{zH_({qWfclcZZqW8{azY3bJRszB-el7&WiaKfedUV2k>U_do!zU#jW*}hs;F}-o(J2fqm1qdA7;d*8{X*#N zQ1!YH+-Q6LP2H0&U*hf|rG@3@uUHW%PClF%mOSL4ytIq>w zRCE>sazdg#cU6aG6Oxd`Z2%2lTF7{W6fw&p+P7P{@BFJCXPkr-0I@vI7~Gb&aEjb? zutIjCD4N8k5zozL>ouuv>{w&2Mj>-w7!Y$xY=t$GjHmQCH#Gql`?t$GB79k^64^^} zuLeyH<#uYE5XuF?{jwb#W#jBb5sl#_F4AK;ZS5=!@mvH>I-=;X0m2WoeCb3ani49^ zuJlGRHdF>@Y{J!1L)>TAeH996y3WYf6=>Cjf-aVk_`B{WDgj zD%;8(iPjg}_$TpMcA~~0j=b>bHAETcvBWHOpp(oogJqPF=D!q6HeZZ zC+r%mm=};d!RUO&*T@jti*O8L-VJEOu}W1s^IE4#9zDO`6G)P?oDM>^xnajr{Xwd( z_)d(FH<#x@UG9r%f?`&b3i~5BU$9F7fye3uWLeMz$R=!fzIj3xMI!o#Q)8Zyo7a$S6^fh=$dVS{|CfBzX8q`&`7zcx+bFR)z&-A561r^v&gF;dqh(k8qfG+ zQmQ=$p3}LfaD8UgKVJC=!jE~hrVklEHaB)waKCj@xWvCCqLC-Ss9l> zL_A$-RY^Qm$tLx}l7>n6wZD8eS^nAP+?k`|f6o$<x%!~WehxAEH0cnI4PLf`!=%>{A;go$6=80%9Kw(s(u;nGnk%9JeAiM zDO2v}H+u8fJKqYX8(v49+s{u8yA-?)`2~8HC-Wi;L{~D^1pwOwY0I8=Wt?MTXMl0VG(=mjolO6Lo^IA%(*QleuLw;)O)U5Nd1m&3P7y)M=;9dBFYhji#{#Kk?WY9*T# z_^EYVO}Xk)b)sP(Nz5&B5%0T{HF6$^l~74vS$FpvqraY+n-AG>#mdl+d!=KW#b}P( zsqYH&wCGKE*T1styVGp@O$Ca>_@vg0{}F_oat6qP@@YO4(0ov^0<+~}9y_egUvhaf z$tVtGj^}LgWM)YaRxY(vPZnF;wd*gg)TC;8&e%YkgY%DaZ7}WvB`puXA*eGj9OmwX z<~;a(a{e*T6A`I2l@<6vbso~wz;537~7+<-x$u*^BRt@{^2`J{`%R`>zdj zMZUqz%!^d1B>Amcdh7rrFTsLMPlR(`r2VRMSjG9I?hehdh@RlbwWUUR%NyJe}66SI(7Bobl_ zAkPd=my<_*Sqq%mNx{X27*Fhu?)j|0D4{-~NGP*#(4OwRP(!6Yy&WW@aH@i*r+eGMI9wN_{%^i&MdC5d zy5?vXMc!KU8~3rNh{%0!KS_X+{)nHNn=_&dL=4aKe_-|wB12}i_%rbE1B>g`y;0P% zo5HJ2jOsI?;PD?C5FWOLQWquy4~2rND%4xjpAOT1(~@KPgg(Vwem!~LX?XJZgSa`X zXU`rqg|s3?ZE;7k`hKA3mf~k@J@;dNk-3blXJX_bLb+c=SV}%;<__+xQh7X%$<00Z zj#;0x&l~SxayZng%q?0WiwkTf#jAi9EaXSIUTrORXMkk#tE&EtpSRi0>KMh9@;0u` zT(VR!AuEaxTyOMp%YK8rUgl1sw^Wb6KTP|1GZd#ir=S{Xigkz<#D{#Kq?`-3C;XY_ zp!&KlK1RF4K)&uLCu_=KVDatZ562m&s&;!Jz|Q4uK0zBfMYJv6rS!`U>KjfAWh+Ry z5(ya7rJ8^~>vswQVtd9-avv;}5!NzSiU&~p_MCrkAX#M&2~5a>JNdd8RX2w}?|oQ| zw2twC$kXMhuvNJ?sXSJ&QpKAJ$<{&a%+mo{Ct=0ZTDpYWH(n&T>V#}YY#*Rb-|4k^ zLbn0z@g%$3W{t$OU1xzgf>el^OFT_^fR zO5WBP`wYPgLP_^8%bG|M6!24Dd7F0_Cy-oPdA)k@;MsFTWluFFw7}A6alFHj;nId{ zbU|n9 z`>H3AnL{5`9JYuB_@nt0EJE2H&CcYUUxLEkzLEx0R8>q0DMtYN6R~A(p!v$1-b4yYF0|n1`G6Gylmwq& zfF-=491J;S%C3CXn!Xo!k(XgA$=zu2?`rV5Z)X1nw4f9mp%0mPM@1Y zKeWEw5ZpwnO}cT**a#uxn}eC5wt?;TA}kGmb6&2^JU;BVujI7lQA{Ca7J)0I&!XYT zr}Y#SFAL{Q4F1*%2vg>(9eISxtUQuG{XAhD(^U91k@8*?`y$m`O>TAKy2Cx3s*q-+(5b~9zW{f9?1jJwzSMMKb<+sj5gh#Mt1 zAULN>j@xMXQ&`Ox2omwvDDcDC5iRasKDMy@V8+Lfy{^9Rr^2~8IY{#aFXph=Ii3pt zZPZ%r(#?@0&xNG(LUZ*v0`1uCRDZCAB8C!i)iU*~et8q(m9_G{L%roVbwg`=YggIQ z0c9&)vwml?L8w`huxanCxTxoiO;z%pYW0uH@gLfVUhB!2YRAc~n4!h1VFC#eDHnoa z?Q{|cDg5lUqwD@HFZXTyygoflU`C^4KH_73zpJtf%YO7NKfe592U@L%m(i{fiXOxt zA)ii>SK}ia4J$-xW#VXos05~3(t>A~W$BFm{_WFLbXLfpxHX3pH|N@#teSRkIYy)O zt^6cyy)cyyad08kt`$*a!+S1ppyK?5U4vhUl*OyRo*p_B#PwN4Kp2-s<36t6aaUAv zXFjsqb4!8l9LMrQii11m+BBS9kQC(PL0NxxiH^6MXjy5M$UjQqqqMok=5E4S$v8DQ zc#`^tsdwe4%sh_(ry2^9g^p#UmWVIa=p2#bJfWEvG3A=k1y-W)_nCx-r?3%H zzFskKxN6Q`$k)c@m*$5NDiWLrolQ}?x?k21S+S7#y1~mBvyYn}-Vx!R%{8}2q<84M zh7XQKYk4b-QK4~?-hvv2jCwU(?a`{+)PjU(Z{=(55mgo8QI9l=D?>iOc*u;=zCbF_ zhpWR$^y|r=r;UC!$L#QmzHqx0mGsG46|)A}eI;{opR(e)T2{Wz(>b{lsm$$KB;mTj zB8|`VpSA)!Rd8J=!lY#|_FIM{JE%OOBKKCb^}TP7cs<&{bSHMj&SCA4m5pIm_d2 z-_D6J5UG%+>?gS&<1B}!i?Rg$8E8`$!A&mO8hbfvaEPm#(T1zD@^7raOVf{+dc!w0 zwhYoF;iyV(<$l@FspY87G$Y9cXX&xMa-Y@pO8~x|xnlL8pjsV18bXEdMX>MLI`NwJ z7PX|}NLb8dVM@}(t0MlR?khSGAzlhoJjusT=pr&8J*?CCQTy&!PQPKUs2#(d2pPJ; zVmISHge$U*@2u~vQxsv$qP(@uI+A=x@qR5wMbzQ2 z_GI?**Sd4lI91ryuzWh&-&rqIR?01~?+Omuif*CP3o zG?_t!pc1qOX(x??v)XR$!xnjLP?QBw5WJY0cYOZkkiNVOGF#Dq*uF(2HNj8PN3H#L zXWP2`Rmz3+SCY&7GKt@RQgrMzHqSc=y|f}yd@XQXK@B*B806Kl$^VwVn0OWiM9hB_ z?4zVf7~_`0UG}1KmD%ApAQDo7YC#yciV}V%>K%?=zP0lDQAvXA8ghJXf>!myuK9KA z?P7$J$lBR3FX2Y{P~`g-KSLRnc2uf+iu$Sw`}+`6$_N%W2l}L)tlbTRZmOy9Sg(#G z4*j9jtj_dj%D(stVyU~Hs84JsE|Ml+QaNYBH90Xe;P{#790D#g5GnFh_Zle+(1?JX z0!{*cn|B<(WCob-%)isC60|9osl{Fgvz6Gq8s}X}0K{MV z=f9WzT{3?CXszuazayTLjmzCup#mQufs$g6Y#C(5y`ne7+c%1DwBJ)5EFV~_@fc!h zdV^(|E4329FpO9&+HcHcnRpAn66_@~{NNLK%IzB)fx$p#Y{xBc=+%=J6D)ngdqyUP zxQj}5^U`#-X5ZtA6Z48JmF7`11L@!be68q1V&<6woYBF+NOX;AN)?kKRL1(FK*(KF zGF6tfA&~J5KiUfbds9SM}!=)z*XYZQK`;rQFiBSD}@6eIa2i3M^G6zdyp8j3y;@*WkCOVl{}Zp*LqFDepxoO z1eei|BA)sAIqnUrd3CUOMv@+`vu)(1!>{LzjcKMg0Tp` zGSd*4P}Jkwi2P>%PL5SU#iNo$LNVLhd-)1}AE?}}6TmZ%tmsS8Q;J%mAxH^iqT@U( zp_lFie41laa2sJ!l-GV9e(CwKu1sE4zVj{#F|8b^h&f6SO`d&O-1Zf9wyL*oAXOEk z^f#ZCA6!Ubh#?t{!47jXt1@+3x7+kgb+k2my4AT!^IPG-)*jQvWT2#e-o!YYn6dBZTJzzS zqFD}WGJV}0vi}Qz8_7J>et@yBD9SMt3E1gL1>BxEmWuxL`BJR9vE@%xHff){@sE86 z_1jx{9rb4Um3Y~Aqof=x{V|fx81{e-rC*jeNVCd*2LHH+bfcY1%#E_l)B zo)xJzK7glNcTWUmUKpzA2HL$4v-z|Jks)vJlDVmR3{F<}Ms}=!ZRuH*@sqD^fr%+$ zA;sB4DW9wmtdXz*Vz1yVa=$>hO9v>+>_(?!J;WXJnlV)mWVUi7%)2BNvvX|a)ng4S z2DIp3KNqwA);)Dv#m!&{<$S}0rC+fRtel`;Af1m1KG{n#4x!=gODT4%<(X9EFJw&U z<~q195cXo%UHW&kjFp7lW%c`K;!o6l%N8@Tr;rbnHMJdm!}d$sdfr+J|8U&$_Fv_j zO~NS9X^O*3Gt0%PuSh7Zle*#4YS1^x{U3&D)?R@Tb4vb;C&>Ii?xe$bo+0;L+D~j%9-}}fbhXpC+2`P;tF)`A8HUH*@J40pHfyj5p!wZ& zTjd7w^2sfM;pY1Eb=KdNOp#^44+7gLA~MGM1$9S3hmDs6M$TouMZAiKJMGD&uCk2v z=J#Kj0typ&AeRpE{IiTX4RQCD2M0uolPUSYmh(cB#_dKb?fK??dmf_b*^EzHKC*LM zCIACIvV`u->|1;`&pTK#X1v_GMWmBXFt19lAcobI)6O$qGm5b)E+0(cGEhb4aF^P* z#~#p2MIhMZ?oQBq9{-69=zaP5$P)Mas9fY;TPy~u{qU^<3(l_v_|s=r%GYXD1t!9z za&55}MczM9(nK`Tz8ngBeXmxywDXqH(MMvs=8i?=0DIyCQ$K{b`jum8Ot*hWS5E0|&n}vMpc)=DyQ3Ng?w_ z!vtXx*7?D6Gz8Lrkz2-Z>*D_QnCBT3w+i?5##fL`s_zxt6_BKRXtz{eVW=LuC;lst z{U-SDJ0@aKno$<^*B5Jzt;5AHw2qq<$@J&cKq!%R>c@r4i6*3F?$TP0IdX#)|JsJv zEg)Py^3y-359YA+)D0|sCb3*L_25^`q?p0Gr~G0^0wo9m1@0Ee;biAR1jDn1{{C#R zVM4nIfc?oex03B-vRmkw_c+1Bor0HFmn2wI()q%xnYB72I#K7!FuD?k>L zJKENJ^ley`X=~I-bq!`h%*M#d(uC+blL@!lq;(P~$=Xtf-}{2*CQCQj6&P&oQ>}dX zrKsK?lab=8JsDQYbAP0WVf+qn)V?r zk^gnlI&(s|vbO|Hf!faaxi@+lrFH|Z+dKf;w&G~I*<3IHz?M>~K+NPYM!bn!KBEKj zM|+8T4XXo-y(Pb0xv4j>u@|2m(nD*CKN`LL1a+`^$20_QLV||}eYI8*Uim>X1JaSW z%1;xQBq~O^Nsq|nF76Y6x=+BNHD>8M?Wywxj%4wr#>jgGt_>$c6q~m#FA)#Zi z*~(U-bu;A-J73wcw8m1TRk6r|z}ry^W?^W!iiaS1=zRVhqAE4|@%I&KD}SrCqw9j5 z+oHWY)vZL^GOYoa=X@wRL9#d9XtIzgYVYDXmxXE^?phL)cq1gX@$X-yO1n zm7~GJ*R8hJHF9*o5?~F~7-;$`?L7%2l!E<&!hx52ADKNAhO3#>`*o5(U3aR~aJW)F zHblGbm7cGOtQY+14PH4Y82Bn1_v?BytMXgs0j#ngNitA}i(#SPa0~@$m>xi&`tZMk|_k`Cs^YO~242HMdGEEFe zsO3n``;>Aae-2EmA~w6vR_8+gd+1qadi%3sy$_Jy7q(1L(Wrn2{?@z@ zo2P3!;?F#jZ3t>N1VaZOk|9qTv4L-t&wM)SPqXpjiE^Z~ASfEEUgn;xsNZ{N6YtdUv3V>?i!hW;JS$dA#wfMOw1UdRA1W}`20gvjC22VC5 z-1i#M`wVp5kcsR)VP?(e6AmX
65><3gHOizU!N0{W-70UfcLHf(+g(E_*Jrs5j zFaM2m$}e539LGjZJXw-F*?U~R>1XpjVYX))J0Xh&oe-s>F|UBoN%h`WB9f1wrgobG zFi>j$ecAh-Sh8CL*h)&C5irvNzVkwhRop4Bj=A^Z8X-u#+K>%D$zSul>wR0;Q)F)f zKzP|v=yzYXxFYAiuo(?=ZW}NWq%F8K7kiEP9c0V2{*ff45CzjnQ|GgG!)KcDO^_LT zC@+Fb3(Z}pu0uKYt*WawRrSJ1O!oZxD3Z^P3}Ocl-A!Lq$$|T36fAUIRz)VckgZU? zLwjkJf_E`FnLP9HDXt)e3D}Y_g!(*V+DoNt z@n=t5mdD*CSl?3l^{24{g;q3p**oh^_p8}vd|0%Dh9sj<^ca16JOS)@NwYVII(h$R^__fqt}to?xXYaM9dFGex0S z-ZyvqFhdj}iKz;UyfbUld_wFF>D3~5Ifvod^$&cn{XBfYdR_&dlCgDg>s7R~CAZzr zPRq5Hh2(!x7vq}wx9C6hTp!t6tI+wBsd2-5%B5Fe^Cy6#GFmQ@q82}@YUC0;0f|TQ zK+eeNT)LU&_%{@oFE2YKlr-*F@lx!ugg13YkfDzrsRiQ!R14LCOhhNM1iYg^R7U5( zm?J(9>^t(POvK0mvZ_fu9ijX_|! zNwc<#FKJ!oiW~WPre!j$eq!u)Y}>HzBM+5)xslbBwM;(57u2t zmP$d*CBu;sX6bu^bW`WK`_7HxLGJDg^qMMdxv{%=Jli@rZDCf^;|9qo=lc1P3O>u0 zhMy-!%Qh1sz_XuNJmGgoUWx8HqkU9;bZbsbD3bJfTg$MmgWwU-Nl`}I1or~0ro0GC zj$waw8(rxpBoM#om9s`^WQK%W-kMZ4)g1nLK=_TQmdu}j({*Oh+h9fJpYKs_@4SBF8}!o$a^O_XS%)Ue_t9$DVrxHTn%utQ z5#;B~<^A$JuY3}{_%ED+NH<1mttW--vfX_cqe+6h)TT(^Nl4t~GimY&-2B*wL0d0? zOf?2Jw`4k%e1rZ7rS=;j4LdcHkw7sJhAY2a0b*Z5z>5iYu zhkIk`y1HW4Ein}9C&dvS)Mny1Dt|Qn_Q6dGvv#luaZV+OagUkG;|r(#jvYEY6T1b{&9M-?n| z94?QM5l9aNS3mMT)?2|sL;!uhq$4N^OsH7aOd^A60Px4>Z~N&S6n=A`AIkuH4-kWb z>K8Z`M;h=vkW&3t^1}3*%aH={AH!<@^Drv!U|$DCQ|c#T{-O=vzA5_-3D3I7hl|Mp zVODw76z7Y2RCBFGg$*Hv9~r_PpvMksK)c5#3WI>dV+i!eOx(!{qS9ZlYSH02eLkSj zm8EOjZ!{^(|a~oKO^V77Q*pSuIX0B6Q_^lNs06Pr`vZZwAD*bJe_mA?Yp-R)Pv2i4@ z3v_R<$7~>VqKd_*@9plLf?L7)e$d&u+*F8y>lx=_Piq>Ay=oH{)HJJhdVzMX?@}CM zKfbQ+d&rbzgBP#Y7ZCO(Dge2|^2s%*0(nJBo*4^jFjb|TcLo$E&ZtHfQ+isUArl=L z`#V=&wF*Pc5i~9Nqp31G#rjS^o&nfGjpSq@b1W}U=&jtrfJhM(ARPH!TqfH3H0L!N0CtWKfhZijHkz-b+MVdO9mm&C_m8E14Mt7AYy_Bb*fS-hok?Z;4 zzkbJEsK2u(mt|s?kjN{}OAMQ7RjD~FSg`-;Zpzb^2m=1B7o)t&2dp|%4gMq={Q(pe zsYboPrxCL34^)tBOUloa-5O*5Y=Y|`4Cg=qmPT392g1+B$-z|tp;NRi+bPxQX!bKt z06u3~5k7@ej~T$Z$@7?5$&4Cvg&lhgf<4ZRIwf^1INgeS($&48AbmE;xL;9C*RS_8 zWX$aOCiiXfr2`J%oC9Ac_#6ak5hp&^?ib_HV(tydmf$hCE{6L#R zLH(wB0OoCh@vrzLt(z_fRX$>;`^aJKx*15Z&rq5m!4j_iG$2p$w`VhXrYsA?yzAB& zj6E+eI_sFtoTtHBc<-Z$grMCR-5nu{su2|9mArbDsb%@hMlqQ`dGdEe+y9Nl3|)O7 zAcCa_ko%Lu=H+bpKZ<##Pe)B#A%tYJHs{!(-FnG>D!wSdE1cPJ#Y$bAmsmxL`~h!w z1M_ZB1M$}bhQ%<&TJ*7UfWR$xBW!F)(#hV=`nzFJe3MKI8o-alWl<9D`it{k@;Y@p z*3c^A&nnJ>v4#qSlV3QHW7I%&Ab7uv_EL)7ICY5ZH|PaU)8IE;dlDFSBKZfjDD^NK zMrfJogs#{aRc;@PZ3dabcjp3pgUaEhb`J7OxUXVE(M=u^%y1VdYPLfvXV*(qg&iPJ zgW!~78@5#Ow7P)Kcl6w35-SXjL&7jU;cPraOZHw!n#g-RZBfJgqr(lp&nn73q_mgC2lmSuNMsKlgmIk9p? zH^W=9FX$P7iGx!7G<>hiNr0h_7`;=97hylQi;7Z083w=laKBr8E}5qR5pyr`oUoMrjr5xKNH}*O2BIG1-28SR6BSffN^|)cadq z?7PAz@sos>pvldWa4niCu3&unI!VkmZ{OZ13hKPe3XawYKx^e|pjS6)-&@iH+EQ2k zd~M%k{{0FY<>ugjBcTtp!y~K5PvzjuTdnEuiUxo>>}E_oi#B^(#5Dy0&G`Slsg5t2 zKMBm29QA@BY3Op8Q+f+&=i3M`P}xrk$+o9z3x`V?TL6}+*B1df5Grws_->$lzGJ|% z2pUB>cZ%Ygd4v82fw>){2PC4hSyabNfA=>-nT)1^MZ^aNfsSA5qDlbycQuT7+sxya z5{M;Mrql9qhu?6d7F3k*W=2S+y@X2GzeE0iPdW1g)Di3fXbyOe65O{1a(W2aL$$p9 z^5%O^DbU|2A#$;-~euYtVGCZwphxDp-N?&8~ z`4?RZ;mCkgBTVZP6tK%d8O*gc)eZIrF8%dkfZnB9E-_8yq{nF>UwvRv+ovk<(Ptah zml?z2|L^AFd)g2_5S7JpfpWZ`K=ngze{RYzUvj}d@}o9TzQRBnqIo1^ZK_Uf{j3@3 z^{x#raIz41x*7i%luE5V!tFv)Z)&drLknltL4JKN3|uy{?zaeH=QuUP*YiO{^=~Z- z_5xM~Uk+3RtAY=$=Sn`OA~hM^@pP-Ni$`MaWNmi!jC+D#v3;WLyM!4&6Vh5OkT+`O z{m*g)y|=c~KmbQ|fN#9|<%s-Eh9^0^El+etVAL!g{VztP=Q3L|_TOb5B3%3IVpG9q zYwA3}|EJCZPuYCEA{QZn5`x|24dI14k=kVjT4lh74&PXzjQ49eg;(er@n0ls3?Qvk4lV-9nUpz6@hMS=0r{YtXGWPolV4J$+GA*7P zlh*3=sq2LshNQQ9|Jh;l-9r2ma>q>S6$6#=X`({Y`|py<1g8!e8xd&EKtF)3zXM5e zX**v5L$Fhm@6YT~fEXCN;|Dh{p z+vX-a5!j{mSglo?{-se*ur#BLZXs;+M1mZUNkIs@?ha=E;rZU+RZ#qSCE+qqW0{=H z1qS*_36*fxlQkeu#l!c`)%g??1wNjMXh(rVj|QekAVfL*k`Y4;I~| z{KGF{$f?<|b@=>J7>c#3wVHIt6vh$?zqdq<`)+DKG4<qfQcxxg)np3dF7EtYM3?^_SB`1p znEdd_zspvJJxh|uA0HM16_`+x_Ca;6SZ`+_+L}r*{k)ZmXd`pGCDS`x*1K!QYk0@W zo>Ia7h%|6k?7l<(lii0oExHu^6_hn45Cx=i0DCC{Zn)vG%B#vV$v;p3{gN=relOwx zWYr<+C#&!f!FllPyI5vISv*UzXC9$I1e^i1CrPDS1-V!i<- z=O_e6VQ0dXSeg)ec3(;d@06v9)@Row9T&Jk0u zU;v)I9OZuHv0p#FS%P}ixy_TY0f9>U5k^9Qq}u_urswwE^YzA8(xha)&hCgGsQj(C zC5yP~ep|lb#}1@@bVB=Wdu6KOk#E_aXdJ@E#)S~s!qsQdG4hw95|NicOpi}5#Ye6I z@l6TfIgI8HRVJgK6B%-($PK^N2pKUj^x1GG-1pXJKSeXNEYK(4KWTY0N->r+)|j@3 zS_DWI#{o*=?EJb2P+R@_5!qfZ1JQQ{7h9nF4ta^&>RZ6j0Ado)r4XL>lPd$TZRaU3 zv2R#eQ45w0n0EVVR+3tfQWU0bMpC=QM$y9e9%0KVL7VgPcoXnKsT1HWZD8pP;>mVT z%TXK3N8JJ(n6!iD9e-)xY{qiU@@Hsiz6fBTVidUZ@Mi>cYT!%XCAR`+G44S2%` z-)1*fX9Cek$UNT3wU>4*44idZcz>{&{Ncw%2kT;tm*t~mN9qIhDtlJL(O)FrcLc(Y}R9xMV_ z@*dUElyKYAY)x@nuBf@pBQ-0zZfo@Q(s%mVw*lWpJ_`H z%MiFVr^~jbhjyEB&1J>&bd)Ocb1!XKn%bPDbj6r=ZxxVK8Lhv2u@PJRor(_sJX(US zzkvpV)XX`9xB9WeMd?wm0xfo^y3h{LbvpbjrTkD;bdbjW=|TK0UfCmt&h+rwV#Jbg zHY`M+JDyUe0N(l2anVgw|I_l}E1Ah8zFmEQz%E_B$?EU5R=l6lk&J9vMtKmFltp7l zZwsNQN(Fu5V*Wlqe2}8<+JqYuG%LIK)H^6V2DU;b$3y# zyTDx-J=OVpgOjb#NPic+hdS6#A?iqvP?DM;0n5ncc`zsXIh@FY0&%g!m1ie{INymO zOQKq8j}R|gettJ*>rFJ(zJ@*3`4SpHMpD$&cqqQg+C#qaZ&a$iw?rYBmWx|HFDEZ` zFtJso?l^DYtBj@^*HNY_Y|U?F^qoFE17}+2r9Dwml5)C|f_<+mpsF`?e_m0napXNk z5Z9ZxWzPm!#6SI5P5~ThvG^%o9wr~7@QrsvkUh&uf{4DS4JZWn>{sph2q5UEw@YSA z%^cip5g3Fl*N(9p${(XX1JZmRro|a4&2Ly2i}h_XQbRPr>3vxUfmnP4?2ukNL(a-x ztKZ4(Zhth@utI2QY;OWSUCvWFV6v0Dl6%Y=*(o_o5Z)vw3TE7^<;s-``d?PDl2N zIMLF7;zk<}!fJIdpKmWh$K(_71jD$t!U1uI_fEJ z2~tzc;sUUUDQ3KRhhj&0&E7r;T;-3wDCb|o6|Si`dxhKoDFMI%7>>>^`H|GzB7~q-_RU8yX)h- zk#rFo7JJoujt5Tq#3Nv0y36-3Qwx#)Vz(4JFhaY@c(isdThO#NF&e`0xQ~p)=O;Pq z=XEYC070c_`9CB8mIKgHFF+9hU|n5bA^sgh9bXlq8L9{EeSvMy2+@!|z0`lAwiF0( z+^%R&nYnMG2w=M={GV-{@)XP($_uKYjRRkY0Bx(0W)Io+dELeRw9DGA1zPRkt;SS1 z6Et9a#Ew>f8LbpW1Q43Xj!&ug0q;8GlSPRG&-OnZtb_~>P)@(=$g_P77~~TSG9DmU zxSJ|y`c+F)jSx`wedMH)&{}r(Zr@kf7m5N|p|eXb6k$NI4Hz+@a9*neUrUPx)(wp% zc_7-o>aoM#J*~5U-fu2GfuqeKX+dgs(>_6EE6nn2#B3}m$Q3_cc0?AskrYDnXxOISG$eQY+ zck)WvlL)R0S4Z%Fr8rqUO;?r0lt>q==8`;1GbJxqvl)T2<_}z@Zae<*8h_E2M~S~} zxW0T!+J>8%+4!A5FH@7Oke^Vs7S^nXmK}}Q2pvp+X8tTtrXpZFp>g+bFMZ~D`iZwH zwQ7+qyP^VvOo$5GjtlttAe*}RXHMebRZM*Uf+-hhZY(JMLV|b(_P+j~lFs}YcPa_m zH0QRgnAnOD3}SEKUZJvwoQZ}B(=dCQ*}p>RLISAKSmFXs;8mQR7$;FAtN*8Wp#Ufkdu(K<1XB4^9_58@yB42itUZXu!28~- zzwQbFKl9^Ss65b`f>dz`h$M_Rb@pgOpB)b=>k(+`FlS-|>mofk6uE&hD;gN2Zp zUcw;w=znoMNh=GPjjBACM~2^O=Q@;D{h>XtGMKGyX}m#&t}wq zcii~B1#P?j%HZnL?e0O8GqFLHcg(8t>RMy@Zrb{TzuM-RQ@|6NeL&urj=K~J5NMEk zDQRa2Ldj#E9j#J*o!|07DxGg@W0R3|H*GkZoBNvA%6PghX->Oc;C!1M2w9~4y{^D! z!>NI(-%SM^{fWy=0zB{dzw_zBXJ58Svi#f#SKJ+-?cxp5-HwuP0v534;Zdw(+6K6(BAY34G69EbX!E7jtWR8Kvzz`r5YH9Kia`T7va7y8JY-?SU6j69iW!!met8xH z0KolEh4AHA4yb_Ljj1ZPN8SH>YPuWF*L>D}M?!@sPFVXPzf67W71P!O@nM(mNU-#6 zi#uL+By_x+Qn83`nKI}#Rw(3lKrTojWKnp_tk5S?r2V82#gF@at}3f!2CO1*y>aQ8 zKOl=*a@z_imCCpQ{41B%uoeH>M9RAB+rpzI=aMpW{TWaYRYq2cd#3*8IZOH7>QQyB3AZZUIbcY*(Wef-PRoyq3}#sJBcOQ)>2u0D@qE z4G=n_tQ>gBD~|`uGWFOt?N8g&qlrMulQ6!K+E#i2~ zVx*a3q`)V;tdaWH;<`;Pn#Z+Q@AlTU%n+L#&A zzM>|jg(8-wv&*wiVER3C0KU@8K#rzEF_E9B`HrFr;__9}(y`jU757bqcDW150!oPV pWdh(})BpDz<^SJ*yDOjZRMYz36hsV~14lxGK1wM_mc#UZ{2!N<+hqU% literal 0 HcmV?d00001 diff --git a/previews/PR663/assets/logo.svg b/previews/PR663/assets/logo.svg new file mode 100644 index 000000000..1e8f0ad0a --- /dev/null +++ b/previews/PR663/assets/logo.svg @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/assets/make_logo.jl b/previews/PR663/assets/make_logo.jl new file mode 100644 index 000000000..c023c308f --- /dev/null +++ b/previews/PR663/assets/make_logo.jl @@ -0,0 +1,84 @@ +using Pkg: @pkg_str +# For reproducability only dependency this has is Luxor, +# and it was created with Luxor v1.6.0 +pkg"add Luxor@v1.6" + +using Luxor +using Random + +const bridge_len = 50 + +function chain(jiggle=0) + shaky_rotate(θ) = rotate(θ + jiggle * (rand() - 0.5)) + + ### 1 + shaky_rotate(0) + sethue(Luxor.julia_red) + link() + m1 = getmatrix() + + ### 2 + sethue(Luxor.julia_green) + translate(-50, 130) + shaky_rotate(π / 3) + link() + m2 = getmatrix() + + setmatrix(m1) + sethue(Luxor.julia_red) + overlap(-1.3π) + setmatrix(m2) + + ### 3 + shaky_rotate(-π / 3) + translate(-120, 80) + sethue(Luxor.julia_purple) + link() + + setmatrix(m2) + setcolor(Luxor.julia_green) + return overlap(-1.5π) +end + +function link() + sector(50, 90, π, 0, :fill) + sector(Point(0, bridge_len), 50, 90, 0, -π, :fill) + + rect(50, -3, 40, bridge_len + 6, :fill) + rect(-50 - 40, -3, 40, bridge_len + 6, :fill) + + sethue("black") + move(Point(-50, bridge_len)) + arc(Point(0, 0), 50, π, 0, :stoke) + arc(Point(0, bridge_len), 50, 0, -π, :stroke) + + move(Point(-90, bridge_len)) + arc(Point(0, 0), 90, π, 0, :stoke) + arc(Point(0, bridge_len), 90, 0, -π, :stroke) + return strokepath() +end + +function overlap(ang_end) + sector(Point(0, bridge_len), 50, 90, -0.0, ang_end, :fill) + sethue("black") + arc(Point(0, bridge_len), 50, 0, ang_end, :stoke) + move(Point(90, bridge_len)) + arc(Point(0, bridge_len), 90, 0, ang_end, :stoke) + + return strokepath() +end + +# Actually draw it + +function save_logo(filename) + Random.seed!(16) + Drawing(450, 450, filename) + origin() + translate(50, -130) + chain(0.5) + finish() + return preview() +end + +save_logo("logo.svg") +save_logo("logo.png") diff --git a/previews/PR663/assets/themes/documenter-dark.css b/previews/PR663/assets/themes/documenter-dark.css new file mode 100644 index 000000000..9f5449f4b --- /dev/null +++ b/previews/PR663/assets/themes/documenter-dark.css @@ -0,0 +1,7 @@ +html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-ellipsis,html.theme--documenter-dark .file-cta,html.theme--documenter-dark .file-name,html.theme--documenter-dark .select select,html.theme--documenter-dark .textarea,html.theme--documenter-dark .input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark .button{-moz-appearance:none;-webkit-appearance:none;align-items:center;border:1px solid transparent;border-radius:.4em;box-shadow:none;display:inline-flex;font-size:1rem;height:2.5em;justify-content:flex-start;line-height:1.5;padding-bottom:calc(0.5em - 1px);padding-left:calc(0.75em - 1px);padding-right:calc(0.75em - 1px);padding-top:calc(0.5em - 1px);position:relative;vertical-align:top}html.theme--documenter-dark .pagination-previous:focus,html.theme--documenter-dark .pagination-next:focus,html.theme--documenter-dark .pagination-link:focus,html.theme--documenter-dark .pagination-ellipsis:focus,html.theme--documenter-dark .file-cta:focus,html.theme--documenter-dark .file-name:focus,html.theme--documenter-dark .select select:focus,html.theme--documenter-dark .textarea:focus,html.theme--documenter-dark .input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:focus,html.theme--documenter-dark .button:focus,html.theme--documenter-dark .is-focused.pagination-previous,html.theme--documenter-dark .is-focused.pagination-next,html.theme--documenter-dark .is-focused.pagination-link,html.theme--documenter-dark .is-focused.pagination-ellipsis,html.theme--documenter-dark .is-focused.file-cta,html.theme--documenter-dark .is-focused.file-name,html.theme--documenter-dark .select select.is-focused,html.theme--documenter-dark .is-focused.textarea,html.theme--documenter-dark .is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-focused.button,html.theme--documenter-dark .pagination-previous:active,html.theme--documenter-dark .pagination-next:active,html.theme--documenter-dark .pagination-link:active,html.theme--documenter-dark .pagination-ellipsis:active,html.theme--documenter-dark .file-cta:active,html.theme--documenter-dark .file-name:active,html.theme--documenter-dark .select select:active,html.theme--documenter-dark .textarea:active,html.theme--documenter-dark .input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:active,html.theme--documenter-dark .button:active,html.theme--documenter-dark .is-active.pagination-previous,html.theme--documenter-dark .is-active.pagination-next,html.theme--documenter-dark .is-active.pagination-link,html.theme--documenter-dark .is-active.pagination-ellipsis,html.theme--documenter-dark .is-active.file-cta,html.theme--documenter-dark .is-active.file-name,html.theme--documenter-dark .select select.is-active,html.theme--documenter-dark .is-active.textarea,html.theme--documenter-dark .is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active,html.theme--documenter-dark .is-active.button{outline:none}html.theme--documenter-dark .pagination-previous[disabled],html.theme--documenter-dark .pagination-next[disabled],html.theme--documenter-dark .pagination-link[disabled],html.theme--documenter-dark .pagination-ellipsis[disabled],html.theme--documenter-dark .file-cta[disabled],html.theme--documenter-dark .file-name[disabled],html.theme--documenter-dark .select select[disabled],html.theme--documenter-dark .textarea[disabled],html.theme--documenter-dark .input[disabled],html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled],html.theme--documenter-dark .button[disabled],fieldset[disabled] html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark fieldset[disabled] .pagination-previous,fieldset[disabled] html.theme--documenter-dark .pagination-next,html.theme--documenter-dark fieldset[disabled] .pagination-next,fieldset[disabled] html.theme--documenter-dark .pagination-link,html.theme--documenter-dark fieldset[disabled] .pagination-link,fieldset[disabled] html.theme--documenter-dark .pagination-ellipsis,html.theme--documenter-dark fieldset[disabled] .pagination-ellipsis,fieldset[disabled] html.theme--documenter-dark .file-cta,html.theme--documenter-dark fieldset[disabled] .file-cta,fieldset[disabled] html.theme--documenter-dark .file-name,html.theme--documenter-dark fieldset[disabled] .file-name,fieldset[disabled] html.theme--documenter-dark .select select,fieldset[disabled] html.theme--documenter-dark .textarea,fieldset[disabled] html.theme--documenter-dark .input,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark fieldset[disabled] .select select,html.theme--documenter-dark .select fieldset[disabled] select,html.theme--documenter-dark fieldset[disabled] .textarea,html.theme--documenter-dark fieldset[disabled] .input,html.theme--documenter-dark fieldset[disabled] #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar fieldset[disabled] form.docs-search>input,fieldset[disabled] html.theme--documenter-dark .button,html.theme--documenter-dark fieldset[disabled] .button{cursor:not-allowed}html.theme--documenter-dark .tabs,html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-ellipsis,html.theme--documenter-dark .breadcrumb,html.theme--documenter-dark .file,html.theme--documenter-dark .button,.is-unselectable{-webkit-touch-callout:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}html.theme--documenter-dark .navbar-link:not(.is-arrowless)::after,html.theme--documenter-dark .select:not(.is-multiple):not(.is-loading)::after{border:3px solid rgba(0,0,0,0);border-radius:2px;border-right:0;border-top:0;content:" ";display:block;height:0.625em;margin-top:-0.4375em;pointer-events:none;position:absolute;top:50%;transform:rotate(-45deg);transform-origin:center;width:0.625em}html.theme--documenter-dark .admonition:not(:last-child),html.theme--documenter-dark .tabs:not(:last-child),html.theme--documenter-dark .pagination:not(:last-child),html.theme--documenter-dark .message:not(:last-child),html.theme--documenter-dark .level:not(:last-child),html.theme--documenter-dark .breadcrumb:not(:last-child),html.theme--documenter-dark .block:not(:last-child),html.theme--documenter-dark .title:not(:last-child),html.theme--documenter-dark .subtitle:not(:last-child),html.theme--documenter-dark .table-container:not(:last-child),html.theme--documenter-dark .table:not(:last-child),html.theme--documenter-dark .progress:not(:last-child),html.theme--documenter-dark .notification:not(:last-child),html.theme--documenter-dark .content:not(:last-child),html.theme--documenter-dark .box:not(:last-child){margin-bottom:1.5rem}html.theme--documenter-dark .modal-close,html.theme--documenter-dark .delete{-webkit-touch-callout:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;-moz-appearance:none;-webkit-appearance:none;background-color:rgba(10,10,10,0.2);border:none;border-radius:9999px;cursor:pointer;pointer-events:auto;display:inline-block;flex-grow:0;flex-shrink:0;font-size:0;height:20px;max-height:20px;max-width:20px;min-height:20px;min-width:20px;outline:none;position:relative;vertical-align:top;width:20px}html.theme--documenter-dark .modal-close::before,html.theme--documenter-dark .delete::before,html.theme--documenter-dark .modal-close::after,html.theme--documenter-dark .delete::after{background-color:#fff;content:"";display:block;left:50%;position:absolute;top:50%;transform:translateX(-50%) translateY(-50%) rotate(45deg);transform-origin:center center}html.theme--documenter-dark .modal-close::before,html.theme--documenter-dark .delete::before{height:2px;width:50%}html.theme--documenter-dark .modal-close::after,html.theme--documenter-dark .delete::after{height:50%;width:2px}html.theme--documenter-dark .modal-close:hover,html.theme--documenter-dark .delete:hover,html.theme--documenter-dark .modal-close:focus,html.theme--documenter-dark .delete:focus{background-color:rgba(10,10,10,0.3)}html.theme--documenter-dark .modal-close:active,html.theme--documenter-dark .delete:active{background-color:rgba(10,10,10,0.4)}html.theme--documenter-dark .is-small.modal-close,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.modal-close,html.theme--documenter-dark .is-small.delete,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.delete{height:16px;max-height:16px;max-width:16px;min-height:16px;min-width:16px;width:16px}html.theme--documenter-dark .is-medium.modal-close,html.theme--documenter-dark .is-medium.delete{height:24px;max-height:24px;max-width:24px;min-height:24px;min-width:24px;width:24px}html.theme--documenter-dark .is-large.modal-close,html.theme--documenter-dark .is-large.delete{height:32px;max-height:32px;max-width:32px;min-height:32px;min-width:32px;width:32px}html.theme--documenter-dark .control.is-loading::after,html.theme--documenter-dark .select.is-loading::after,html.theme--documenter-dark .loader,html.theme--documenter-dark .button.is-loading::after{animation:spinAround 500ms infinite linear;border:2px solid #dbdee0;border-radius:9999px;border-right-color:transparent;border-top-color:transparent;content:"";display:block;height:1em;position:relative;width:1em}html.theme--documenter-dark .hero-video,html.theme--documenter-dark .modal-background,html.theme--documenter-dark .modal,html.theme--documenter-dark .image.is-square img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-square img,html.theme--documenter-dark .image.is-square .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-square .has-ratio,html.theme--documenter-dark .image.is-1by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by1 img,html.theme--documenter-dark .image.is-1by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by1 .has-ratio,html.theme--documenter-dark .image.is-5by4 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by4 img,html.theme--documenter-dark .image.is-5by4 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by4 .has-ratio,html.theme--documenter-dark .image.is-4by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by3 img,html.theme--documenter-dark .image.is-4by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by3 .has-ratio,html.theme--documenter-dark .image.is-3by2 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by2 img,html.theme--documenter-dark .image.is-3by2 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by2 .has-ratio,html.theme--documenter-dark .image.is-5by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by3 img,html.theme--documenter-dark .image.is-5by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by3 .has-ratio,html.theme--documenter-dark .image.is-16by9 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16by9 img,html.theme--documenter-dark .image.is-16by9 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16by9 .has-ratio,html.theme--documenter-dark .image.is-2by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by1 img,html.theme--documenter-dark .image.is-2by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by1 .has-ratio,html.theme--documenter-dark .image.is-3by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by1 img,html.theme--documenter-dark .image.is-3by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by1 .has-ratio,html.theme--documenter-dark .image.is-4by5 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by5 img,html.theme--documenter-dark .image.is-4by5 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by5 .has-ratio,html.theme--documenter-dark .image.is-3by4 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by4 img,html.theme--documenter-dark .image.is-3by4 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by4 .has-ratio,html.theme--documenter-dark .image.is-2by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by3 img,html.theme--documenter-dark .image.is-2by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by3 .has-ratio,html.theme--documenter-dark .image.is-3by5 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by5 img,html.theme--documenter-dark .image.is-3by5 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by5 .has-ratio,html.theme--documenter-dark .image.is-9by16 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-9by16 img,html.theme--documenter-dark .image.is-9by16 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-9by16 .has-ratio,html.theme--documenter-dark .image.is-1by2 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by2 img,html.theme--documenter-dark .image.is-1by2 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by2 .has-ratio,html.theme--documenter-dark .image.is-1by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by3 img,html.theme--documenter-dark .image.is-1by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by3 .has-ratio,.is-overlay{bottom:0;left:0;position:absolute;right:0;top:0}html.theme--documenter-dark .navbar-burger{-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;color:currentColor;font-family:inherit;font-size:1em;margin:0;padding:0}/*! minireset.css v0.0.6 | MIT License | github.com/jgthms/minireset.css */html,body,p,ol,ul,li,dl,dt,dd,blockquote,figure,fieldset,legend,textarea,pre,iframe,hr,h1,h2,h3,h4,h5,h6{margin:0;padding:0}h1,h2,h3,h4,h5,h6{font-size:100%;font-weight:normal}ul{list-style:none}button,input,select,textarea{margin:0}html{box-sizing:border-box}*,*::before,*::after{box-sizing:inherit}img,video{height:auto;max-width:100%}iframe{border:0}table{border-collapse:collapse;border-spacing:0}td,th{padding:0}td:not([align]),th:not([align]){text-align:inherit}.has-text-white{color:#fff !important}a.has-text-white:hover,a.has-text-white:focus{color:#e6e6e6 !important}.has-background-white{background-color:#fff !important}.has-text-black{color:#0a0a0a !important}a.has-text-black:hover,a.has-text-black:focus{color:#000 !important}.has-background-black{background-color:#0a0a0a !important}.has-text-light{color:#ecf0f1 !important}a.has-text-light:hover,a.has-text-light:focus{color:#cfd9db !important}.has-background-light{background-color:#ecf0f1 !important}.has-text-dark{color:#282f2f !important}a.has-text-dark:hover,a.has-text-dark:focus{color:#111414 !important}.has-background-dark{background-color:#282f2f !important}.has-text-primary{color:#375a7f !important}a.has-text-primary:hover,a.has-text-primary:focus{color:#28415b !important}.has-background-primary{background-color:#375a7f !important}.has-text-primary-light{color:#f1f5f9 !important}a.has-text-primary-light:hover,a.has-text-primary-light:focus{color:#cddbe9 !important}.has-background-primary-light{background-color:#f1f5f9 !important}.has-text-primary-dark{color:#4d7eb2 !important}a.has-text-primary-dark:hover,a.has-text-primary-dark:focus{color:#7198c1 !important}.has-background-primary-dark{background-color:#4d7eb2 !important}.has-text-link{color:#1abc9c !important}a.has-text-link:hover,a.has-text-link:focus{color:#148f77 !important}.has-background-link{background-color:#1abc9c !important}.has-text-link-light{color:#edfdf9 !important}a.has-text-link-light:hover,a.has-text-link-light:focus{color:#c0f6ec !important}.has-background-link-light{background-color:#edfdf9 !important}.has-text-link-dark{color:#15987e !important}a.has-text-link-dark:hover,a.has-text-link-dark:focus{color:#1bc5a4 !important}.has-background-link-dark{background-color:#15987e !important}.has-text-info{color:#024c7d !important}a.has-text-info:hover,a.has-text-info:focus{color:#012d4b !important}.has-background-info{background-color:#024c7d !important}.has-text-info-light{color:#ebf7ff !important}a.has-text-info-light:hover,a.has-text-info-light:focus{color:#b9e2fe !important}.has-background-info-light{background-color:#ebf7ff !important}.has-text-info-dark{color:#0e9dfb !important}a.has-text-info-dark:hover,a.has-text-info-dark:focus{color:#40b1fc !important}.has-background-info-dark{background-color:#0e9dfb !important}.has-text-success{color:#008438 !important}a.has-text-success:hover,a.has-text-success:focus{color:#005122 !important}.has-background-success{background-color:#008438 !important}.has-text-success-light{color:#ebfff3 !important}a.has-text-success-light:hover,a.has-text-success-light:focus{color:#b8ffd6 !important}.has-background-success-light{background-color:#ebfff3 !important}.has-text-success-dark{color:#00eb64 !important}a.has-text-success-dark:hover,a.has-text-success-dark:focus{color:#1fff7e !important}.has-background-success-dark{background-color:#00eb64 !important}.has-text-warning{color:#ad8100 !important}a.has-text-warning:hover,a.has-text-warning:focus{color:#7a5b00 !important}.has-background-warning{background-color:#ad8100 !important}.has-text-warning-light{color:#fffaeb !important}a.has-text-warning-light:hover,a.has-text-warning-light:focus{color:#ffedb8 !important}.has-background-warning-light{background-color:#fffaeb !important}.has-text-warning-dark{color:#d19c00 !important}a.has-text-warning-dark:hover,a.has-text-warning-dark:focus{color:#ffbf05 !important}.has-background-warning-dark{background-color:#d19c00 !important}.has-text-danger{color:#9e1b0d !important}a.has-text-danger:hover,a.has-text-danger:focus{color:#6f1309 !important}.has-background-danger{background-color:#9e1b0d !important}.has-text-danger-light{color:#fdeeec !important}a.has-text-danger-light:hover,a.has-text-danger-light:focus{color:#fac3bd !important}.has-background-danger-light{background-color:#fdeeec !important}.has-text-danger-dark{color:#ec311d !important}a.has-text-danger-dark:hover,a.has-text-danger-dark:focus{color:#f05c4c !important}.has-background-danger-dark{background-color:#ec311d !important}.has-text-black-bis{color:#121212 !important}.has-background-black-bis{background-color:#121212 !important}.has-text-black-ter{color:#242424 !important}.has-background-black-ter{background-color:#242424 !important}.has-text-grey-darker{color:#282f2f !important}.has-background-grey-darker{background-color:#282f2f !important}.has-text-grey-dark{color:#343c3d !important}.has-background-grey-dark{background-color:#343c3d !important}.has-text-grey{color:#5e6d6f !important}.has-background-grey{background-color:#5e6d6f !important}.has-text-grey-light{color:#8c9b9d !important}.has-background-grey-light{background-color:#8c9b9d !important}.has-text-grey-lighter{color:#dbdee0 !important}.has-background-grey-lighter{background-color:#dbdee0 !important}.has-text-white-ter{color:#ecf0f1 !important}.has-background-white-ter{background-color:#ecf0f1 !important}.has-text-white-bis{color:#fafafa !important}.has-background-white-bis{background-color:#fafafa !important}.is-flex-direction-row{flex-direction:row !important}.is-flex-direction-row-reverse{flex-direction:row-reverse !important}.is-flex-direction-column{flex-direction:column !important}.is-flex-direction-column-reverse{flex-direction:column-reverse !important}.is-flex-wrap-nowrap{flex-wrap:nowrap !important}.is-flex-wrap-wrap{flex-wrap:wrap !important}.is-flex-wrap-wrap-reverse{flex-wrap:wrap-reverse !important}.is-justify-content-flex-start{justify-content:flex-start !important}.is-justify-content-flex-end{justify-content:flex-end !important}.is-justify-content-center{justify-content:center !important}.is-justify-content-space-between{justify-content:space-between !important}.is-justify-content-space-around{justify-content:space-around !important}.is-justify-content-space-evenly{justify-content:space-evenly !important}.is-justify-content-start{justify-content:start !important}.is-justify-content-end{justify-content:end !important}.is-justify-content-left{justify-content:left !important}.is-justify-content-right{justify-content:right !important}.is-align-content-flex-start{align-content:flex-start !important}.is-align-content-flex-end{align-content:flex-end !important}.is-align-content-center{align-content:center !important}.is-align-content-space-between{align-content:space-between !important}.is-align-content-space-around{align-content:space-around !important}.is-align-content-space-evenly{align-content:space-evenly !important}.is-align-content-stretch{align-content:stretch !important}.is-align-content-start{align-content:start !important}.is-align-content-end{align-content:end !important}.is-align-content-baseline{align-content:baseline !important}.is-align-items-stretch{align-items:stretch !important}.is-align-items-flex-start{align-items:flex-start !important}.is-align-items-flex-end{align-items:flex-end !important}.is-align-items-center{align-items:center !important}.is-align-items-baseline{align-items:baseline !important}.is-align-items-start{align-items:start !important}.is-align-items-end{align-items:end !important}.is-align-items-self-start{align-items:self-start !important}.is-align-items-self-end{align-items:self-end !important}.is-align-self-auto{align-self:auto !important}.is-align-self-flex-start{align-self:flex-start !important}.is-align-self-flex-end{align-self:flex-end !important}.is-align-self-center{align-self:center !important}.is-align-self-baseline{align-self:baseline !important}.is-align-self-stretch{align-self:stretch !important}.is-flex-grow-0{flex-grow:0 !important}.is-flex-grow-1{flex-grow:1 !important}.is-flex-grow-2{flex-grow:2 !important}.is-flex-grow-3{flex-grow:3 !important}.is-flex-grow-4{flex-grow:4 !important}.is-flex-grow-5{flex-grow:5 !important}.is-flex-shrink-0{flex-shrink:0 !important}.is-flex-shrink-1{flex-shrink:1 !important}.is-flex-shrink-2{flex-shrink:2 !important}.is-flex-shrink-3{flex-shrink:3 !important}.is-flex-shrink-4{flex-shrink:4 !important}.is-flex-shrink-5{flex-shrink:5 !important}.is-clearfix::after{clear:both;content:" ";display:table}.is-pulled-left{float:left !important}.is-pulled-right{float:right !important}.is-radiusless{border-radius:0 !important}.is-shadowless{box-shadow:none !important}.is-clickable{cursor:pointer !important;pointer-events:all !important}.is-clipped{overflow:hidden !important}.is-relative{position:relative !important}.is-marginless{margin:0 !important}.is-paddingless{padding:0 !important}.m-0{margin:0 !important}.mt-0{margin-top:0 !important}.mr-0{margin-right:0 !important}.mb-0{margin-bottom:0 !important}.ml-0{margin-left:0 !important}.mx-0{margin-left:0 !important;margin-right:0 !important}.my-0{margin-top:0 !important;margin-bottom:0 !important}.m-1{margin:.25rem !important}.mt-1{margin-top:.25rem !important}.mr-1{margin-right:.25rem !important}.mb-1{margin-bottom:.25rem !important}.ml-1{margin-left:.25rem !important}.mx-1{margin-left:.25rem !important;margin-right:.25rem !important}.my-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.m-2{margin:.5rem !important}.mt-2{margin-top:.5rem !important}.mr-2{margin-right:.5rem !important}.mb-2{margin-bottom:.5rem !important}.ml-2{margin-left:.5rem !important}.mx-2{margin-left:.5rem !important;margin-right:.5rem !important}.my-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.m-3{margin:.75rem !important}.mt-3{margin-top:.75rem !important}.mr-3{margin-right:.75rem !important}.mb-3{margin-bottom:.75rem !important}.ml-3{margin-left:.75rem !important}.mx-3{margin-left:.75rem !important;margin-right:.75rem !important}.my-3{margin-top:.75rem !important;margin-bottom:.75rem !important}.m-4{margin:1rem !important}.mt-4{margin-top:1rem !important}.mr-4{margin-right:1rem !important}.mb-4{margin-bottom:1rem !important}.ml-4{margin-left:1rem !important}.mx-4{margin-left:1rem !important;margin-right:1rem !important}.my-4{margin-top:1rem !important;margin-bottom:1rem !important}.m-5{margin:1.5rem !important}.mt-5{margin-top:1.5rem !important}.mr-5{margin-right:1.5rem !important}.mb-5{margin-bottom:1.5rem !important}.ml-5{margin-left:1.5rem !important}.mx-5{margin-left:1.5rem !important;margin-right:1.5rem !important}.my-5{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.m-6{margin:3rem !important}.mt-6{margin-top:3rem !important}.mr-6{margin-right:3rem !important}.mb-6{margin-bottom:3rem !important}.ml-6{margin-left:3rem !important}.mx-6{margin-left:3rem !important;margin-right:3rem !important}.my-6{margin-top:3rem !important;margin-bottom:3rem !important}.m-auto{margin:auto !important}.mt-auto{margin-top:auto !important}.mr-auto{margin-right:auto !important}.mb-auto{margin-bottom:auto !important}.ml-auto{margin-left:auto !important}.mx-auto{margin-left:auto !important;margin-right:auto !important}.my-auto{margin-top:auto !important;margin-bottom:auto !important}.p-0{padding:0 !important}.pt-0{padding-top:0 !important}.pr-0{padding-right:0 !important}.pb-0{padding-bottom:0 !important}.pl-0{padding-left:0 !important}.px-0{padding-left:0 !important;padding-right:0 !important}.py-0{padding-top:0 !important;padding-bottom:0 !important}.p-1{padding:.25rem !important}.pt-1{padding-top:.25rem !important}.pr-1{padding-right:.25rem !important}.pb-1{padding-bottom:.25rem !important}.pl-1{padding-left:.25rem !important}.px-1{padding-left:.25rem !important;padding-right:.25rem !important}.py-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.p-2{padding:.5rem !important}.pt-2{padding-top:.5rem !important}.pr-2{padding-right:.5rem !important}.pb-2{padding-bottom:.5rem !important}.pl-2{padding-left:.5rem !important}.px-2{padding-left:.5rem !important;padding-right:.5rem !important}.py-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.p-3{padding:.75rem !important}.pt-3{padding-top:.75rem !important}.pr-3{padding-right:.75rem !important}.pb-3{padding-bottom:.75rem !important}.pl-3{padding-left:.75rem !important}.px-3{padding-left:.75rem !important;padding-right:.75rem !important}.py-3{padding-top:.75rem !important;padding-bottom:.75rem !important}.p-4{padding:1rem !important}.pt-4{padding-top:1rem !important}.pr-4{padding-right:1rem !important}.pb-4{padding-bottom:1rem !important}.pl-4{padding-left:1rem !important}.px-4{padding-left:1rem !important;padding-right:1rem !important}.py-4{padding-top:1rem !important;padding-bottom:1rem !important}.p-5{padding:1.5rem !important}.pt-5{padding-top:1.5rem !important}.pr-5{padding-right:1.5rem !important}.pb-5{padding-bottom:1.5rem !important}.pl-5{padding-left:1.5rem !important}.px-5{padding-left:1.5rem !important;padding-right:1.5rem !important}.py-5{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.p-6{padding:3rem !important}.pt-6{padding-top:3rem !important}.pr-6{padding-right:3rem !important}.pb-6{padding-bottom:3rem !important}.pl-6{padding-left:3rem !important}.px-6{padding-left:3rem !important;padding-right:3rem !important}.py-6{padding-top:3rem !important;padding-bottom:3rem !important}.p-auto{padding:auto !important}.pt-auto{padding-top:auto !important}.pr-auto{padding-right:auto !important}.pb-auto{padding-bottom:auto !important}.pl-auto{padding-left:auto !important}.px-auto{padding-left:auto !important;padding-right:auto !important}.py-auto{padding-top:auto !important;padding-bottom:auto !important}.is-size-1{font-size:3rem !important}.is-size-2{font-size:2.5rem !important}.is-size-3{font-size:2rem !important}.is-size-4{font-size:1.5rem !important}.is-size-5{font-size:1.25rem !important}.is-size-6{font-size:1rem !important}.is-size-7,html.theme--documenter-dark .docstring>section>a.docs-sourcelink{font-size:.75rem !important}@media screen and (max-width: 768px){.is-size-1-mobile{font-size:3rem !important}.is-size-2-mobile{font-size:2.5rem !important}.is-size-3-mobile{font-size:2rem !important}.is-size-4-mobile{font-size:1.5rem !important}.is-size-5-mobile{font-size:1.25rem !important}.is-size-6-mobile{font-size:1rem !important}.is-size-7-mobile{font-size:.75rem !important}}@media screen and (min-width: 769px),print{.is-size-1-tablet{font-size:3rem !important}.is-size-2-tablet{font-size:2.5rem !important}.is-size-3-tablet{font-size:2rem !important}.is-size-4-tablet{font-size:1.5rem !important}.is-size-5-tablet{font-size:1.25rem !important}.is-size-6-tablet{font-size:1rem !important}.is-size-7-tablet{font-size:.75rem !important}}@media screen and (max-width: 1055px){.is-size-1-touch{font-size:3rem !important}.is-size-2-touch{font-size:2.5rem !important}.is-size-3-touch{font-size:2rem !important}.is-size-4-touch{font-size:1.5rem !important}.is-size-5-touch{font-size:1.25rem !important}.is-size-6-touch{font-size:1rem !important}.is-size-7-touch{font-size:.75rem !important}}@media screen and (min-width: 1056px){.is-size-1-desktop{font-size:3rem !important}.is-size-2-desktop{font-size:2.5rem !important}.is-size-3-desktop{font-size:2rem !important}.is-size-4-desktop{font-size:1.5rem !important}.is-size-5-desktop{font-size:1.25rem !important}.is-size-6-desktop{font-size:1rem !important}.is-size-7-desktop{font-size:.75rem !important}}@media screen and (min-width: 1216px){.is-size-1-widescreen{font-size:3rem !important}.is-size-2-widescreen{font-size:2.5rem !important}.is-size-3-widescreen{font-size:2rem !important}.is-size-4-widescreen{font-size:1.5rem !important}.is-size-5-widescreen{font-size:1.25rem !important}.is-size-6-widescreen{font-size:1rem !important}.is-size-7-widescreen{font-size:.75rem !important}}@media screen and (min-width: 1408px){.is-size-1-fullhd{font-size:3rem !important}.is-size-2-fullhd{font-size:2.5rem !important}.is-size-3-fullhd{font-size:2rem !important}.is-size-4-fullhd{font-size:1.5rem !important}.is-size-5-fullhd{font-size:1.25rem !important}.is-size-6-fullhd{font-size:1rem !important}.is-size-7-fullhd{font-size:.75rem !important}}.has-text-centered{text-align:center !important}.has-text-justified{text-align:justify !important}.has-text-left{text-align:left !important}.has-text-right{text-align:right !important}@media screen and (max-width: 768px){.has-text-centered-mobile{text-align:center !important}}@media screen and (min-width: 769px),print{.has-text-centered-tablet{text-align:center !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-centered-tablet-only{text-align:center !important}}@media screen and (max-width: 1055px){.has-text-centered-touch{text-align:center !important}}@media screen and (min-width: 1056px){.has-text-centered-desktop{text-align:center !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-centered-desktop-only{text-align:center !important}}@media screen and (min-width: 1216px){.has-text-centered-widescreen{text-align:center !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-centered-widescreen-only{text-align:center !important}}@media screen and (min-width: 1408px){.has-text-centered-fullhd{text-align:center !important}}@media screen and (max-width: 768px){.has-text-justified-mobile{text-align:justify !important}}@media screen and (min-width: 769px),print{.has-text-justified-tablet{text-align:justify !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-justified-tablet-only{text-align:justify !important}}@media screen and (max-width: 1055px){.has-text-justified-touch{text-align:justify !important}}@media screen and (min-width: 1056px){.has-text-justified-desktop{text-align:justify !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-justified-desktop-only{text-align:justify !important}}@media screen and (min-width: 1216px){.has-text-justified-widescreen{text-align:justify !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-justified-widescreen-only{text-align:justify !important}}@media screen and (min-width: 1408px){.has-text-justified-fullhd{text-align:justify !important}}@media screen and (max-width: 768px){.has-text-left-mobile{text-align:left !important}}@media screen and (min-width: 769px),print{.has-text-left-tablet{text-align:left !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-left-tablet-only{text-align:left !important}}@media screen and (max-width: 1055px){.has-text-left-touch{text-align:left !important}}@media screen and (min-width: 1056px){.has-text-left-desktop{text-align:left !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-left-desktop-only{text-align:left !important}}@media screen and (min-width: 1216px){.has-text-left-widescreen{text-align:left !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-left-widescreen-only{text-align:left !important}}@media screen and (min-width: 1408px){.has-text-left-fullhd{text-align:left !important}}@media screen and (max-width: 768px){.has-text-right-mobile{text-align:right !important}}@media screen and (min-width: 769px),print{.has-text-right-tablet{text-align:right !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-right-tablet-only{text-align:right !important}}@media screen and (max-width: 1055px){.has-text-right-touch{text-align:right !important}}@media screen and (min-width: 1056px){.has-text-right-desktop{text-align:right !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-right-desktop-only{text-align:right !important}}@media screen and (min-width: 1216px){.has-text-right-widescreen{text-align:right !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-right-widescreen-only{text-align:right !important}}@media screen and (min-width: 1408px){.has-text-right-fullhd{text-align:right !important}}.is-capitalized{text-transform:capitalize !important}.is-lowercase{text-transform:lowercase !important}.is-uppercase{text-transform:uppercase !important}.is-italic{font-style:italic !important}.is-underlined{text-decoration:underline !important}.has-text-weight-light{font-weight:300 !important}.has-text-weight-normal{font-weight:400 !important}.has-text-weight-medium{font-weight:500 !important}.has-text-weight-semibold{font-weight:600 !important}.has-text-weight-bold{font-weight:700 !important}.is-family-primary{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-secondary{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-sans-serif{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-monospace{font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace !important}.is-family-code{font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace !important}.is-block{display:block !important}@media screen and (max-width: 768px){.is-block-mobile{display:block !important}}@media screen and (min-width: 769px),print{.is-block-tablet{display:block !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-block-tablet-only{display:block !important}}@media screen and (max-width: 1055px){.is-block-touch{display:block !important}}@media screen and (min-width: 1056px){.is-block-desktop{display:block !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-block-desktop-only{display:block !important}}@media screen and (min-width: 1216px){.is-block-widescreen{display:block !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-block-widescreen-only{display:block !important}}@media screen and (min-width: 1408px){.is-block-fullhd{display:block !important}}.is-flex{display:flex !important}@media screen and (max-width: 768px){.is-flex-mobile{display:flex !important}}@media screen and (min-width: 769px),print{.is-flex-tablet{display:flex !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-flex-tablet-only{display:flex !important}}@media screen and (max-width: 1055px){.is-flex-touch{display:flex !important}}@media screen and (min-width: 1056px){.is-flex-desktop{display:flex !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-flex-desktop-only{display:flex !important}}@media screen and (min-width: 1216px){.is-flex-widescreen{display:flex !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-flex-widescreen-only{display:flex !important}}@media screen and (min-width: 1408px){.is-flex-fullhd{display:flex !important}}.is-inline{display:inline !important}@media screen and (max-width: 768px){.is-inline-mobile{display:inline !important}}@media screen and (min-width: 769px),print{.is-inline-tablet{display:inline !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-tablet-only{display:inline !important}}@media screen and (max-width: 1055px){.is-inline-touch{display:inline !important}}@media screen and (min-width: 1056px){.is-inline-desktop{display:inline !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-desktop-only{display:inline !important}}@media screen and (min-width: 1216px){.is-inline-widescreen{display:inline !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-widescreen-only{display:inline !important}}@media screen and (min-width: 1408px){.is-inline-fullhd{display:inline !important}}.is-inline-block{display:inline-block !important}@media screen and (max-width: 768px){.is-inline-block-mobile{display:inline-block !important}}@media screen and (min-width: 769px),print{.is-inline-block-tablet{display:inline-block !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-block-tablet-only{display:inline-block !important}}@media screen and (max-width: 1055px){.is-inline-block-touch{display:inline-block !important}}@media screen and (min-width: 1056px){.is-inline-block-desktop{display:inline-block !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-block-desktop-only{display:inline-block !important}}@media screen and (min-width: 1216px){.is-inline-block-widescreen{display:inline-block !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-block-widescreen-only{display:inline-block !important}}@media screen and (min-width: 1408px){.is-inline-block-fullhd{display:inline-block !important}}.is-inline-flex{display:inline-flex !important}@media screen and (max-width: 768px){.is-inline-flex-mobile{display:inline-flex !important}}@media screen and (min-width: 769px),print{.is-inline-flex-tablet{display:inline-flex !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-flex-tablet-only{display:inline-flex !important}}@media screen and (max-width: 1055px){.is-inline-flex-touch{display:inline-flex !important}}@media screen and (min-width: 1056px){.is-inline-flex-desktop{display:inline-flex !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-flex-desktop-only{display:inline-flex !important}}@media screen and (min-width: 1216px){.is-inline-flex-widescreen{display:inline-flex !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-flex-widescreen-only{display:inline-flex !important}}@media screen and (min-width: 1408px){.is-inline-flex-fullhd{display:inline-flex !important}}.is-hidden{display:none !important}.is-sr-only{border:none !important;clip:rect(0, 0, 0, 0) !important;height:0.01em !important;overflow:hidden !important;padding:0 !important;position:absolute !important;white-space:nowrap !important;width:0.01em !important}@media screen and (max-width: 768px){.is-hidden-mobile{display:none !important}}@media screen and (min-width: 769px),print{.is-hidden-tablet{display:none !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-hidden-tablet-only{display:none !important}}@media screen and (max-width: 1055px){.is-hidden-touch{display:none !important}}@media screen and (min-width: 1056px){.is-hidden-desktop{display:none !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-hidden-desktop-only{display:none !important}}@media screen and (min-width: 1216px){.is-hidden-widescreen{display:none !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-hidden-widescreen-only{display:none !important}}@media screen and (min-width: 1408px){.is-hidden-fullhd{display:none !important}}.is-invisible{visibility:hidden !important}@media screen and (max-width: 768px){.is-invisible-mobile{visibility:hidden !important}}@media screen and (min-width: 769px),print{.is-invisible-tablet{visibility:hidden !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-invisible-tablet-only{visibility:hidden !important}}@media screen and (max-width: 1055px){.is-invisible-touch{visibility:hidden !important}}@media screen and (min-width: 1056px){.is-invisible-desktop{visibility:hidden !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-invisible-desktop-only{visibility:hidden !important}}@media screen and (min-width: 1216px){.is-invisible-widescreen{visibility:hidden !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-invisible-widescreen-only{visibility:hidden !important}}@media screen and (min-width: 1408px){.is-invisible-fullhd{visibility:hidden !important}}html.theme--documenter-dark{/*! + Theme: a11y-dark + Author: @ericwbailey + Maintainer: @ericwbailey + + Based on the Tomorrow Night Eighties theme: https://github.com/isagalaev/highlight.js/blob/master/src/styles/tomorrow-night-eighties.css +*/}html.theme--documenter-dark html{background-color:#1f2424;font-size:16px;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased;min-width:300px;overflow-x:auto;overflow-y:scroll;text-rendering:optimizeLegibility;text-size-adjust:100%}html.theme--documenter-dark article,html.theme--documenter-dark aside,html.theme--documenter-dark figure,html.theme--documenter-dark footer,html.theme--documenter-dark header,html.theme--documenter-dark hgroup,html.theme--documenter-dark section{display:block}html.theme--documenter-dark body,html.theme--documenter-dark button,html.theme--documenter-dark input,html.theme--documenter-dark optgroup,html.theme--documenter-dark select,html.theme--documenter-dark textarea{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif}html.theme--documenter-dark code,html.theme--documenter-dark pre{-moz-osx-font-smoothing:auto;-webkit-font-smoothing:auto;font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace}html.theme--documenter-dark body{color:#fff;font-size:1em;font-weight:400;line-height:1.5}html.theme--documenter-dark a{color:#1abc9c;cursor:pointer;text-decoration:none}html.theme--documenter-dark a strong{color:currentColor}html.theme--documenter-dark a:hover{color:#1dd2af}html.theme--documenter-dark code{background-color:rgba(255,255,255,0.05);color:#ececec;font-size:.875em;font-weight:normal;padding:.1em}html.theme--documenter-dark hr{background-color:#282f2f;border:none;display:block;height:2px;margin:1.5rem 0}html.theme--documenter-dark img{height:auto;max-width:100%}html.theme--documenter-dark input[type="checkbox"],html.theme--documenter-dark input[type="radio"]{vertical-align:baseline}html.theme--documenter-dark small{font-size:.875em}html.theme--documenter-dark span{font-style:inherit;font-weight:inherit}html.theme--documenter-dark strong{color:#f2f2f2;font-weight:700}html.theme--documenter-dark fieldset{border:none}html.theme--documenter-dark pre{-webkit-overflow-scrolling:touch;background-color:#282f2f;color:#fff;font-size:.875em;overflow-x:auto;padding:1.25rem 1.5rem;white-space:pre;word-wrap:normal}html.theme--documenter-dark pre code{background-color:transparent;color:currentColor;font-size:1em;padding:0}html.theme--documenter-dark table td,html.theme--documenter-dark table th{vertical-align:top}html.theme--documenter-dark table td:not([align]),html.theme--documenter-dark table th:not([align]){text-align:inherit}html.theme--documenter-dark table th{color:#f2f2f2}html.theme--documenter-dark .box{background-color:#343c3d;border-radius:8px;box-shadow:none;color:#fff;display:block;padding:1.25rem}html.theme--documenter-dark a.box:hover,html.theme--documenter-dark a.box:focus{box-shadow:0 0.5em 1em -0.125em rgba(10,10,10,0.1),0 0 0 1px #1abc9c}html.theme--documenter-dark a.box:active{box-shadow:inset 0 1px 2px rgba(10,10,10,0.2),0 0 0 1px #1abc9c}html.theme--documenter-dark .button{background-color:#282f2f;border-color:#4c5759;border-width:1px;color:#375a7f;cursor:pointer;justify-content:center;padding-bottom:calc(0.5em - 1px);padding-left:1em;padding-right:1em;padding-top:calc(0.5em - 1px);text-align:center;white-space:nowrap}html.theme--documenter-dark .button strong{color:inherit}html.theme--documenter-dark .button .icon,html.theme--documenter-dark .button .icon.is-small,html.theme--documenter-dark .button #documenter .docs-sidebar form.docs-search>input.icon,html.theme--documenter-dark #documenter .docs-sidebar .button form.docs-search>input.icon,html.theme--documenter-dark .button .icon.is-medium,html.theme--documenter-dark .button .icon.is-large{height:1.5em;width:1.5em}html.theme--documenter-dark .button .icon:first-child:not(:last-child){margin-left:calc(-0.5em - 1px);margin-right:.25em}html.theme--documenter-dark .button .icon:last-child:not(:first-child){margin-left:.25em;margin-right:calc(-0.5em - 1px)}html.theme--documenter-dark .button .icon:first-child:last-child{margin-left:calc(-0.5em - 1px);margin-right:calc(-0.5em - 1px)}html.theme--documenter-dark .button:hover,html.theme--documenter-dark .button.is-hovered{border-color:#8c9b9d;color:#f2f2f2}html.theme--documenter-dark .button:focus,html.theme--documenter-dark .button.is-focused{border-color:#8c9b9d;color:#17a689}html.theme--documenter-dark .button:focus:not(:active),html.theme--documenter-dark .button.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(26,188,156,0.25)}html.theme--documenter-dark .button:active,html.theme--documenter-dark .button.is-active{border-color:#343c3d;color:#f2f2f2}html.theme--documenter-dark .button.is-text{background-color:transparent;border-color:transparent;color:#fff;text-decoration:underline}html.theme--documenter-dark .button.is-text:hover,html.theme--documenter-dark .button.is-text.is-hovered,html.theme--documenter-dark .button.is-text:focus,html.theme--documenter-dark .button.is-text.is-focused{background-color:#282f2f;color:#f2f2f2}html.theme--documenter-dark .button.is-text:active,html.theme--documenter-dark .button.is-text.is-active{background-color:#1d2122;color:#f2f2f2}html.theme--documenter-dark .button.is-text[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-text{background-color:transparent;border-color:transparent;box-shadow:none}html.theme--documenter-dark .button.is-ghost{background:none;border-color:rgba(0,0,0,0);color:#1abc9c;text-decoration:none}html.theme--documenter-dark .button.is-ghost:hover,html.theme--documenter-dark .button.is-ghost.is-hovered{color:#1abc9c;text-decoration:underline}html.theme--documenter-dark .button.is-white{background-color:#fff;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .button.is-white:hover,html.theme--documenter-dark .button.is-white.is-hovered{background-color:#f9f9f9;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .button.is-white:focus,html.theme--documenter-dark .button.is-white.is-focused{border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .button.is-white:focus:not(:active),html.theme--documenter-dark .button.is-white.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}html.theme--documenter-dark .button.is-white:active,html.theme--documenter-dark .button.is-white.is-active{background-color:#f2f2f2;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .button.is-white[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-white{background-color:#fff;border-color:#fff;box-shadow:none}html.theme--documenter-dark .button.is-white.is-inverted{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .button.is-white.is-inverted:hover,html.theme--documenter-dark .button.is-white.is-inverted.is-hovered{background-color:#000}html.theme--documenter-dark .button.is-white.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-white.is-inverted{background-color:#0a0a0a;border-color:transparent;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-white.is-loading::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}html.theme--documenter-dark .button.is-white.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-white.is-outlined:hover,html.theme--documenter-dark .button.is-white.is-outlined.is-hovered,html.theme--documenter-dark .button.is-white.is-outlined:focus,html.theme--documenter-dark .button.is-white.is-outlined.is-focused{background-color:#fff;border-color:#fff;color:#0a0a0a}html.theme--documenter-dark .button.is-white.is-outlined.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-white.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-white.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-white.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-white.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}html.theme--documenter-dark .button.is-white.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-white.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-white.is-inverted.is-outlined{background-color:transparent;border-color:#0a0a0a;color:#0a0a0a}html.theme--documenter-dark .button.is-white.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-focused{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-white.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-white.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-white.is-inverted.is-outlined{background-color:transparent;border-color:#0a0a0a;box-shadow:none;color:#0a0a0a}html.theme--documenter-dark .button.is-black{background-color:#0a0a0a;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-black:hover,html.theme--documenter-dark .button.is-black.is-hovered{background-color:#040404;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-black:focus,html.theme--documenter-dark .button.is-black.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-black:focus:not(:active),html.theme--documenter-dark .button.is-black.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}html.theme--documenter-dark .button.is-black:active,html.theme--documenter-dark .button.is-black.is-active{background-color:#000;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-black[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-black{background-color:#0a0a0a;border-color:#0a0a0a;box-shadow:none}html.theme--documenter-dark .button.is-black.is-inverted{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .button.is-black.is-inverted:hover,html.theme--documenter-dark .button.is-black.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-black.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-black.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#0a0a0a}html.theme--documenter-dark .button.is-black.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-black.is-outlined{background-color:transparent;border-color:#0a0a0a;color:#0a0a0a}html.theme--documenter-dark .button.is-black.is-outlined:hover,html.theme--documenter-dark .button.is-black.is-outlined.is-hovered,html.theme--documenter-dark .button.is-black.is-outlined:focus,html.theme--documenter-dark .button.is-black.is-outlined.is-focused{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}html.theme--documenter-dark .button.is-black.is-outlined.is-loading::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}html.theme--documenter-dark .button.is-black.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-black.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-black.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-black.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-black.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-black.is-outlined{background-color:transparent;border-color:#0a0a0a;box-shadow:none;color:#0a0a0a}html.theme--documenter-dark .button.is-black.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-black.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-focused{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-black.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}html.theme--documenter-dark .button.is-black.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-black.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-light{background-color:#ecf0f1;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light:hover,html.theme--documenter-dark .button.is-light.is-hovered{background-color:#e5eaec;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light:focus,html.theme--documenter-dark .button.is-light.is-focused{border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light:focus:not(:active),html.theme--documenter-dark .button.is-light.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(236,240,241,0.25)}html.theme--documenter-dark .button.is-light:active,html.theme--documenter-dark .button.is-light.is-active{background-color:#dde4e6;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-light{background-color:#ecf0f1;border-color:#ecf0f1;box-shadow:none}html.theme--documenter-dark .button.is-light.is-inverted{background-color:rgba(0,0,0,0.7);color:#ecf0f1}html.theme--documenter-dark .button.is-light.is-inverted:hover,html.theme--documenter-dark .button.is-light.is-inverted.is-hovered{background-color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-light.is-inverted{background-color:rgba(0,0,0,0.7);border-color:transparent;box-shadow:none;color:#ecf0f1}html.theme--documenter-dark .button.is-light.is-loading::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}html.theme--documenter-dark .button.is-light.is-outlined{background-color:transparent;border-color:#ecf0f1;color:#ecf0f1}html.theme--documenter-dark .button.is-light.is-outlined:hover,html.theme--documenter-dark .button.is-light.is-outlined.is-hovered,html.theme--documenter-dark .button.is-light.is-outlined:focus,html.theme--documenter-dark .button.is-light.is-outlined.is-focused{background-color:#ecf0f1;border-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light.is-outlined.is-loading::after{border-color:transparent transparent #ecf0f1 #ecf0f1 !important}html.theme--documenter-dark .button.is-light.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-light.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-light.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-light.is-outlined.is-loading.is-focused::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}html.theme--documenter-dark .button.is-light.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-light.is-outlined{background-color:transparent;border-color:#ecf0f1;box-shadow:none;color:#ecf0f1}html.theme--documenter-dark .button.is-light.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-light.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-focused{background-color:rgba(0,0,0,0.7);color:#ecf0f1}html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-light.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #ecf0f1 #ecf0f1 !important}html.theme--documenter-dark .button.is-light.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-light.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);box-shadow:none;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .button.is-dark,html.theme--documenter-dark .content kbd.button{background-color:#282f2f;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-dark:hover,html.theme--documenter-dark .content kbd.button:hover,html.theme--documenter-dark .button.is-dark.is-hovered,html.theme--documenter-dark .content kbd.button.is-hovered{background-color:#232829;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-dark:focus,html.theme--documenter-dark .content kbd.button:focus,html.theme--documenter-dark .button.is-dark.is-focused,html.theme--documenter-dark .content kbd.button.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-dark:focus:not(:active),html.theme--documenter-dark .content kbd.button:focus:not(:active),html.theme--documenter-dark .button.is-dark.is-focused:not(:active),html.theme--documenter-dark .content kbd.button.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(40,47,47,0.25)}html.theme--documenter-dark .button.is-dark:active,html.theme--documenter-dark .content kbd.button:active,html.theme--documenter-dark .button.is-dark.is-active,html.theme--documenter-dark .content kbd.button.is-active{background-color:#1d2122;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-dark[disabled],html.theme--documenter-dark .content kbd.button[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-dark,fieldset[disabled] html.theme--documenter-dark .content kbd.button{background-color:#282f2f;border-color:#282f2f;box-shadow:none}html.theme--documenter-dark .button.is-dark.is-inverted,html.theme--documenter-dark .content kbd.button.is-inverted{background-color:#fff;color:#282f2f}html.theme--documenter-dark .button.is-dark.is-inverted:hover,html.theme--documenter-dark .content kbd.button.is-inverted:hover,html.theme--documenter-dark .button.is-dark.is-inverted.is-hovered,html.theme--documenter-dark .content kbd.button.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-dark.is-inverted[disabled],html.theme--documenter-dark .content kbd.button.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-dark.is-inverted,fieldset[disabled] html.theme--documenter-dark .content kbd.button.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#282f2f}html.theme--documenter-dark .button.is-dark.is-loading::after,html.theme--documenter-dark .content kbd.button.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-dark.is-outlined,html.theme--documenter-dark .content kbd.button.is-outlined{background-color:transparent;border-color:#282f2f;color:#282f2f}html.theme--documenter-dark .button.is-dark.is-outlined:hover,html.theme--documenter-dark .content kbd.button.is-outlined:hover,html.theme--documenter-dark .button.is-dark.is-outlined.is-hovered,html.theme--documenter-dark .content kbd.button.is-outlined.is-hovered,html.theme--documenter-dark .button.is-dark.is-outlined:focus,html.theme--documenter-dark .content kbd.button.is-outlined:focus,html.theme--documenter-dark .button.is-dark.is-outlined.is-focused,html.theme--documenter-dark .content kbd.button.is-outlined.is-focused{background-color:#282f2f;border-color:#282f2f;color:#fff}html.theme--documenter-dark .button.is-dark.is-outlined.is-loading::after,html.theme--documenter-dark .content kbd.button.is-outlined.is-loading::after{border-color:transparent transparent #282f2f #282f2f !important}html.theme--documenter-dark .button.is-dark.is-outlined.is-loading:hover::after,html.theme--documenter-dark .content kbd.button.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-dark.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .content kbd.button.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-dark.is-outlined.is-loading:focus::after,html.theme--documenter-dark .content kbd.button.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-dark.is-outlined.is-loading.is-focused::after,html.theme--documenter-dark .content kbd.button.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-dark.is-outlined[disabled],html.theme--documenter-dark .content kbd.button.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-dark.is-outlined,fieldset[disabled] html.theme--documenter-dark .content kbd.button.is-outlined{background-color:transparent;border-color:#282f2f;box-shadow:none;color:#282f2f}html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined:hover,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined:focus,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-focused,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-focused{background-color:#fff;color:#282f2f}html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined.is-loading.is-focused::after,html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #282f2f #282f2f !important}html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined[disabled],html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-dark.is-inverted.is-outlined,fieldset[disabled] html.theme--documenter-dark .content kbd.button.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-primary,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink{background-color:#375a7f;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-primary:hover,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:hover,html.theme--documenter-dark .button.is-primary.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-hovered.docs-sourcelink{background-color:#335476;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-primary:focus,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:focus,html.theme--documenter-dark .button.is-primary.is-focused,html.theme--documenter-dark .docstring>section>a.button.is-focused.docs-sourcelink{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-primary:focus:not(:active),html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:focus:not(:active),html.theme--documenter-dark .button.is-primary.is-focused:not(:active),html.theme--documenter-dark .docstring>section>a.button.is-focused.docs-sourcelink:not(:active){box-shadow:0 0 0 0.125em rgba(55,90,127,0.25)}html.theme--documenter-dark .button.is-primary:active,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:active,html.theme--documenter-dark .button.is-primary.is-active,html.theme--documenter-dark .docstring>section>a.button.is-active.docs-sourcelink{background-color:#2f4d6d;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-primary[disabled],html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-primary,fieldset[disabled] html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink{background-color:#375a7f;border-color:#375a7f;box-shadow:none}html.theme--documenter-dark .button.is-primary.is-inverted,html.theme--documenter-dark .docstring>section>a.button.is-inverted.docs-sourcelink{background-color:#fff;color:#375a7f}html.theme--documenter-dark .button.is-primary.is-inverted:hover,html.theme--documenter-dark .docstring>section>a.button.is-inverted.docs-sourcelink:hover,html.theme--documenter-dark .button.is-primary.is-inverted.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-hovered.docs-sourcelink{background-color:#f2f2f2}html.theme--documenter-dark .button.is-primary.is-inverted[disabled],html.theme--documenter-dark .docstring>section>a.button.is-inverted.docs-sourcelink[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-primary.is-inverted,fieldset[disabled] html.theme--documenter-dark .docstring>section>a.button.is-inverted.docs-sourcelink{background-color:#fff;border-color:transparent;box-shadow:none;color:#375a7f}html.theme--documenter-dark .button.is-primary.is-loading::after,html.theme--documenter-dark .docstring>section>a.button.is-loading.docs-sourcelink::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-primary.is-outlined,html.theme--documenter-dark .docstring>section>a.button.is-outlined.docs-sourcelink{background-color:transparent;border-color:#375a7f;color:#375a7f}html.theme--documenter-dark .button.is-primary.is-outlined:hover,html.theme--documenter-dark .docstring>section>a.button.is-outlined.docs-sourcelink:hover,html.theme--documenter-dark .button.is-primary.is-outlined.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-hovered.docs-sourcelink,html.theme--documenter-dark .button.is-primary.is-outlined:focus,html.theme--documenter-dark .docstring>section>a.button.is-outlined.docs-sourcelink:focus,html.theme--documenter-dark .button.is-primary.is-outlined.is-focused,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-focused.docs-sourcelink{background-color:#375a7f;border-color:#375a7f;color:#fff}html.theme--documenter-dark .button.is-primary.is-outlined.is-loading::after,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-loading.docs-sourcelink::after{border-color:transparent transparent #375a7f #375a7f !important}html.theme--documenter-dark .button.is-primary.is-outlined.is-loading:hover::after,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-loading.docs-sourcelink:hover::after,html.theme--documenter-dark .button.is-primary.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-loading.is-hovered.docs-sourcelink::after,html.theme--documenter-dark .button.is-primary.is-outlined.is-loading:focus::after,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-loading.docs-sourcelink:focus::after,html.theme--documenter-dark .button.is-primary.is-outlined.is-loading.is-focused::after,html.theme--documenter-dark .docstring>section>a.button.is-outlined.is-loading.is-focused.docs-sourcelink::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-primary.is-outlined[disabled],html.theme--documenter-dark .docstring>section>a.button.is-outlined.docs-sourcelink[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-primary.is-outlined,fieldset[disabled] html.theme--documenter-dark .docstring>section>a.button.is-outlined.docs-sourcelink{background-color:transparent;border-color:#375a7f;box-shadow:none;color:#375a7f}html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined:hover,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink:hover,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-hovered.docs-sourcelink,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined:focus,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink:focus,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-focused,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-focused.docs-sourcelink{background-color:#fff;color:#375a7f}html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-loading.docs-sourcelink:hover::after,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-loading.is-hovered.docs-sourcelink::after,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-loading.docs-sourcelink:focus::after,html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined.is-loading.is-focused::after,html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.is-loading.is-focused.docs-sourcelink::after{border-color:transparent transparent #375a7f #375a7f !important}html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined[disabled],html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-primary.is-inverted.is-outlined,fieldset[disabled] html.theme--documenter-dark .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-primary.is-light,html.theme--documenter-dark .docstring>section>a.button.is-light.docs-sourcelink{background-color:#f1f5f9;color:#4d7eb2}html.theme--documenter-dark .button.is-primary.is-light:hover,html.theme--documenter-dark .docstring>section>a.button.is-light.docs-sourcelink:hover,html.theme--documenter-dark .button.is-primary.is-light.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-light.is-hovered.docs-sourcelink{background-color:#e8eef5;border-color:transparent;color:#4d7eb2}html.theme--documenter-dark .button.is-primary.is-light:active,html.theme--documenter-dark .docstring>section>a.button.is-light.docs-sourcelink:active,html.theme--documenter-dark .button.is-primary.is-light.is-active,html.theme--documenter-dark .docstring>section>a.button.is-light.is-active.docs-sourcelink{background-color:#dfe8f1;border-color:transparent;color:#4d7eb2}html.theme--documenter-dark .button.is-link{background-color:#1abc9c;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-link:hover,html.theme--documenter-dark .button.is-link.is-hovered{background-color:#18b193;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-link:focus,html.theme--documenter-dark .button.is-link.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-link:focus:not(:active),html.theme--documenter-dark .button.is-link.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(26,188,156,0.25)}html.theme--documenter-dark .button.is-link:active,html.theme--documenter-dark .button.is-link.is-active{background-color:#17a689;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-link[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-link{background-color:#1abc9c;border-color:#1abc9c;box-shadow:none}html.theme--documenter-dark .button.is-link.is-inverted{background-color:#fff;color:#1abc9c}html.theme--documenter-dark .button.is-link.is-inverted:hover,html.theme--documenter-dark .button.is-link.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-link.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-link.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#1abc9c}html.theme--documenter-dark .button.is-link.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-link.is-outlined{background-color:transparent;border-color:#1abc9c;color:#1abc9c}html.theme--documenter-dark .button.is-link.is-outlined:hover,html.theme--documenter-dark .button.is-link.is-outlined.is-hovered,html.theme--documenter-dark .button.is-link.is-outlined:focus,html.theme--documenter-dark .button.is-link.is-outlined.is-focused{background-color:#1abc9c;border-color:#1abc9c;color:#fff}html.theme--documenter-dark .button.is-link.is-outlined.is-loading::after{border-color:transparent transparent #1abc9c #1abc9c !important}html.theme--documenter-dark .button.is-link.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-link.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-link.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-link.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-link.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-link.is-outlined{background-color:transparent;border-color:#1abc9c;box-shadow:none;color:#1abc9c}html.theme--documenter-dark .button.is-link.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-link.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-focused{background-color:#fff;color:#1abc9c}html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-link.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #1abc9c #1abc9c !important}html.theme--documenter-dark .button.is-link.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-link.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-link.is-light{background-color:#edfdf9;color:#15987e}html.theme--documenter-dark .button.is-link.is-light:hover,html.theme--documenter-dark .button.is-link.is-light.is-hovered{background-color:#e2fbf6;border-color:transparent;color:#15987e}html.theme--documenter-dark .button.is-link.is-light:active,html.theme--documenter-dark .button.is-link.is-light.is-active{background-color:#d7f9f3;border-color:transparent;color:#15987e}html.theme--documenter-dark .button.is-info{background-color:#024c7d;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-info:hover,html.theme--documenter-dark .button.is-info.is-hovered{background-color:#024470;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-info:focus,html.theme--documenter-dark .button.is-info.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-info:focus:not(:active),html.theme--documenter-dark .button.is-info.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(2,76,125,0.25)}html.theme--documenter-dark .button.is-info:active,html.theme--documenter-dark .button.is-info.is-active{background-color:#023d64;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-info[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-info{background-color:#024c7d;border-color:#024c7d;box-shadow:none}html.theme--documenter-dark .button.is-info.is-inverted{background-color:#fff;color:#024c7d}html.theme--documenter-dark .button.is-info.is-inverted:hover,html.theme--documenter-dark .button.is-info.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-info.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-info.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#024c7d}html.theme--documenter-dark .button.is-info.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-info.is-outlined{background-color:transparent;border-color:#024c7d;color:#024c7d}html.theme--documenter-dark .button.is-info.is-outlined:hover,html.theme--documenter-dark .button.is-info.is-outlined.is-hovered,html.theme--documenter-dark .button.is-info.is-outlined:focus,html.theme--documenter-dark .button.is-info.is-outlined.is-focused{background-color:#024c7d;border-color:#024c7d;color:#fff}html.theme--documenter-dark .button.is-info.is-outlined.is-loading::after{border-color:transparent transparent #024c7d #024c7d !important}html.theme--documenter-dark .button.is-info.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-info.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-info.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-info.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-info.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-info.is-outlined{background-color:transparent;border-color:#024c7d;box-shadow:none;color:#024c7d}html.theme--documenter-dark .button.is-info.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-info.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-focused{background-color:#fff;color:#024c7d}html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-info.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #024c7d #024c7d !important}html.theme--documenter-dark .button.is-info.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-info.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-info.is-light{background-color:#ebf7ff;color:#0e9dfb}html.theme--documenter-dark .button.is-info.is-light:hover,html.theme--documenter-dark .button.is-info.is-light.is-hovered{background-color:#def2fe;border-color:transparent;color:#0e9dfb}html.theme--documenter-dark .button.is-info.is-light:active,html.theme--documenter-dark .button.is-info.is-light.is-active{background-color:#d2edfe;border-color:transparent;color:#0e9dfb}html.theme--documenter-dark .button.is-success{background-color:#008438;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-success:hover,html.theme--documenter-dark .button.is-success.is-hovered{background-color:#073;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-success:focus,html.theme--documenter-dark .button.is-success.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-success:focus:not(:active),html.theme--documenter-dark .button.is-success.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(0,132,56,0.25)}html.theme--documenter-dark .button.is-success:active,html.theme--documenter-dark .button.is-success.is-active{background-color:#006b2d;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-success[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-success{background-color:#008438;border-color:#008438;box-shadow:none}html.theme--documenter-dark .button.is-success.is-inverted{background-color:#fff;color:#008438}html.theme--documenter-dark .button.is-success.is-inverted:hover,html.theme--documenter-dark .button.is-success.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-success.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-success.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#008438}html.theme--documenter-dark .button.is-success.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-success.is-outlined{background-color:transparent;border-color:#008438;color:#008438}html.theme--documenter-dark .button.is-success.is-outlined:hover,html.theme--documenter-dark .button.is-success.is-outlined.is-hovered,html.theme--documenter-dark .button.is-success.is-outlined:focus,html.theme--documenter-dark .button.is-success.is-outlined.is-focused{background-color:#008438;border-color:#008438;color:#fff}html.theme--documenter-dark .button.is-success.is-outlined.is-loading::after{border-color:transparent transparent #008438 #008438 !important}html.theme--documenter-dark .button.is-success.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-success.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-success.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-success.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-success.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-success.is-outlined{background-color:transparent;border-color:#008438;box-shadow:none;color:#008438}html.theme--documenter-dark .button.is-success.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-success.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-focused{background-color:#fff;color:#008438}html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-success.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #008438 #008438 !important}html.theme--documenter-dark .button.is-success.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-success.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-success.is-light{background-color:#ebfff3;color:#00eb64}html.theme--documenter-dark .button.is-success.is-light:hover,html.theme--documenter-dark .button.is-success.is-light.is-hovered{background-color:#deffec;border-color:transparent;color:#00eb64}html.theme--documenter-dark .button.is-success.is-light:active,html.theme--documenter-dark .button.is-success.is-light.is-active{background-color:#d1ffe5;border-color:transparent;color:#00eb64}html.theme--documenter-dark .button.is-warning{background-color:#ad8100;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-warning:hover,html.theme--documenter-dark .button.is-warning.is-hovered{background-color:#a07700;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-warning:focus,html.theme--documenter-dark .button.is-warning.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-warning:focus:not(:active),html.theme--documenter-dark .button.is-warning.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(173,129,0,0.25)}html.theme--documenter-dark .button.is-warning:active,html.theme--documenter-dark .button.is-warning.is-active{background-color:#946e00;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-warning[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-warning{background-color:#ad8100;border-color:#ad8100;box-shadow:none}html.theme--documenter-dark .button.is-warning.is-inverted{background-color:#fff;color:#ad8100}html.theme--documenter-dark .button.is-warning.is-inverted:hover,html.theme--documenter-dark .button.is-warning.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-warning.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-warning.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#ad8100}html.theme--documenter-dark .button.is-warning.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-warning.is-outlined{background-color:transparent;border-color:#ad8100;color:#ad8100}html.theme--documenter-dark .button.is-warning.is-outlined:hover,html.theme--documenter-dark .button.is-warning.is-outlined.is-hovered,html.theme--documenter-dark .button.is-warning.is-outlined:focus,html.theme--documenter-dark .button.is-warning.is-outlined.is-focused{background-color:#ad8100;border-color:#ad8100;color:#fff}html.theme--documenter-dark .button.is-warning.is-outlined.is-loading::after{border-color:transparent transparent #ad8100 #ad8100 !important}html.theme--documenter-dark .button.is-warning.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-warning.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-warning.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-warning.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-warning.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-warning.is-outlined{background-color:transparent;border-color:#ad8100;box-shadow:none;color:#ad8100}html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-focused{background-color:#fff;color:#ad8100}html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #ad8100 #ad8100 !important}html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-warning.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-warning.is-light{background-color:#fffaeb;color:#d19c00}html.theme--documenter-dark .button.is-warning.is-light:hover,html.theme--documenter-dark .button.is-warning.is-light.is-hovered{background-color:#fff7de;border-color:transparent;color:#d19c00}html.theme--documenter-dark .button.is-warning.is-light:active,html.theme--documenter-dark .button.is-warning.is-light.is-active{background-color:#fff3d1;border-color:transparent;color:#d19c00}html.theme--documenter-dark .button.is-danger{background-color:#9e1b0d;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-danger:hover,html.theme--documenter-dark .button.is-danger.is-hovered{background-color:#92190c;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-danger:focus,html.theme--documenter-dark .button.is-danger.is-focused{border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-danger:focus:not(:active),html.theme--documenter-dark .button.is-danger.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(158,27,13,0.25)}html.theme--documenter-dark .button.is-danger:active,html.theme--documenter-dark .button.is-danger.is-active{background-color:#86170b;border-color:transparent;color:#fff}html.theme--documenter-dark .button.is-danger[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-danger{background-color:#9e1b0d;border-color:#9e1b0d;box-shadow:none}html.theme--documenter-dark .button.is-danger.is-inverted{background-color:#fff;color:#9e1b0d}html.theme--documenter-dark .button.is-danger.is-inverted:hover,html.theme--documenter-dark .button.is-danger.is-inverted.is-hovered{background-color:#f2f2f2}html.theme--documenter-dark .button.is-danger.is-inverted[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-danger.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#9e1b0d}html.theme--documenter-dark .button.is-danger.is-loading::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-danger.is-outlined{background-color:transparent;border-color:#9e1b0d;color:#9e1b0d}html.theme--documenter-dark .button.is-danger.is-outlined:hover,html.theme--documenter-dark .button.is-danger.is-outlined.is-hovered,html.theme--documenter-dark .button.is-danger.is-outlined:focus,html.theme--documenter-dark .button.is-danger.is-outlined.is-focused{background-color:#9e1b0d;border-color:#9e1b0d;color:#fff}html.theme--documenter-dark .button.is-danger.is-outlined.is-loading::after{border-color:transparent transparent #9e1b0d #9e1b0d !important}html.theme--documenter-dark .button.is-danger.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-danger.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-danger.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-danger.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}html.theme--documenter-dark .button.is-danger.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-danger.is-outlined{background-color:transparent;border-color:#9e1b0d;box-shadow:none;color:#9e1b0d}html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined:hover,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-hovered,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined:focus,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-focused{background-color:#fff;color:#9e1b0d}html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-loading:hover::after,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-loading.is-hovered::after,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-loading:focus::after,html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #9e1b0d #9e1b0d !important}html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined[disabled],fieldset[disabled] html.theme--documenter-dark .button.is-danger.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}html.theme--documenter-dark .button.is-danger.is-light{background-color:#fdeeec;color:#ec311d}html.theme--documenter-dark .button.is-danger.is-light:hover,html.theme--documenter-dark .button.is-danger.is-light.is-hovered{background-color:#fce3e0;border-color:transparent;color:#ec311d}html.theme--documenter-dark .button.is-danger.is-light:active,html.theme--documenter-dark .button.is-danger.is-light.is-active{background-color:#fcd8d5;border-color:transparent;color:#ec311d}html.theme--documenter-dark .button.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.button{font-size:.75rem}html.theme--documenter-dark .button.is-small:not(.is-rounded),html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.button:not(.is-rounded){border-radius:3px}html.theme--documenter-dark .button.is-normal{font-size:1rem}html.theme--documenter-dark .button.is-medium{font-size:1.25rem}html.theme--documenter-dark .button.is-large{font-size:1.5rem}html.theme--documenter-dark .button[disabled],fieldset[disabled] html.theme--documenter-dark .button{background-color:#8c9b9d;border-color:#5e6d6f;box-shadow:none;opacity:.5}html.theme--documenter-dark .button.is-fullwidth{display:flex;width:100%}html.theme--documenter-dark .button.is-loading{color:transparent !important;pointer-events:none}html.theme--documenter-dark .button.is-loading::after{position:absolute;left:calc(50% - (1em * 0.5));top:calc(50% - (1em * 0.5));position:absolute !important}html.theme--documenter-dark .button.is-static{background-color:#282f2f;border-color:#5e6d6f;color:#dbdee0;box-shadow:none;pointer-events:none}html.theme--documenter-dark .button.is-rounded,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.button{border-radius:9999px;padding-left:calc(1em + 0.25em);padding-right:calc(1em + 0.25em)}html.theme--documenter-dark .buttons{align-items:center;display:flex;flex-wrap:wrap;justify-content:flex-start}html.theme--documenter-dark .buttons .button{margin-bottom:0.5rem}html.theme--documenter-dark .buttons .button:not(:last-child):not(.is-fullwidth){margin-right:.5rem}html.theme--documenter-dark .buttons:last-child{margin-bottom:-0.5rem}html.theme--documenter-dark .buttons:not(:last-child){margin-bottom:1rem}html.theme--documenter-dark .buttons.are-small .button:not(.is-normal):not(.is-medium):not(.is-large){font-size:.75rem}html.theme--documenter-dark .buttons.are-small .button:not(.is-normal):not(.is-medium):not(.is-large):not(.is-rounded){border-radius:3px}html.theme--documenter-dark .buttons.are-medium .button:not(.is-small):not(.is-normal):not(.is-large){font-size:1.25rem}html.theme--documenter-dark .buttons.are-large .button:not(.is-small):not(.is-normal):not(.is-medium){font-size:1.5rem}html.theme--documenter-dark .buttons.has-addons .button:not(:first-child){border-bottom-left-radius:0;border-top-left-radius:0}html.theme--documenter-dark .buttons.has-addons .button:not(:last-child){border-bottom-right-radius:0;border-top-right-radius:0;margin-right:-1px}html.theme--documenter-dark .buttons.has-addons .button:last-child{margin-right:0}html.theme--documenter-dark .buttons.has-addons .button:hover,html.theme--documenter-dark .buttons.has-addons .button.is-hovered{z-index:2}html.theme--documenter-dark .buttons.has-addons .button:focus,html.theme--documenter-dark .buttons.has-addons .button.is-focused,html.theme--documenter-dark .buttons.has-addons .button:active,html.theme--documenter-dark .buttons.has-addons .button.is-active,html.theme--documenter-dark .buttons.has-addons .button.is-selected{z-index:3}html.theme--documenter-dark .buttons.has-addons .button:focus:hover,html.theme--documenter-dark .buttons.has-addons .button.is-focused:hover,html.theme--documenter-dark .buttons.has-addons .button:active:hover,html.theme--documenter-dark .buttons.has-addons .button.is-active:hover,html.theme--documenter-dark .buttons.has-addons .button.is-selected:hover{z-index:4}html.theme--documenter-dark .buttons.has-addons .button.is-expanded{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .buttons.is-centered{justify-content:center}html.theme--documenter-dark .buttons.is-centered:not(.has-addons) .button:not(.is-fullwidth){margin-left:0.25rem;margin-right:0.25rem}html.theme--documenter-dark .buttons.is-right{justify-content:flex-end}html.theme--documenter-dark .buttons.is-right:not(.has-addons) .button:not(.is-fullwidth){margin-left:0.25rem;margin-right:0.25rem}@media screen and (max-width: 768px){html.theme--documenter-dark .button.is-responsive.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-responsive{font-size:.5625rem}html.theme--documenter-dark .button.is-responsive,html.theme--documenter-dark .button.is-responsive.is-normal{font-size:.65625rem}html.theme--documenter-dark .button.is-responsive.is-medium{font-size:.75rem}html.theme--documenter-dark .button.is-responsive.is-large{font-size:1rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .button.is-responsive.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-responsive{font-size:.65625rem}html.theme--documenter-dark .button.is-responsive,html.theme--documenter-dark .button.is-responsive.is-normal{font-size:.75rem}html.theme--documenter-dark .button.is-responsive.is-medium{font-size:1rem}html.theme--documenter-dark .button.is-responsive.is-large{font-size:1.25rem}}html.theme--documenter-dark .container{flex-grow:1;margin:0 auto;position:relative;width:auto}html.theme--documenter-dark .container.is-fluid{max-width:none !important;padding-left:32px;padding-right:32px;width:100%}@media screen and (min-width: 1056px){html.theme--documenter-dark .container{max-width:992px}}@media screen and (max-width: 1215px){html.theme--documenter-dark .container.is-widescreen:not(.is-max-desktop){max-width:1152px}}@media screen and (max-width: 1407px){html.theme--documenter-dark .container.is-fullhd:not(.is-max-desktop):not(.is-max-widescreen){max-width:1344px}}@media screen and (min-width: 1216px){html.theme--documenter-dark .container:not(.is-max-desktop){max-width:1152px}}@media screen and (min-width: 1408px){html.theme--documenter-dark .container:not(.is-max-desktop):not(.is-max-widescreen){max-width:1344px}}html.theme--documenter-dark .content li+li{margin-top:0.25em}html.theme--documenter-dark .content p:not(:last-child),html.theme--documenter-dark .content dl:not(:last-child),html.theme--documenter-dark .content ol:not(:last-child),html.theme--documenter-dark .content ul:not(:last-child),html.theme--documenter-dark .content blockquote:not(:last-child),html.theme--documenter-dark .content pre:not(:last-child),html.theme--documenter-dark .content table:not(:last-child){margin-bottom:1em}html.theme--documenter-dark .content h1,html.theme--documenter-dark .content h2,html.theme--documenter-dark .content h3,html.theme--documenter-dark .content h4,html.theme--documenter-dark .content h5,html.theme--documenter-dark .content h6{color:#f2f2f2;font-weight:600;line-height:1.125}html.theme--documenter-dark .content h1{font-size:2em;margin-bottom:0.5em}html.theme--documenter-dark .content h1:not(:first-child){margin-top:1em}html.theme--documenter-dark .content h2{font-size:1.75em;margin-bottom:0.5714em}html.theme--documenter-dark .content h2:not(:first-child){margin-top:1.1428em}html.theme--documenter-dark .content h3{font-size:1.5em;margin-bottom:0.6666em}html.theme--documenter-dark .content h3:not(:first-child){margin-top:1.3333em}html.theme--documenter-dark .content h4{font-size:1.25em;margin-bottom:0.8em}html.theme--documenter-dark .content h5{font-size:1.125em;margin-bottom:0.8888em}html.theme--documenter-dark .content h6{font-size:1em;margin-bottom:1em}html.theme--documenter-dark .content blockquote{background-color:#282f2f;border-left:5px solid #5e6d6f;padding:1.25em 1.5em}html.theme--documenter-dark .content ol{list-style-position:outside;margin-left:2em;margin-top:1em}html.theme--documenter-dark .content ol:not([type]){list-style-type:decimal}html.theme--documenter-dark .content ol.is-lower-alpha:not([type]){list-style-type:lower-alpha}html.theme--documenter-dark .content ol.is-lower-roman:not([type]){list-style-type:lower-roman}html.theme--documenter-dark .content ol.is-upper-alpha:not([type]){list-style-type:upper-alpha}html.theme--documenter-dark .content ol.is-upper-roman:not([type]){list-style-type:upper-roman}html.theme--documenter-dark .content ul{list-style:disc outside;margin-left:2em;margin-top:1em}html.theme--documenter-dark .content ul ul{list-style-type:circle;margin-top:0.5em}html.theme--documenter-dark .content ul ul ul{list-style-type:square}html.theme--documenter-dark .content dd{margin-left:2em}html.theme--documenter-dark .content figure{margin-left:2em;margin-right:2em;text-align:center}html.theme--documenter-dark .content figure:not(:first-child){margin-top:2em}html.theme--documenter-dark .content figure:not(:last-child){margin-bottom:2em}html.theme--documenter-dark .content figure img{display:inline-block}html.theme--documenter-dark .content figure figcaption{font-style:italic}html.theme--documenter-dark .content pre{-webkit-overflow-scrolling:touch;overflow-x:auto;padding:0;white-space:pre;word-wrap:normal}html.theme--documenter-dark .content sup,html.theme--documenter-dark .content sub{font-size:75%}html.theme--documenter-dark .content table{width:100%}html.theme--documenter-dark .content table td,html.theme--documenter-dark .content table th{border:1px solid #5e6d6f;border-width:0 0 1px;padding:0.5em 0.75em;vertical-align:top}html.theme--documenter-dark .content table th{color:#f2f2f2}html.theme--documenter-dark .content table th:not([align]){text-align:inherit}html.theme--documenter-dark .content table thead td,html.theme--documenter-dark .content table thead th{border-width:0 0 2px;color:#f2f2f2}html.theme--documenter-dark .content table tfoot td,html.theme--documenter-dark .content table tfoot th{border-width:2px 0 0;color:#f2f2f2}html.theme--documenter-dark .content table tbody tr:last-child td,html.theme--documenter-dark .content table tbody tr:last-child th{border-bottom-width:0}html.theme--documenter-dark .content .tabs li+li{margin-top:0}html.theme--documenter-dark .content.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.content{font-size:.75rem}html.theme--documenter-dark .content.is-normal{font-size:1rem}html.theme--documenter-dark .content.is-medium{font-size:1.25rem}html.theme--documenter-dark .content.is-large{font-size:1.5rem}html.theme--documenter-dark .icon{align-items:center;display:inline-flex;justify-content:center;height:1.5rem;width:1.5rem}html.theme--documenter-dark .icon.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.icon{height:1rem;width:1rem}html.theme--documenter-dark .icon.is-medium{height:2rem;width:2rem}html.theme--documenter-dark .icon.is-large{height:3rem;width:3rem}html.theme--documenter-dark .icon-text{align-items:flex-start;color:inherit;display:inline-flex;flex-wrap:wrap;line-height:1.5rem;vertical-align:top}html.theme--documenter-dark .icon-text .icon{flex-grow:0;flex-shrink:0}html.theme--documenter-dark .icon-text .icon:not(:last-child){margin-right:.25em}html.theme--documenter-dark .icon-text .icon:not(:first-child){margin-left:.25em}html.theme--documenter-dark div.icon-text{display:flex}html.theme--documenter-dark .image,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img{display:block;position:relative}html.theme--documenter-dark .image img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img img{display:block;height:auto;width:100%}html.theme--documenter-dark .image img.is-rounded,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img img.is-rounded{border-radius:9999px}html.theme--documenter-dark .image.is-fullwidth,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-fullwidth{width:100%}html.theme--documenter-dark .image.is-square img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-square img,html.theme--documenter-dark .image.is-square .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-square .has-ratio,html.theme--documenter-dark .image.is-1by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by1 img,html.theme--documenter-dark .image.is-1by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by1 .has-ratio,html.theme--documenter-dark .image.is-5by4 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by4 img,html.theme--documenter-dark .image.is-5by4 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by4 .has-ratio,html.theme--documenter-dark .image.is-4by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by3 img,html.theme--documenter-dark .image.is-4by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by3 .has-ratio,html.theme--documenter-dark .image.is-3by2 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by2 img,html.theme--documenter-dark .image.is-3by2 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by2 .has-ratio,html.theme--documenter-dark .image.is-5by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by3 img,html.theme--documenter-dark .image.is-5by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by3 .has-ratio,html.theme--documenter-dark .image.is-16by9 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16by9 img,html.theme--documenter-dark .image.is-16by9 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16by9 .has-ratio,html.theme--documenter-dark .image.is-2by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by1 img,html.theme--documenter-dark .image.is-2by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by1 .has-ratio,html.theme--documenter-dark .image.is-3by1 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by1 img,html.theme--documenter-dark .image.is-3by1 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by1 .has-ratio,html.theme--documenter-dark .image.is-4by5 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by5 img,html.theme--documenter-dark .image.is-4by5 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by5 .has-ratio,html.theme--documenter-dark .image.is-3by4 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by4 img,html.theme--documenter-dark .image.is-3by4 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by4 .has-ratio,html.theme--documenter-dark .image.is-2by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by3 img,html.theme--documenter-dark .image.is-2by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by3 .has-ratio,html.theme--documenter-dark .image.is-3by5 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by5 img,html.theme--documenter-dark .image.is-3by5 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by5 .has-ratio,html.theme--documenter-dark .image.is-9by16 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-9by16 img,html.theme--documenter-dark .image.is-9by16 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-9by16 .has-ratio,html.theme--documenter-dark .image.is-1by2 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by2 img,html.theme--documenter-dark .image.is-1by2 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by2 .has-ratio,html.theme--documenter-dark .image.is-1by3 img,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by3 img,html.theme--documenter-dark .image.is-1by3 .has-ratio,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by3 .has-ratio{height:100%;width:100%}html.theme--documenter-dark .image.is-square,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-square,html.theme--documenter-dark .image.is-1by1,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by1{padding-top:100%}html.theme--documenter-dark .image.is-5by4,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by4{padding-top:80%}html.theme--documenter-dark .image.is-4by3,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by3{padding-top:75%}html.theme--documenter-dark .image.is-3by2,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by2{padding-top:66.6666%}html.theme--documenter-dark .image.is-5by3,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-5by3{padding-top:60%}html.theme--documenter-dark .image.is-16by9,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16by9{padding-top:56.25%}html.theme--documenter-dark .image.is-2by1,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by1{padding-top:50%}html.theme--documenter-dark .image.is-3by1,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by1{padding-top:33.3333%}html.theme--documenter-dark .image.is-4by5,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-4by5{padding-top:125%}html.theme--documenter-dark .image.is-3by4,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by4{padding-top:133.3333%}html.theme--documenter-dark .image.is-2by3,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-2by3{padding-top:150%}html.theme--documenter-dark .image.is-3by5,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-3by5{padding-top:166.6666%}html.theme--documenter-dark .image.is-9by16,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-9by16{padding-top:177.7777%}html.theme--documenter-dark .image.is-1by2,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by2{padding-top:200%}html.theme--documenter-dark .image.is-1by3,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-1by3{padding-top:300%}html.theme--documenter-dark .image.is-16x16,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-16x16{height:16px;width:16px}html.theme--documenter-dark .image.is-24x24,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-24x24{height:24px;width:24px}html.theme--documenter-dark .image.is-32x32,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-32x32{height:32px;width:32px}html.theme--documenter-dark .image.is-48x48,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-48x48{height:48px;width:48px}html.theme--documenter-dark .image.is-64x64,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-64x64{height:64px;width:64px}html.theme--documenter-dark .image.is-96x96,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-96x96{height:96px;width:96px}html.theme--documenter-dark .image.is-128x128,html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img.is-128x128{height:128px;width:128px}html.theme--documenter-dark .notification{background-color:#282f2f;border-radius:.4em;position:relative;padding:1.25rem 2.5rem 1.25rem 1.5rem}html.theme--documenter-dark .notification a:not(.button):not(.dropdown-item){color:currentColor;text-decoration:underline}html.theme--documenter-dark .notification strong{color:currentColor}html.theme--documenter-dark .notification code,html.theme--documenter-dark .notification pre{background:#fff}html.theme--documenter-dark .notification pre code{background:transparent}html.theme--documenter-dark .notification>.delete{right:.5rem;position:absolute;top:0.5rem}html.theme--documenter-dark .notification .title,html.theme--documenter-dark .notification .subtitle,html.theme--documenter-dark .notification .content{color:currentColor}html.theme--documenter-dark .notification.is-white{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .notification.is-black{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .notification.is-light{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .notification.is-dark,html.theme--documenter-dark .content kbd.notification{background-color:#282f2f;color:#fff}html.theme--documenter-dark .notification.is-primary,html.theme--documenter-dark .docstring>section>a.notification.docs-sourcelink{background-color:#375a7f;color:#fff}html.theme--documenter-dark .notification.is-primary.is-light,html.theme--documenter-dark .docstring>section>a.notification.is-light.docs-sourcelink{background-color:#f1f5f9;color:#4d7eb2}html.theme--documenter-dark .notification.is-link{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .notification.is-link.is-light{background-color:#edfdf9;color:#15987e}html.theme--documenter-dark .notification.is-info{background-color:#024c7d;color:#fff}html.theme--documenter-dark .notification.is-info.is-light{background-color:#ebf7ff;color:#0e9dfb}html.theme--documenter-dark .notification.is-success{background-color:#008438;color:#fff}html.theme--documenter-dark .notification.is-success.is-light{background-color:#ebfff3;color:#00eb64}html.theme--documenter-dark .notification.is-warning{background-color:#ad8100;color:#fff}html.theme--documenter-dark .notification.is-warning.is-light{background-color:#fffaeb;color:#d19c00}html.theme--documenter-dark .notification.is-danger{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .notification.is-danger.is-light{background-color:#fdeeec;color:#ec311d}html.theme--documenter-dark .progress{-moz-appearance:none;-webkit-appearance:none;border:none;border-radius:9999px;display:block;height:1rem;overflow:hidden;padding:0;width:100%}html.theme--documenter-dark .progress::-webkit-progress-bar{background-color:#343c3d}html.theme--documenter-dark .progress::-webkit-progress-value{background-color:#dbdee0}html.theme--documenter-dark .progress::-moz-progress-bar{background-color:#dbdee0}html.theme--documenter-dark .progress::-ms-fill{background-color:#dbdee0;border:none}html.theme--documenter-dark .progress.is-white::-webkit-progress-value{background-color:#fff}html.theme--documenter-dark .progress.is-white::-moz-progress-bar{background-color:#fff}html.theme--documenter-dark .progress.is-white::-ms-fill{background-color:#fff}html.theme--documenter-dark .progress.is-white:indeterminate{background-image:linear-gradient(to right, #fff 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-black::-webkit-progress-value{background-color:#0a0a0a}html.theme--documenter-dark .progress.is-black::-moz-progress-bar{background-color:#0a0a0a}html.theme--documenter-dark .progress.is-black::-ms-fill{background-color:#0a0a0a}html.theme--documenter-dark .progress.is-black:indeterminate{background-image:linear-gradient(to right, #0a0a0a 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-light::-webkit-progress-value{background-color:#ecf0f1}html.theme--documenter-dark .progress.is-light::-moz-progress-bar{background-color:#ecf0f1}html.theme--documenter-dark .progress.is-light::-ms-fill{background-color:#ecf0f1}html.theme--documenter-dark .progress.is-light:indeterminate{background-image:linear-gradient(to right, #ecf0f1 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-dark::-webkit-progress-value,html.theme--documenter-dark .content kbd.progress::-webkit-progress-value{background-color:#282f2f}html.theme--documenter-dark .progress.is-dark::-moz-progress-bar,html.theme--documenter-dark .content kbd.progress::-moz-progress-bar{background-color:#282f2f}html.theme--documenter-dark .progress.is-dark::-ms-fill,html.theme--documenter-dark .content kbd.progress::-ms-fill{background-color:#282f2f}html.theme--documenter-dark .progress.is-dark:indeterminate,html.theme--documenter-dark .content kbd.progress:indeterminate{background-image:linear-gradient(to right, #282f2f 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-primary::-webkit-progress-value,html.theme--documenter-dark .docstring>section>a.progress.docs-sourcelink::-webkit-progress-value{background-color:#375a7f}html.theme--documenter-dark .progress.is-primary::-moz-progress-bar,html.theme--documenter-dark .docstring>section>a.progress.docs-sourcelink::-moz-progress-bar{background-color:#375a7f}html.theme--documenter-dark .progress.is-primary::-ms-fill,html.theme--documenter-dark .docstring>section>a.progress.docs-sourcelink::-ms-fill{background-color:#375a7f}html.theme--documenter-dark .progress.is-primary:indeterminate,html.theme--documenter-dark .docstring>section>a.progress.docs-sourcelink:indeterminate{background-image:linear-gradient(to right, #375a7f 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-link::-webkit-progress-value{background-color:#1abc9c}html.theme--documenter-dark .progress.is-link::-moz-progress-bar{background-color:#1abc9c}html.theme--documenter-dark .progress.is-link::-ms-fill{background-color:#1abc9c}html.theme--documenter-dark .progress.is-link:indeterminate{background-image:linear-gradient(to right, #1abc9c 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-info::-webkit-progress-value{background-color:#024c7d}html.theme--documenter-dark .progress.is-info::-moz-progress-bar{background-color:#024c7d}html.theme--documenter-dark .progress.is-info::-ms-fill{background-color:#024c7d}html.theme--documenter-dark .progress.is-info:indeterminate{background-image:linear-gradient(to right, #024c7d 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-success::-webkit-progress-value{background-color:#008438}html.theme--documenter-dark .progress.is-success::-moz-progress-bar{background-color:#008438}html.theme--documenter-dark .progress.is-success::-ms-fill{background-color:#008438}html.theme--documenter-dark .progress.is-success:indeterminate{background-image:linear-gradient(to right, #008438 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-warning::-webkit-progress-value{background-color:#ad8100}html.theme--documenter-dark .progress.is-warning::-moz-progress-bar{background-color:#ad8100}html.theme--documenter-dark .progress.is-warning::-ms-fill{background-color:#ad8100}html.theme--documenter-dark .progress.is-warning:indeterminate{background-image:linear-gradient(to right, #ad8100 30%, #343c3d 30%)}html.theme--documenter-dark .progress.is-danger::-webkit-progress-value{background-color:#9e1b0d}html.theme--documenter-dark .progress.is-danger::-moz-progress-bar{background-color:#9e1b0d}html.theme--documenter-dark .progress.is-danger::-ms-fill{background-color:#9e1b0d}html.theme--documenter-dark .progress.is-danger:indeterminate{background-image:linear-gradient(to right, #9e1b0d 30%, #343c3d 30%)}html.theme--documenter-dark .progress:indeterminate{animation-duration:1.5s;animation-iteration-count:infinite;animation-name:moveIndeterminate;animation-timing-function:linear;background-color:#343c3d;background-image:linear-gradient(to right, #fff 30%, #343c3d 30%);background-position:top left;background-repeat:no-repeat;background-size:150% 150%}html.theme--documenter-dark .progress:indeterminate::-webkit-progress-bar{background-color:transparent}html.theme--documenter-dark .progress:indeterminate::-moz-progress-bar{background-color:transparent}html.theme--documenter-dark .progress:indeterminate::-ms-fill{animation-name:none}html.theme--documenter-dark .progress.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.progress{height:.75rem}html.theme--documenter-dark .progress.is-medium{height:1.25rem}html.theme--documenter-dark .progress.is-large{height:1.5rem}@keyframes moveIndeterminate{from{background-position:200% 0}to{background-position:-200% 0}}html.theme--documenter-dark .table{background-color:#343c3d;color:#fff}html.theme--documenter-dark .table td,html.theme--documenter-dark .table th{border:1px solid #5e6d6f;border-width:0 0 1px;padding:0.5em 0.75em;vertical-align:top}html.theme--documenter-dark .table td.is-white,html.theme--documenter-dark .table th.is-white{background-color:#fff;border-color:#fff;color:#0a0a0a}html.theme--documenter-dark .table td.is-black,html.theme--documenter-dark .table th.is-black{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}html.theme--documenter-dark .table td.is-light,html.theme--documenter-dark .table th.is-light{background-color:#ecf0f1;border-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .table td.is-dark,html.theme--documenter-dark .table th.is-dark{background-color:#282f2f;border-color:#282f2f;color:#fff}html.theme--documenter-dark .table td.is-primary,html.theme--documenter-dark .table th.is-primary{background-color:#375a7f;border-color:#375a7f;color:#fff}html.theme--documenter-dark .table td.is-link,html.theme--documenter-dark .table th.is-link{background-color:#1abc9c;border-color:#1abc9c;color:#fff}html.theme--documenter-dark .table td.is-info,html.theme--documenter-dark .table th.is-info{background-color:#024c7d;border-color:#024c7d;color:#fff}html.theme--documenter-dark .table td.is-success,html.theme--documenter-dark .table th.is-success{background-color:#008438;border-color:#008438;color:#fff}html.theme--documenter-dark .table td.is-warning,html.theme--documenter-dark .table th.is-warning{background-color:#ad8100;border-color:#ad8100;color:#fff}html.theme--documenter-dark .table td.is-danger,html.theme--documenter-dark .table th.is-danger{background-color:#9e1b0d;border-color:#9e1b0d;color:#fff}html.theme--documenter-dark .table td.is-narrow,html.theme--documenter-dark .table th.is-narrow{white-space:nowrap;width:1%}html.theme--documenter-dark .table td.is-selected,html.theme--documenter-dark .table th.is-selected{background-color:#375a7f;color:#fff}html.theme--documenter-dark .table td.is-selected a,html.theme--documenter-dark .table td.is-selected strong,html.theme--documenter-dark .table th.is-selected a,html.theme--documenter-dark .table th.is-selected strong{color:currentColor}html.theme--documenter-dark .table td.is-vcentered,html.theme--documenter-dark .table th.is-vcentered{vertical-align:middle}html.theme--documenter-dark .table th{color:#f2f2f2}html.theme--documenter-dark .table th:not([align]){text-align:left}html.theme--documenter-dark .table tr.is-selected{background-color:#375a7f;color:#fff}html.theme--documenter-dark .table tr.is-selected a,html.theme--documenter-dark .table tr.is-selected strong{color:currentColor}html.theme--documenter-dark .table tr.is-selected td,html.theme--documenter-dark .table tr.is-selected th{border-color:#fff;color:currentColor}html.theme--documenter-dark .table thead{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .table thead td,html.theme--documenter-dark .table thead th{border-width:0 0 2px;color:#f2f2f2}html.theme--documenter-dark .table tfoot{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .table tfoot td,html.theme--documenter-dark .table tfoot th{border-width:2px 0 0;color:#f2f2f2}html.theme--documenter-dark .table tbody{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .table tbody tr:last-child td,html.theme--documenter-dark .table tbody tr:last-child th{border-bottom-width:0}html.theme--documenter-dark .table.is-bordered td,html.theme--documenter-dark .table.is-bordered th{border-width:1px}html.theme--documenter-dark .table.is-bordered tr:last-child td,html.theme--documenter-dark .table.is-bordered tr:last-child th{border-bottom-width:1px}html.theme--documenter-dark .table.is-fullwidth{width:100%}html.theme--documenter-dark .table.is-hoverable tbody tr:not(.is-selected):hover{background-color:#282f2f}html.theme--documenter-dark .table.is-hoverable.is-striped tbody tr:not(.is-selected):hover{background-color:#282f2f}html.theme--documenter-dark .table.is-hoverable.is-striped tbody tr:not(.is-selected):hover:nth-child(even){background-color:#2d3435}html.theme--documenter-dark .table.is-narrow td,html.theme--documenter-dark .table.is-narrow th{padding:0.25em 0.5em}html.theme--documenter-dark .table.is-striped tbody tr:not(.is-selected):nth-child(even){background-color:#282f2f}html.theme--documenter-dark .table-container{-webkit-overflow-scrolling:touch;overflow:auto;overflow-y:hidden;max-width:100%}html.theme--documenter-dark .tags{align-items:center;display:flex;flex-wrap:wrap;justify-content:flex-start}html.theme--documenter-dark .tags .tag,html.theme--documenter-dark .tags .content kbd,html.theme--documenter-dark .content .tags kbd,html.theme--documenter-dark .tags .docstring>section>a.docs-sourcelink{margin-bottom:0.5rem}html.theme--documenter-dark .tags .tag:not(:last-child),html.theme--documenter-dark .tags .content kbd:not(:last-child),html.theme--documenter-dark .content .tags kbd:not(:last-child),html.theme--documenter-dark .tags .docstring>section>a.docs-sourcelink:not(:last-child){margin-right:.5rem}html.theme--documenter-dark .tags:last-child{margin-bottom:-0.5rem}html.theme--documenter-dark .tags:not(:last-child){margin-bottom:1rem}html.theme--documenter-dark .tags.are-medium .tag:not(.is-normal):not(.is-large),html.theme--documenter-dark .tags.are-medium .content kbd:not(.is-normal):not(.is-large),html.theme--documenter-dark .content .tags.are-medium kbd:not(.is-normal):not(.is-large),html.theme--documenter-dark .tags.are-medium .docstring>section>a.docs-sourcelink:not(.is-normal):not(.is-large){font-size:1rem}html.theme--documenter-dark .tags.are-large .tag:not(.is-normal):not(.is-medium),html.theme--documenter-dark .tags.are-large .content kbd:not(.is-normal):not(.is-medium),html.theme--documenter-dark .content .tags.are-large kbd:not(.is-normal):not(.is-medium),html.theme--documenter-dark .tags.are-large .docstring>section>a.docs-sourcelink:not(.is-normal):not(.is-medium){font-size:1.25rem}html.theme--documenter-dark .tags.is-centered{justify-content:center}html.theme--documenter-dark .tags.is-centered .tag,html.theme--documenter-dark .tags.is-centered .content kbd,html.theme--documenter-dark .content .tags.is-centered kbd,html.theme--documenter-dark .tags.is-centered .docstring>section>a.docs-sourcelink{margin-right:0.25rem;margin-left:0.25rem}html.theme--documenter-dark .tags.is-right{justify-content:flex-end}html.theme--documenter-dark .tags.is-right .tag:not(:first-child),html.theme--documenter-dark .tags.is-right .content kbd:not(:first-child),html.theme--documenter-dark .content .tags.is-right kbd:not(:first-child),html.theme--documenter-dark .tags.is-right .docstring>section>a.docs-sourcelink:not(:first-child){margin-left:0.5rem}html.theme--documenter-dark .tags.is-right .tag:not(:last-child),html.theme--documenter-dark .tags.is-right .content kbd:not(:last-child),html.theme--documenter-dark .content .tags.is-right kbd:not(:last-child),html.theme--documenter-dark .tags.is-right .docstring>section>a.docs-sourcelink:not(:last-child){margin-right:0}html.theme--documenter-dark .tags.has-addons .tag,html.theme--documenter-dark .tags.has-addons .content kbd,html.theme--documenter-dark .content .tags.has-addons kbd,html.theme--documenter-dark .tags.has-addons .docstring>section>a.docs-sourcelink{margin-right:0}html.theme--documenter-dark .tags.has-addons .tag:not(:first-child),html.theme--documenter-dark .tags.has-addons .content kbd:not(:first-child),html.theme--documenter-dark .content .tags.has-addons kbd:not(:first-child),html.theme--documenter-dark .tags.has-addons .docstring>section>a.docs-sourcelink:not(:first-child){margin-left:0;border-top-left-radius:0;border-bottom-left-radius:0}html.theme--documenter-dark .tags.has-addons .tag:not(:last-child),html.theme--documenter-dark .tags.has-addons .content kbd:not(:last-child),html.theme--documenter-dark .content .tags.has-addons kbd:not(:last-child),html.theme--documenter-dark .tags.has-addons .docstring>section>a.docs-sourcelink:not(:last-child){border-top-right-radius:0;border-bottom-right-radius:0}html.theme--documenter-dark .tag:not(body),html.theme--documenter-dark .content kbd:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body){align-items:center;background-color:#282f2f;border-radius:.4em;color:#fff;display:inline-flex;font-size:.75rem;height:2em;justify-content:center;line-height:1.5;padding-left:0.75em;padding-right:0.75em;white-space:nowrap}html.theme--documenter-dark .tag:not(body) .delete,html.theme--documenter-dark .content kbd:not(body) .delete,html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body) .delete{margin-left:.25rem;margin-right:-.375rem}html.theme--documenter-dark .tag.is-white:not(body),html.theme--documenter-dark .content kbd.is-white:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-white:not(body){background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .tag.is-black:not(body),html.theme--documenter-dark .content kbd.is-black:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-black:not(body){background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .tag.is-light:not(body),html.theme--documenter-dark .content kbd.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-light:not(body){background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .tag.is-dark:not(body),html.theme--documenter-dark .content kbd:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-dark:not(body),html.theme--documenter-dark .content .docstring>section>kbd:not(body){background-color:#282f2f;color:#fff}html.theme--documenter-dark .tag.is-primary:not(body),html.theme--documenter-dark .content kbd.is-primary:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body){background-color:#375a7f;color:#fff}html.theme--documenter-dark .tag.is-primary.is-light:not(body),html.theme--documenter-dark .content kbd.is-primary.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-light:not(body){background-color:#f1f5f9;color:#4d7eb2}html.theme--documenter-dark .tag.is-link:not(body),html.theme--documenter-dark .content kbd.is-link:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-link:not(body){background-color:#1abc9c;color:#fff}html.theme--documenter-dark .tag.is-link.is-light:not(body),html.theme--documenter-dark .content kbd.is-link.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-link.is-light:not(body){background-color:#edfdf9;color:#15987e}html.theme--documenter-dark .tag.is-info:not(body),html.theme--documenter-dark .content kbd.is-info:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-info:not(body){background-color:#024c7d;color:#fff}html.theme--documenter-dark .tag.is-info.is-light:not(body),html.theme--documenter-dark .content kbd.is-info.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-info.is-light:not(body){background-color:#ebf7ff;color:#0e9dfb}html.theme--documenter-dark .tag.is-success:not(body),html.theme--documenter-dark .content kbd.is-success:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-success:not(body){background-color:#008438;color:#fff}html.theme--documenter-dark .tag.is-success.is-light:not(body),html.theme--documenter-dark .content kbd.is-success.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-success.is-light:not(body){background-color:#ebfff3;color:#00eb64}html.theme--documenter-dark .tag.is-warning:not(body),html.theme--documenter-dark .content kbd.is-warning:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-warning:not(body){background-color:#ad8100;color:#fff}html.theme--documenter-dark .tag.is-warning.is-light:not(body),html.theme--documenter-dark .content kbd.is-warning.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-warning.is-light:not(body){background-color:#fffaeb;color:#d19c00}html.theme--documenter-dark .tag.is-danger:not(body),html.theme--documenter-dark .content kbd.is-danger:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-danger:not(body){background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .tag.is-danger.is-light:not(body),html.theme--documenter-dark .content kbd.is-danger.is-light:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-danger.is-light:not(body){background-color:#fdeeec;color:#ec311d}html.theme--documenter-dark .tag.is-normal:not(body),html.theme--documenter-dark .content kbd.is-normal:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-normal:not(body){font-size:.75rem}html.theme--documenter-dark .tag.is-medium:not(body),html.theme--documenter-dark .content kbd.is-medium:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-medium:not(body){font-size:1rem}html.theme--documenter-dark .tag.is-large:not(body),html.theme--documenter-dark .content kbd.is-large:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-large:not(body){font-size:1.25rem}html.theme--documenter-dark .tag:not(body) .icon:first-child:not(:last-child),html.theme--documenter-dark .content kbd:not(body) .icon:first-child:not(:last-child),html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body) .icon:first-child:not(:last-child){margin-left:-.375em;margin-right:.1875em}html.theme--documenter-dark .tag:not(body) .icon:last-child:not(:first-child),html.theme--documenter-dark .content kbd:not(body) .icon:last-child:not(:first-child),html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body) .icon:last-child:not(:first-child){margin-left:.1875em;margin-right:-.375em}html.theme--documenter-dark .tag:not(body) .icon:first-child:last-child,html.theme--documenter-dark .content kbd:not(body) .icon:first-child:last-child,html.theme--documenter-dark .docstring>section>a.docs-sourcelink:not(body) .icon:first-child:last-child{margin-left:-.375em;margin-right:-.375em}html.theme--documenter-dark .tag.is-delete:not(body),html.theme--documenter-dark .content kbd.is-delete:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body){margin-left:1px;padding:0;position:relative;width:2em}html.theme--documenter-dark .tag.is-delete:not(body)::before,html.theme--documenter-dark .content kbd.is-delete:not(body)::before,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body)::before,html.theme--documenter-dark .tag.is-delete:not(body)::after,html.theme--documenter-dark .content kbd.is-delete:not(body)::after,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body)::after{background-color:currentColor;content:"";display:block;left:50%;position:absolute;top:50%;transform:translateX(-50%) translateY(-50%) rotate(45deg);transform-origin:center center}html.theme--documenter-dark .tag.is-delete:not(body)::before,html.theme--documenter-dark .content kbd.is-delete:not(body)::before,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body)::before{height:1px;width:50%}html.theme--documenter-dark .tag.is-delete:not(body)::after,html.theme--documenter-dark .content kbd.is-delete:not(body)::after,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body)::after{height:50%;width:1px}html.theme--documenter-dark .tag.is-delete:not(body):hover,html.theme--documenter-dark .content kbd.is-delete:not(body):hover,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body):hover,html.theme--documenter-dark .tag.is-delete:not(body):focus,html.theme--documenter-dark .content kbd.is-delete:not(body):focus,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body):focus{background-color:#1d2122}html.theme--documenter-dark .tag.is-delete:not(body):active,html.theme--documenter-dark .content kbd.is-delete:not(body):active,html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-delete:not(body):active{background-color:#111414}html.theme--documenter-dark .tag.is-rounded:not(body),html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:not(body),html.theme--documenter-dark .content kbd.is-rounded:not(body),html.theme--documenter-dark #documenter .docs-sidebar .content form.docs-search>input:not(body),html.theme--documenter-dark .docstring>section>a.docs-sourcelink.is-rounded:not(body){border-radius:9999px}html.theme--documenter-dark a.tag:hover,html.theme--documenter-dark .docstring>section>a.docs-sourcelink:hover{text-decoration:underline}html.theme--documenter-dark .title,html.theme--documenter-dark .subtitle{word-break:break-word}html.theme--documenter-dark .title em,html.theme--documenter-dark .title span,html.theme--documenter-dark .subtitle em,html.theme--documenter-dark .subtitle span{font-weight:inherit}html.theme--documenter-dark .title sub,html.theme--documenter-dark .subtitle sub{font-size:.75em}html.theme--documenter-dark .title sup,html.theme--documenter-dark .subtitle sup{font-size:.75em}html.theme--documenter-dark .title .tag,html.theme--documenter-dark .title .content kbd,html.theme--documenter-dark .content .title kbd,html.theme--documenter-dark .title .docstring>section>a.docs-sourcelink,html.theme--documenter-dark .subtitle .tag,html.theme--documenter-dark .subtitle .content kbd,html.theme--documenter-dark .content .subtitle kbd,html.theme--documenter-dark .subtitle .docstring>section>a.docs-sourcelink{vertical-align:middle}html.theme--documenter-dark .title{color:#fff;font-size:2rem;font-weight:500;line-height:1.125}html.theme--documenter-dark .title strong{color:inherit;font-weight:inherit}html.theme--documenter-dark .title:not(.is-spaced)+.subtitle{margin-top:-1.25rem}html.theme--documenter-dark .title.is-1{font-size:3rem}html.theme--documenter-dark .title.is-2{font-size:2.5rem}html.theme--documenter-dark .title.is-3{font-size:2rem}html.theme--documenter-dark .title.is-4{font-size:1.5rem}html.theme--documenter-dark .title.is-5{font-size:1.25rem}html.theme--documenter-dark .title.is-6{font-size:1rem}html.theme--documenter-dark .title.is-7{font-size:.75rem}html.theme--documenter-dark .subtitle{color:#8c9b9d;font-size:1.25rem;font-weight:400;line-height:1.25}html.theme--documenter-dark .subtitle strong{color:#8c9b9d;font-weight:600}html.theme--documenter-dark .subtitle:not(.is-spaced)+.title{margin-top:-1.25rem}html.theme--documenter-dark .subtitle.is-1{font-size:3rem}html.theme--documenter-dark .subtitle.is-2{font-size:2.5rem}html.theme--documenter-dark .subtitle.is-3{font-size:2rem}html.theme--documenter-dark .subtitle.is-4{font-size:1.5rem}html.theme--documenter-dark .subtitle.is-5{font-size:1.25rem}html.theme--documenter-dark .subtitle.is-6{font-size:1rem}html.theme--documenter-dark .subtitle.is-7{font-size:.75rem}html.theme--documenter-dark .heading{display:block;font-size:11px;letter-spacing:1px;margin-bottom:5px;text-transform:uppercase}html.theme--documenter-dark .number{align-items:center;background-color:#282f2f;border-radius:9999px;display:inline-flex;font-size:1.25rem;height:2em;justify-content:center;margin-right:1.5rem;min-width:2.5em;padding:0.25rem 0.5rem;text-align:center;vertical-align:top}html.theme--documenter-dark .select select,html.theme--documenter-dark .textarea,html.theme--documenter-dark .input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{background-color:#1f2424;border-color:#5e6d6f;border-radius:.4em;color:#dbdee0}html.theme--documenter-dark .select select::-moz-placeholder,html.theme--documenter-dark .textarea::-moz-placeholder,html.theme--documenter-dark .input::-moz-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input::-moz-placeholder{color:#868c98}html.theme--documenter-dark .select select::-webkit-input-placeholder,html.theme--documenter-dark .textarea::-webkit-input-placeholder,html.theme--documenter-dark .input::-webkit-input-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input::-webkit-input-placeholder{color:#868c98}html.theme--documenter-dark .select select:-moz-placeholder,html.theme--documenter-dark .textarea:-moz-placeholder,html.theme--documenter-dark .input:-moz-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:-moz-placeholder{color:#868c98}html.theme--documenter-dark .select select:-ms-input-placeholder,html.theme--documenter-dark .textarea:-ms-input-placeholder,html.theme--documenter-dark .input:-ms-input-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:-ms-input-placeholder{color:#868c98}html.theme--documenter-dark .select select:hover,html.theme--documenter-dark .textarea:hover,html.theme--documenter-dark .input:hover,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:hover,html.theme--documenter-dark .select select.is-hovered,html.theme--documenter-dark .is-hovered.textarea,html.theme--documenter-dark .is-hovered.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-hovered{border-color:#8c9b9d}html.theme--documenter-dark .select select:focus,html.theme--documenter-dark .textarea:focus,html.theme--documenter-dark .input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:focus,html.theme--documenter-dark .select select.is-focused,html.theme--documenter-dark .is-focused.textarea,html.theme--documenter-dark .is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .select select:active,html.theme--documenter-dark .textarea:active,html.theme--documenter-dark .input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:active,html.theme--documenter-dark .select select.is-active,html.theme--documenter-dark .is-active.textarea,html.theme--documenter-dark .is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{border-color:#1abc9c;box-shadow:0 0 0 0.125em rgba(26,188,156,0.25)}html.theme--documenter-dark .select select[disabled],html.theme--documenter-dark .textarea[disabled],html.theme--documenter-dark .input[disabled],html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled],fieldset[disabled] html.theme--documenter-dark .select select,fieldset[disabled] html.theme--documenter-dark .textarea,fieldset[disabled] html.theme--documenter-dark .input,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{background-color:#8c9b9d;border-color:#282f2f;box-shadow:none;color:#fff}html.theme--documenter-dark .select select[disabled]::-moz-placeholder,html.theme--documenter-dark .textarea[disabled]::-moz-placeholder,html.theme--documenter-dark .input[disabled]::-moz-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled]::-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .select select::-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .textarea::-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .input::-moz-placeholder,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input::-moz-placeholder{color:rgba(255,255,255,0.3)}html.theme--documenter-dark .select select[disabled]::-webkit-input-placeholder,html.theme--documenter-dark .textarea[disabled]::-webkit-input-placeholder,html.theme--documenter-dark .input[disabled]::-webkit-input-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled]::-webkit-input-placeholder,fieldset[disabled] html.theme--documenter-dark .select select::-webkit-input-placeholder,fieldset[disabled] html.theme--documenter-dark .textarea::-webkit-input-placeholder,fieldset[disabled] html.theme--documenter-dark .input::-webkit-input-placeholder,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input::-webkit-input-placeholder{color:rgba(255,255,255,0.3)}html.theme--documenter-dark .select select[disabled]:-moz-placeholder,html.theme--documenter-dark .textarea[disabled]:-moz-placeholder,html.theme--documenter-dark .input[disabled]:-moz-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled]:-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .select select:-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .textarea:-moz-placeholder,fieldset[disabled] html.theme--documenter-dark .input:-moz-placeholder,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:-moz-placeholder{color:rgba(255,255,255,0.3)}html.theme--documenter-dark .select select[disabled]:-ms-input-placeholder,html.theme--documenter-dark .textarea[disabled]:-ms-input-placeholder,html.theme--documenter-dark .input[disabled]:-ms-input-placeholder,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[disabled]:-ms-input-placeholder,fieldset[disabled] html.theme--documenter-dark .select select:-ms-input-placeholder,fieldset[disabled] html.theme--documenter-dark .textarea:-ms-input-placeholder,fieldset[disabled] html.theme--documenter-dark .input:-ms-input-placeholder,fieldset[disabled] html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input:-ms-input-placeholder{color:rgba(255,255,255,0.3)}html.theme--documenter-dark .textarea,html.theme--documenter-dark .input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{box-shadow:inset 0 0.0625em 0.125em rgba(10,10,10,0.05);max-width:100%;width:100%}html.theme--documenter-dark .textarea[readonly],html.theme--documenter-dark .input[readonly],html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input[readonly]{box-shadow:none}html.theme--documenter-dark .is-white.textarea,html.theme--documenter-dark .is-white.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-white{border-color:#fff}html.theme--documenter-dark .is-white.textarea:focus,html.theme--documenter-dark .is-white.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-white:focus,html.theme--documenter-dark .is-white.is-focused.textarea,html.theme--documenter-dark .is-white.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-white.textarea:active,html.theme--documenter-dark .is-white.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-white:active,html.theme--documenter-dark .is-white.is-active.textarea,html.theme--documenter-dark .is-white.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}html.theme--documenter-dark .is-black.textarea,html.theme--documenter-dark .is-black.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-black{border-color:#0a0a0a}html.theme--documenter-dark .is-black.textarea:focus,html.theme--documenter-dark .is-black.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-black:focus,html.theme--documenter-dark .is-black.is-focused.textarea,html.theme--documenter-dark .is-black.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-black.textarea:active,html.theme--documenter-dark .is-black.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-black:active,html.theme--documenter-dark .is-black.is-active.textarea,html.theme--documenter-dark .is-black.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}html.theme--documenter-dark .is-light.textarea,html.theme--documenter-dark .is-light.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-light{border-color:#ecf0f1}html.theme--documenter-dark .is-light.textarea:focus,html.theme--documenter-dark .is-light.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-light:focus,html.theme--documenter-dark .is-light.is-focused.textarea,html.theme--documenter-dark .is-light.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-light.textarea:active,html.theme--documenter-dark .is-light.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-light:active,html.theme--documenter-dark .is-light.is-active.textarea,html.theme--documenter-dark .is-light.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(236,240,241,0.25)}html.theme--documenter-dark .is-dark.textarea,html.theme--documenter-dark .content kbd.textarea,html.theme--documenter-dark .is-dark.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-dark,html.theme--documenter-dark .content kbd.input{border-color:#282f2f}html.theme--documenter-dark .is-dark.textarea:focus,html.theme--documenter-dark .content kbd.textarea:focus,html.theme--documenter-dark .is-dark.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-dark:focus,html.theme--documenter-dark .content kbd.input:focus,html.theme--documenter-dark .is-dark.is-focused.textarea,html.theme--documenter-dark .content kbd.is-focused.textarea,html.theme--documenter-dark .is-dark.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .content kbd.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar .content form.docs-search>input.is-focused,html.theme--documenter-dark .is-dark.textarea:active,html.theme--documenter-dark .content kbd.textarea:active,html.theme--documenter-dark .is-dark.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-dark:active,html.theme--documenter-dark .content kbd.input:active,html.theme--documenter-dark .is-dark.is-active.textarea,html.theme--documenter-dark .content kbd.is-active.textarea,html.theme--documenter-dark .is-dark.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active,html.theme--documenter-dark .content kbd.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar .content form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(40,47,47,0.25)}html.theme--documenter-dark .is-primary.textarea,html.theme--documenter-dark .docstring>section>a.textarea.docs-sourcelink,html.theme--documenter-dark .is-primary.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-primary,html.theme--documenter-dark .docstring>section>a.input.docs-sourcelink{border-color:#375a7f}html.theme--documenter-dark .is-primary.textarea:focus,html.theme--documenter-dark .docstring>section>a.textarea.docs-sourcelink:focus,html.theme--documenter-dark .is-primary.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-primary:focus,html.theme--documenter-dark .docstring>section>a.input.docs-sourcelink:focus,html.theme--documenter-dark .is-primary.is-focused.textarea,html.theme--documenter-dark .docstring>section>a.is-focused.textarea.docs-sourcelink,html.theme--documenter-dark .is-primary.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .docstring>section>a.is-focused.input.docs-sourcelink,html.theme--documenter-dark .is-primary.textarea:active,html.theme--documenter-dark .docstring>section>a.textarea.docs-sourcelink:active,html.theme--documenter-dark .is-primary.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-primary:active,html.theme--documenter-dark .docstring>section>a.input.docs-sourcelink:active,html.theme--documenter-dark .is-primary.is-active.textarea,html.theme--documenter-dark .docstring>section>a.is-active.textarea.docs-sourcelink,html.theme--documenter-dark .is-primary.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active,html.theme--documenter-dark .docstring>section>a.is-active.input.docs-sourcelink{box-shadow:0 0 0 0.125em rgba(55,90,127,0.25)}html.theme--documenter-dark .is-link.textarea,html.theme--documenter-dark .is-link.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-link{border-color:#1abc9c}html.theme--documenter-dark .is-link.textarea:focus,html.theme--documenter-dark .is-link.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-link:focus,html.theme--documenter-dark .is-link.is-focused.textarea,html.theme--documenter-dark .is-link.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-link.textarea:active,html.theme--documenter-dark .is-link.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-link:active,html.theme--documenter-dark .is-link.is-active.textarea,html.theme--documenter-dark .is-link.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(26,188,156,0.25)}html.theme--documenter-dark .is-info.textarea,html.theme--documenter-dark .is-info.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-info{border-color:#024c7d}html.theme--documenter-dark .is-info.textarea:focus,html.theme--documenter-dark .is-info.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-info:focus,html.theme--documenter-dark .is-info.is-focused.textarea,html.theme--documenter-dark .is-info.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-info.textarea:active,html.theme--documenter-dark .is-info.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-info:active,html.theme--documenter-dark .is-info.is-active.textarea,html.theme--documenter-dark .is-info.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(2,76,125,0.25)}html.theme--documenter-dark .is-success.textarea,html.theme--documenter-dark .is-success.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-success{border-color:#008438}html.theme--documenter-dark .is-success.textarea:focus,html.theme--documenter-dark .is-success.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-success:focus,html.theme--documenter-dark .is-success.is-focused.textarea,html.theme--documenter-dark .is-success.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-success.textarea:active,html.theme--documenter-dark .is-success.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-success:active,html.theme--documenter-dark .is-success.is-active.textarea,html.theme--documenter-dark .is-success.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(0,132,56,0.25)}html.theme--documenter-dark .is-warning.textarea,html.theme--documenter-dark .is-warning.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-warning{border-color:#ad8100}html.theme--documenter-dark .is-warning.textarea:focus,html.theme--documenter-dark .is-warning.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-warning:focus,html.theme--documenter-dark .is-warning.is-focused.textarea,html.theme--documenter-dark .is-warning.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-warning.textarea:active,html.theme--documenter-dark .is-warning.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-warning:active,html.theme--documenter-dark .is-warning.is-active.textarea,html.theme--documenter-dark .is-warning.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(173,129,0,0.25)}html.theme--documenter-dark .is-danger.textarea,html.theme--documenter-dark .is-danger.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-danger{border-color:#9e1b0d}html.theme--documenter-dark .is-danger.textarea:focus,html.theme--documenter-dark .is-danger.input:focus,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-danger:focus,html.theme--documenter-dark .is-danger.is-focused.textarea,html.theme--documenter-dark .is-danger.is-focused.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-focused,html.theme--documenter-dark .is-danger.textarea:active,html.theme--documenter-dark .is-danger.input:active,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-danger:active,html.theme--documenter-dark .is-danger.is-active.textarea,html.theme--documenter-dark .is-danger.is-active.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(158,27,13,0.25)}html.theme--documenter-dark .is-small.textarea,html.theme--documenter-dark .is-small.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{border-radius:3px;font-size:.75rem}html.theme--documenter-dark .is-medium.textarea,html.theme--documenter-dark .is-medium.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-medium{font-size:1.25rem}html.theme--documenter-dark .is-large.textarea,html.theme--documenter-dark .is-large.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-large{font-size:1.5rem}html.theme--documenter-dark .is-fullwidth.textarea,html.theme--documenter-dark .is-fullwidth.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-fullwidth{display:block;width:100%}html.theme--documenter-dark .is-inline.textarea,html.theme--documenter-dark .is-inline.input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-inline{display:inline;width:auto}html.theme--documenter-dark .input.is-rounded,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{border-radius:9999px;padding-left:calc(calc(0.75em - 1px) + 0.375em);padding-right:calc(calc(0.75em - 1px) + 0.375em)}html.theme--documenter-dark .input.is-static,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-static{background-color:transparent;border-color:transparent;box-shadow:none;padding-left:0;padding-right:0}html.theme--documenter-dark .textarea{display:block;max-width:100%;min-width:100%;padding:calc(0.75em - 1px);resize:vertical}html.theme--documenter-dark .textarea:not([rows]){max-height:40em;min-height:8em}html.theme--documenter-dark .textarea[rows]{height:initial}html.theme--documenter-dark .textarea.has-fixed-size{resize:none}html.theme--documenter-dark .radio,html.theme--documenter-dark .checkbox{cursor:pointer;display:inline-block;line-height:1.25;position:relative}html.theme--documenter-dark .radio input,html.theme--documenter-dark .checkbox input{cursor:pointer}html.theme--documenter-dark .radio:hover,html.theme--documenter-dark .checkbox:hover{color:#8c9b9d}html.theme--documenter-dark .radio[disabled],html.theme--documenter-dark .checkbox[disabled],fieldset[disabled] html.theme--documenter-dark .radio,fieldset[disabled] html.theme--documenter-dark .checkbox,html.theme--documenter-dark .radio input[disabled],html.theme--documenter-dark .checkbox input[disabled]{color:#fff;cursor:not-allowed}html.theme--documenter-dark .radio+.radio{margin-left:.5em}html.theme--documenter-dark .select{display:inline-block;max-width:100%;position:relative;vertical-align:top}html.theme--documenter-dark .select:not(.is-multiple){height:2.5em}html.theme--documenter-dark .select:not(.is-multiple):not(.is-loading)::after{border-color:#1abc9c;right:1.125em;z-index:4}html.theme--documenter-dark .select.is-rounded select,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.select select{border-radius:9999px;padding-left:1em}html.theme--documenter-dark .select select{cursor:pointer;display:block;font-size:1em;max-width:100%;outline:none}html.theme--documenter-dark .select select::-ms-expand{display:none}html.theme--documenter-dark .select select[disabled]:hover,fieldset[disabled] html.theme--documenter-dark .select select:hover{border-color:#282f2f}html.theme--documenter-dark .select select:not([multiple]){padding-right:2.5em}html.theme--documenter-dark .select select[multiple]{height:auto;padding:0}html.theme--documenter-dark .select select[multiple] option{padding:0.5em 1em}html.theme--documenter-dark .select:not(.is-multiple):not(.is-loading):hover::after{border-color:#8c9b9d}html.theme--documenter-dark .select.is-white:not(:hover)::after{border-color:#fff}html.theme--documenter-dark .select.is-white select{border-color:#fff}html.theme--documenter-dark .select.is-white select:hover,html.theme--documenter-dark .select.is-white select.is-hovered{border-color:#f2f2f2}html.theme--documenter-dark .select.is-white select:focus,html.theme--documenter-dark .select.is-white select.is-focused,html.theme--documenter-dark .select.is-white select:active,html.theme--documenter-dark .select.is-white select.is-active{box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}html.theme--documenter-dark .select.is-black:not(:hover)::after{border-color:#0a0a0a}html.theme--documenter-dark .select.is-black select{border-color:#0a0a0a}html.theme--documenter-dark .select.is-black select:hover,html.theme--documenter-dark .select.is-black select.is-hovered{border-color:#000}html.theme--documenter-dark .select.is-black select:focus,html.theme--documenter-dark .select.is-black select.is-focused,html.theme--documenter-dark .select.is-black select:active,html.theme--documenter-dark .select.is-black select.is-active{box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}html.theme--documenter-dark .select.is-light:not(:hover)::after{border-color:#ecf0f1}html.theme--documenter-dark .select.is-light select{border-color:#ecf0f1}html.theme--documenter-dark .select.is-light select:hover,html.theme--documenter-dark .select.is-light select.is-hovered{border-color:#dde4e6}html.theme--documenter-dark .select.is-light select:focus,html.theme--documenter-dark .select.is-light select.is-focused,html.theme--documenter-dark .select.is-light select:active,html.theme--documenter-dark .select.is-light select.is-active{box-shadow:0 0 0 0.125em rgba(236,240,241,0.25)}html.theme--documenter-dark .select.is-dark:not(:hover)::after,html.theme--documenter-dark .content kbd.select:not(:hover)::after{border-color:#282f2f}html.theme--documenter-dark .select.is-dark select,html.theme--documenter-dark .content kbd.select select{border-color:#282f2f}html.theme--documenter-dark .select.is-dark select:hover,html.theme--documenter-dark .content kbd.select select:hover,html.theme--documenter-dark .select.is-dark select.is-hovered,html.theme--documenter-dark .content kbd.select select.is-hovered{border-color:#1d2122}html.theme--documenter-dark .select.is-dark select:focus,html.theme--documenter-dark .content kbd.select select:focus,html.theme--documenter-dark .select.is-dark select.is-focused,html.theme--documenter-dark .content kbd.select select.is-focused,html.theme--documenter-dark .select.is-dark select:active,html.theme--documenter-dark .content kbd.select select:active,html.theme--documenter-dark .select.is-dark select.is-active,html.theme--documenter-dark .content kbd.select select.is-active{box-shadow:0 0 0 0.125em rgba(40,47,47,0.25)}html.theme--documenter-dark .select.is-primary:not(:hover)::after,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink:not(:hover)::after{border-color:#375a7f}html.theme--documenter-dark .select.is-primary select,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select{border-color:#375a7f}html.theme--documenter-dark .select.is-primary select:hover,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select:hover,html.theme--documenter-dark .select.is-primary select.is-hovered,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select.is-hovered{border-color:#2f4d6d}html.theme--documenter-dark .select.is-primary select:focus,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select:focus,html.theme--documenter-dark .select.is-primary select.is-focused,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select.is-focused,html.theme--documenter-dark .select.is-primary select:active,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select:active,html.theme--documenter-dark .select.is-primary select.is-active,html.theme--documenter-dark .docstring>section>a.select.docs-sourcelink select.is-active{box-shadow:0 0 0 0.125em rgba(55,90,127,0.25)}html.theme--documenter-dark .select.is-link:not(:hover)::after{border-color:#1abc9c}html.theme--documenter-dark .select.is-link select{border-color:#1abc9c}html.theme--documenter-dark .select.is-link select:hover,html.theme--documenter-dark .select.is-link select.is-hovered{border-color:#17a689}html.theme--documenter-dark .select.is-link select:focus,html.theme--documenter-dark .select.is-link select.is-focused,html.theme--documenter-dark .select.is-link select:active,html.theme--documenter-dark .select.is-link select.is-active{box-shadow:0 0 0 0.125em rgba(26,188,156,0.25)}html.theme--documenter-dark .select.is-info:not(:hover)::after{border-color:#024c7d}html.theme--documenter-dark .select.is-info select{border-color:#024c7d}html.theme--documenter-dark .select.is-info select:hover,html.theme--documenter-dark .select.is-info select.is-hovered{border-color:#023d64}html.theme--documenter-dark .select.is-info select:focus,html.theme--documenter-dark .select.is-info select.is-focused,html.theme--documenter-dark .select.is-info select:active,html.theme--documenter-dark .select.is-info select.is-active{box-shadow:0 0 0 0.125em rgba(2,76,125,0.25)}html.theme--documenter-dark .select.is-success:not(:hover)::after{border-color:#008438}html.theme--documenter-dark .select.is-success select{border-color:#008438}html.theme--documenter-dark .select.is-success select:hover,html.theme--documenter-dark .select.is-success select.is-hovered{border-color:#006b2d}html.theme--documenter-dark .select.is-success select:focus,html.theme--documenter-dark .select.is-success select.is-focused,html.theme--documenter-dark .select.is-success select:active,html.theme--documenter-dark .select.is-success select.is-active{box-shadow:0 0 0 0.125em rgba(0,132,56,0.25)}html.theme--documenter-dark .select.is-warning:not(:hover)::after{border-color:#ad8100}html.theme--documenter-dark .select.is-warning select{border-color:#ad8100}html.theme--documenter-dark .select.is-warning select:hover,html.theme--documenter-dark .select.is-warning select.is-hovered{border-color:#946e00}html.theme--documenter-dark .select.is-warning select:focus,html.theme--documenter-dark .select.is-warning select.is-focused,html.theme--documenter-dark .select.is-warning select:active,html.theme--documenter-dark .select.is-warning select.is-active{box-shadow:0 0 0 0.125em rgba(173,129,0,0.25)}html.theme--documenter-dark .select.is-danger:not(:hover)::after{border-color:#9e1b0d}html.theme--documenter-dark .select.is-danger select{border-color:#9e1b0d}html.theme--documenter-dark .select.is-danger select:hover,html.theme--documenter-dark .select.is-danger select.is-hovered{border-color:#86170b}html.theme--documenter-dark .select.is-danger select:focus,html.theme--documenter-dark .select.is-danger select.is-focused,html.theme--documenter-dark .select.is-danger select:active,html.theme--documenter-dark .select.is-danger select.is-active{box-shadow:0 0 0 0.125em rgba(158,27,13,0.25)}html.theme--documenter-dark .select.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.select{border-radius:3px;font-size:.75rem}html.theme--documenter-dark .select.is-medium{font-size:1.25rem}html.theme--documenter-dark .select.is-large{font-size:1.5rem}html.theme--documenter-dark .select.is-disabled::after{border-color:#fff !important;opacity:0.5}html.theme--documenter-dark .select.is-fullwidth{width:100%}html.theme--documenter-dark .select.is-fullwidth select{width:100%}html.theme--documenter-dark .select.is-loading::after{margin-top:0;position:absolute;right:.625em;top:0.625em;transform:none}html.theme--documenter-dark .select.is-loading.is-small:after,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-loading:after{font-size:.75rem}html.theme--documenter-dark .select.is-loading.is-medium:after{font-size:1.25rem}html.theme--documenter-dark .select.is-loading.is-large:after{font-size:1.5rem}html.theme--documenter-dark .file{align-items:stretch;display:flex;justify-content:flex-start;position:relative}html.theme--documenter-dark .file.is-white .file-cta{background-color:#fff;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .file.is-white:hover .file-cta,html.theme--documenter-dark .file.is-white.is-hovered .file-cta{background-color:#f9f9f9;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .file.is-white:focus .file-cta,html.theme--documenter-dark .file.is-white.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(255,255,255,0.25);color:#0a0a0a}html.theme--documenter-dark .file.is-white:active .file-cta,html.theme--documenter-dark .file.is-white.is-active .file-cta{background-color:#f2f2f2;border-color:transparent;color:#0a0a0a}html.theme--documenter-dark .file.is-black .file-cta{background-color:#0a0a0a;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-black:hover .file-cta,html.theme--documenter-dark .file.is-black.is-hovered .file-cta{background-color:#040404;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-black:focus .file-cta,html.theme--documenter-dark .file.is-black.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(10,10,10,0.25);color:#fff}html.theme--documenter-dark .file.is-black:active .file-cta,html.theme--documenter-dark .file.is-black.is-active .file-cta{background-color:#000;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-light .file-cta{background-color:#ecf0f1;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .file.is-light:hover .file-cta,html.theme--documenter-dark .file.is-light.is-hovered .file-cta{background-color:#e5eaec;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .file.is-light:focus .file-cta,html.theme--documenter-dark .file.is-light.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(236,240,241,0.25);color:rgba(0,0,0,0.7)}html.theme--documenter-dark .file.is-light:active .file-cta,html.theme--documenter-dark .file.is-light.is-active .file-cta{background-color:#dde4e6;border-color:transparent;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .file.is-dark .file-cta,html.theme--documenter-dark .content kbd.file .file-cta{background-color:#282f2f;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-dark:hover .file-cta,html.theme--documenter-dark .content kbd.file:hover .file-cta,html.theme--documenter-dark .file.is-dark.is-hovered .file-cta,html.theme--documenter-dark .content kbd.file.is-hovered .file-cta{background-color:#232829;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-dark:focus .file-cta,html.theme--documenter-dark .content kbd.file:focus .file-cta,html.theme--documenter-dark .file.is-dark.is-focused .file-cta,html.theme--documenter-dark .content kbd.file.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(40,47,47,0.25);color:#fff}html.theme--documenter-dark .file.is-dark:active .file-cta,html.theme--documenter-dark .content kbd.file:active .file-cta,html.theme--documenter-dark .file.is-dark.is-active .file-cta,html.theme--documenter-dark .content kbd.file.is-active .file-cta{background-color:#1d2122;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-primary .file-cta,html.theme--documenter-dark .docstring>section>a.file.docs-sourcelink .file-cta{background-color:#375a7f;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-primary:hover .file-cta,html.theme--documenter-dark .docstring>section>a.file.docs-sourcelink:hover .file-cta,html.theme--documenter-dark .file.is-primary.is-hovered .file-cta,html.theme--documenter-dark .docstring>section>a.file.is-hovered.docs-sourcelink .file-cta{background-color:#335476;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-primary:focus .file-cta,html.theme--documenter-dark .docstring>section>a.file.docs-sourcelink:focus .file-cta,html.theme--documenter-dark .file.is-primary.is-focused .file-cta,html.theme--documenter-dark .docstring>section>a.file.is-focused.docs-sourcelink .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(55,90,127,0.25);color:#fff}html.theme--documenter-dark .file.is-primary:active .file-cta,html.theme--documenter-dark .docstring>section>a.file.docs-sourcelink:active .file-cta,html.theme--documenter-dark .file.is-primary.is-active .file-cta,html.theme--documenter-dark .docstring>section>a.file.is-active.docs-sourcelink .file-cta{background-color:#2f4d6d;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-link .file-cta{background-color:#1abc9c;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-link:hover .file-cta,html.theme--documenter-dark .file.is-link.is-hovered .file-cta{background-color:#18b193;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-link:focus .file-cta,html.theme--documenter-dark .file.is-link.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(26,188,156,0.25);color:#fff}html.theme--documenter-dark .file.is-link:active .file-cta,html.theme--documenter-dark .file.is-link.is-active .file-cta{background-color:#17a689;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-info .file-cta{background-color:#024c7d;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-info:hover .file-cta,html.theme--documenter-dark .file.is-info.is-hovered .file-cta{background-color:#024470;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-info:focus .file-cta,html.theme--documenter-dark .file.is-info.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(2,76,125,0.25);color:#fff}html.theme--documenter-dark .file.is-info:active .file-cta,html.theme--documenter-dark .file.is-info.is-active .file-cta{background-color:#023d64;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-success .file-cta{background-color:#008438;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-success:hover .file-cta,html.theme--documenter-dark .file.is-success.is-hovered .file-cta{background-color:#073;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-success:focus .file-cta,html.theme--documenter-dark .file.is-success.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(0,132,56,0.25);color:#fff}html.theme--documenter-dark .file.is-success:active .file-cta,html.theme--documenter-dark .file.is-success.is-active .file-cta{background-color:#006b2d;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-warning .file-cta{background-color:#ad8100;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-warning:hover .file-cta,html.theme--documenter-dark .file.is-warning.is-hovered .file-cta{background-color:#a07700;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-warning:focus .file-cta,html.theme--documenter-dark .file.is-warning.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(173,129,0,0.25);color:#fff}html.theme--documenter-dark .file.is-warning:active .file-cta,html.theme--documenter-dark .file.is-warning.is-active .file-cta{background-color:#946e00;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-danger .file-cta{background-color:#9e1b0d;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-danger:hover .file-cta,html.theme--documenter-dark .file.is-danger.is-hovered .file-cta{background-color:#92190c;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-danger:focus .file-cta,html.theme--documenter-dark .file.is-danger.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(158,27,13,0.25);color:#fff}html.theme--documenter-dark .file.is-danger:active .file-cta,html.theme--documenter-dark .file.is-danger.is-active .file-cta{background-color:#86170b;border-color:transparent;color:#fff}html.theme--documenter-dark .file.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.file{font-size:.75rem}html.theme--documenter-dark .file.is-normal{font-size:1rem}html.theme--documenter-dark .file.is-medium{font-size:1.25rem}html.theme--documenter-dark .file.is-medium .file-icon .fa{font-size:21px}html.theme--documenter-dark .file.is-large{font-size:1.5rem}html.theme--documenter-dark .file.is-large .file-icon .fa{font-size:28px}html.theme--documenter-dark .file.has-name .file-cta{border-bottom-right-radius:0;border-top-right-radius:0}html.theme--documenter-dark .file.has-name .file-name{border-bottom-left-radius:0;border-top-left-radius:0}html.theme--documenter-dark .file.has-name.is-empty .file-cta{border-radius:.4em}html.theme--documenter-dark .file.has-name.is-empty .file-name{display:none}html.theme--documenter-dark .file.is-boxed .file-label{flex-direction:column}html.theme--documenter-dark .file.is-boxed .file-cta{flex-direction:column;height:auto;padding:1em 3em}html.theme--documenter-dark .file.is-boxed .file-name{border-width:0 1px 1px}html.theme--documenter-dark .file.is-boxed .file-icon{height:1.5em;width:1.5em}html.theme--documenter-dark .file.is-boxed .file-icon .fa{font-size:21px}html.theme--documenter-dark .file.is-boxed.is-small .file-icon .fa,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-boxed .file-icon .fa{font-size:14px}html.theme--documenter-dark .file.is-boxed.is-medium .file-icon .fa{font-size:28px}html.theme--documenter-dark .file.is-boxed.is-large .file-icon .fa{font-size:35px}html.theme--documenter-dark .file.is-boxed.has-name .file-cta{border-radius:.4em .4em 0 0}html.theme--documenter-dark .file.is-boxed.has-name .file-name{border-radius:0 0 .4em .4em;border-width:0 1px 1px}html.theme--documenter-dark .file.is-centered{justify-content:center}html.theme--documenter-dark .file.is-fullwidth .file-label{width:100%}html.theme--documenter-dark .file.is-fullwidth .file-name{flex-grow:1;max-width:none}html.theme--documenter-dark .file.is-right{justify-content:flex-end}html.theme--documenter-dark .file.is-right .file-cta{border-radius:0 .4em .4em 0}html.theme--documenter-dark .file.is-right .file-name{border-radius:.4em 0 0 .4em;border-width:1px 0 1px 1px;order:-1}html.theme--documenter-dark .file-label{align-items:stretch;display:flex;cursor:pointer;justify-content:flex-start;overflow:hidden;position:relative}html.theme--documenter-dark .file-label:hover .file-cta{background-color:#232829;color:#f2f2f2}html.theme--documenter-dark .file-label:hover .file-name{border-color:#596668}html.theme--documenter-dark .file-label:active .file-cta{background-color:#1d2122;color:#f2f2f2}html.theme--documenter-dark .file-label:active .file-name{border-color:#535f61}html.theme--documenter-dark .file-input{height:100%;left:0;opacity:0;outline:none;position:absolute;top:0;width:100%}html.theme--documenter-dark .file-cta,html.theme--documenter-dark .file-name{border-color:#5e6d6f;border-radius:.4em;font-size:1em;padding-left:1em;padding-right:1em;white-space:nowrap}html.theme--documenter-dark .file-cta{background-color:#282f2f;color:#fff}html.theme--documenter-dark .file-name{border-color:#5e6d6f;border-style:solid;border-width:1px 1px 1px 0;display:block;max-width:16em;overflow:hidden;text-align:inherit;text-overflow:ellipsis}html.theme--documenter-dark .file-icon{align-items:center;display:flex;height:1em;justify-content:center;margin-right:.5em;width:1em}html.theme--documenter-dark .file-icon .fa{font-size:14px}html.theme--documenter-dark .label{color:#f2f2f2;display:block;font-size:1rem;font-weight:700}html.theme--documenter-dark .label:not(:last-child){margin-bottom:0.5em}html.theme--documenter-dark .label.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.label{font-size:.75rem}html.theme--documenter-dark .label.is-medium{font-size:1.25rem}html.theme--documenter-dark .label.is-large{font-size:1.5rem}html.theme--documenter-dark .help{display:block;font-size:.75rem;margin-top:0.25rem}html.theme--documenter-dark .help.is-white{color:#fff}html.theme--documenter-dark .help.is-black{color:#0a0a0a}html.theme--documenter-dark .help.is-light{color:#ecf0f1}html.theme--documenter-dark .help.is-dark,html.theme--documenter-dark .content kbd.help{color:#282f2f}html.theme--documenter-dark .help.is-primary,html.theme--documenter-dark .docstring>section>a.help.docs-sourcelink{color:#375a7f}html.theme--documenter-dark .help.is-link{color:#1abc9c}html.theme--documenter-dark .help.is-info{color:#024c7d}html.theme--documenter-dark .help.is-success{color:#008438}html.theme--documenter-dark .help.is-warning{color:#ad8100}html.theme--documenter-dark .help.is-danger{color:#9e1b0d}html.theme--documenter-dark .field:not(:last-child){margin-bottom:0.75rem}html.theme--documenter-dark .field.has-addons{display:flex;justify-content:flex-start}html.theme--documenter-dark .field.has-addons .control:not(:last-child){margin-right:-1px}html.theme--documenter-dark .field.has-addons .control:not(:first-child):not(:last-child) .button,html.theme--documenter-dark .field.has-addons .control:not(:first-child):not(:last-child) .input,html.theme--documenter-dark .field.has-addons .control:not(:first-child):not(:last-child) #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control:not(:first-child):not(:last-child) form.docs-search>input,html.theme--documenter-dark .field.has-addons .control:not(:first-child):not(:last-child) .select select{border-radius:0}html.theme--documenter-dark .field.has-addons .control:first-child:not(:only-child) .button,html.theme--documenter-dark .field.has-addons .control:first-child:not(:only-child) .input,html.theme--documenter-dark .field.has-addons .control:first-child:not(:only-child) #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control:first-child:not(:only-child) form.docs-search>input,html.theme--documenter-dark .field.has-addons .control:first-child:not(:only-child) .select select{border-bottom-right-radius:0;border-top-right-radius:0}html.theme--documenter-dark .field.has-addons .control:last-child:not(:only-child) .button,html.theme--documenter-dark .field.has-addons .control:last-child:not(:only-child) .input,html.theme--documenter-dark .field.has-addons .control:last-child:not(:only-child) #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control:last-child:not(:only-child) form.docs-search>input,html.theme--documenter-dark .field.has-addons .control:last-child:not(:only-child) .select select{border-bottom-left-radius:0;border-top-left-radius:0}html.theme--documenter-dark .field.has-addons .control .button:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .button.is-hovered:not([disabled]),html.theme--documenter-dark .field.has-addons .control .input:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):hover,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .input.is-hovered:not([disabled]),html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-hovered:not([disabled]),html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-hovered:not([disabled]),html.theme--documenter-dark .field.has-addons .control .select select:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .select select.is-hovered:not([disabled]){z-index:2}html.theme--documenter-dark .field.has-addons .control .button:not([disabled]):focus,html.theme--documenter-dark .field.has-addons .control .button.is-focused:not([disabled]),html.theme--documenter-dark .field.has-addons .control .button:not([disabled]):active,html.theme--documenter-dark .field.has-addons .control .button.is-active:not([disabled]),html.theme--documenter-dark .field.has-addons .control .input:not([disabled]):focus,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):focus,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):focus,html.theme--documenter-dark .field.has-addons .control .input.is-focused:not([disabled]),html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-focused:not([disabled]),html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-focused:not([disabled]),html.theme--documenter-dark .field.has-addons .control .input:not([disabled]):active,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):active,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):active,html.theme--documenter-dark .field.has-addons .control .input.is-active:not([disabled]),html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-active:not([disabled]),html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-active:not([disabled]),html.theme--documenter-dark .field.has-addons .control .select select:not([disabled]):focus,html.theme--documenter-dark .field.has-addons .control .select select.is-focused:not([disabled]),html.theme--documenter-dark .field.has-addons .control .select select:not([disabled]):active,html.theme--documenter-dark .field.has-addons .control .select select.is-active:not([disabled]){z-index:3}html.theme--documenter-dark .field.has-addons .control .button:not([disabled]):focus:hover,html.theme--documenter-dark .field.has-addons .control .button.is-focused:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .button:not([disabled]):active:hover,html.theme--documenter-dark .field.has-addons .control .button.is-active:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .input:not([disabled]):focus:hover,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):focus:hover,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):focus:hover,html.theme--documenter-dark .field.has-addons .control .input.is-focused:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-focused:not([disabled]):hover,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-focused:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .input:not([disabled]):active:hover,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):active:hover,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):active:hover,html.theme--documenter-dark .field.has-addons .control .input.is-active:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-active:not([disabled]):hover,html.theme--documenter-dark #documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-active:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .select select:not([disabled]):focus:hover,html.theme--documenter-dark .field.has-addons .control .select select.is-focused:not([disabled]):hover,html.theme--documenter-dark .field.has-addons .control .select select:not([disabled]):active:hover,html.theme--documenter-dark .field.has-addons .control .select select.is-active:not([disabled]):hover{z-index:4}html.theme--documenter-dark .field.has-addons .control.is-expanded{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .field.has-addons.has-addons-centered{justify-content:center}html.theme--documenter-dark .field.has-addons.has-addons-right{justify-content:flex-end}html.theme--documenter-dark .field.has-addons.has-addons-fullwidth .control{flex-grow:1;flex-shrink:0}html.theme--documenter-dark .field.is-grouped{display:flex;justify-content:flex-start}html.theme--documenter-dark .field.is-grouped>.control{flex-shrink:0}html.theme--documenter-dark .field.is-grouped>.control:not(:last-child){margin-bottom:0;margin-right:.75rem}html.theme--documenter-dark .field.is-grouped>.control.is-expanded{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .field.is-grouped.is-grouped-centered{justify-content:center}html.theme--documenter-dark .field.is-grouped.is-grouped-right{justify-content:flex-end}html.theme--documenter-dark .field.is-grouped.is-grouped-multiline{flex-wrap:wrap}html.theme--documenter-dark .field.is-grouped.is-grouped-multiline>.control:last-child,html.theme--documenter-dark .field.is-grouped.is-grouped-multiline>.control:not(:last-child){margin-bottom:0.75rem}html.theme--documenter-dark .field.is-grouped.is-grouped-multiline:last-child{margin-bottom:-0.75rem}html.theme--documenter-dark .field.is-grouped.is-grouped-multiline:not(:last-child){margin-bottom:0}@media screen and (min-width: 769px),print{html.theme--documenter-dark .field.is-horizontal{display:flex}}html.theme--documenter-dark .field-label .label{font-size:inherit}@media screen and (max-width: 768px){html.theme--documenter-dark .field-label{margin-bottom:0.5rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .field-label{flex-basis:0;flex-grow:1;flex-shrink:0;margin-right:1.5rem;text-align:right}html.theme--documenter-dark .field-label.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.field-label{font-size:.75rem;padding-top:0.375em}html.theme--documenter-dark .field-label.is-normal{padding-top:0.375em}html.theme--documenter-dark .field-label.is-medium{font-size:1.25rem;padding-top:0.375em}html.theme--documenter-dark .field-label.is-large{font-size:1.5rem;padding-top:0.375em}}html.theme--documenter-dark .field-body .field .field{margin-bottom:0}@media screen and (min-width: 769px),print{html.theme--documenter-dark .field-body{display:flex;flex-basis:0;flex-grow:5;flex-shrink:1}html.theme--documenter-dark .field-body .field{margin-bottom:0}html.theme--documenter-dark .field-body>.field{flex-shrink:1}html.theme--documenter-dark .field-body>.field:not(.is-narrow){flex-grow:1}html.theme--documenter-dark .field-body>.field:not(:last-child){margin-right:.75rem}}html.theme--documenter-dark .control{box-sizing:border-box;clear:both;font-size:1rem;position:relative;text-align:inherit}html.theme--documenter-dark .control.has-icons-left .input:focus~.icon,html.theme--documenter-dark .control.has-icons-left #documenter .docs-sidebar form.docs-search>input:focus~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-left form.docs-search>input:focus~.icon,html.theme--documenter-dark .control.has-icons-left .select:focus~.icon,html.theme--documenter-dark .control.has-icons-right .input:focus~.icon,html.theme--documenter-dark .control.has-icons-right #documenter .docs-sidebar form.docs-search>input:focus~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-right form.docs-search>input:focus~.icon,html.theme--documenter-dark .control.has-icons-right .select:focus~.icon{color:#282f2f}html.theme--documenter-dark .control.has-icons-left .input.is-small~.icon,html.theme--documenter-dark .control.has-icons-left #documenter .docs-sidebar form.docs-search>input~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-left form.docs-search>input~.icon,html.theme--documenter-dark .control.has-icons-left .select.is-small~.icon,html.theme--documenter-dark .control.has-icons-right .input.is-small~.icon,html.theme--documenter-dark .control.has-icons-right #documenter .docs-sidebar form.docs-search>input~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-right form.docs-search>input~.icon,html.theme--documenter-dark .control.has-icons-right .select.is-small~.icon{font-size:.75rem}html.theme--documenter-dark .control.has-icons-left .input.is-medium~.icon,html.theme--documenter-dark .control.has-icons-left #documenter .docs-sidebar form.docs-search>input.is-medium~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-left form.docs-search>input.is-medium~.icon,html.theme--documenter-dark .control.has-icons-left .select.is-medium~.icon,html.theme--documenter-dark .control.has-icons-right .input.is-medium~.icon,html.theme--documenter-dark .control.has-icons-right #documenter .docs-sidebar form.docs-search>input.is-medium~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-right form.docs-search>input.is-medium~.icon,html.theme--documenter-dark .control.has-icons-right .select.is-medium~.icon{font-size:1.25rem}html.theme--documenter-dark .control.has-icons-left .input.is-large~.icon,html.theme--documenter-dark .control.has-icons-left #documenter .docs-sidebar form.docs-search>input.is-large~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-left form.docs-search>input.is-large~.icon,html.theme--documenter-dark .control.has-icons-left .select.is-large~.icon,html.theme--documenter-dark .control.has-icons-right .input.is-large~.icon,html.theme--documenter-dark .control.has-icons-right #documenter .docs-sidebar form.docs-search>input.is-large~.icon,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-right form.docs-search>input.is-large~.icon,html.theme--documenter-dark .control.has-icons-right .select.is-large~.icon{font-size:1.5rem}html.theme--documenter-dark .control.has-icons-left .icon,html.theme--documenter-dark .control.has-icons-right .icon{color:#5e6d6f;height:2.5em;pointer-events:none;position:absolute;top:0;width:2.5em;z-index:4}html.theme--documenter-dark .control.has-icons-left .input,html.theme--documenter-dark .control.has-icons-left #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-left form.docs-search>input,html.theme--documenter-dark .control.has-icons-left .select select{padding-left:2.5em}html.theme--documenter-dark .control.has-icons-left .icon.is-left{left:0}html.theme--documenter-dark .control.has-icons-right .input,html.theme--documenter-dark .control.has-icons-right #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .control.has-icons-right form.docs-search>input,html.theme--documenter-dark .control.has-icons-right .select select{padding-right:2.5em}html.theme--documenter-dark .control.has-icons-right .icon.is-right{right:0}html.theme--documenter-dark .control.is-loading::after{position:absolute !important;right:.625em;top:0.625em;z-index:4}html.theme--documenter-dark .control.is-loading.is-small:after,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.is-loading:after{font-size:.75rem}html.theme--documenter-dark .control.is-loading.is-medium:after{font-size:1.25rem}html.theme--documenter-dark .control.is-loading.is-large:after{font-size:1.5rem}html.theme--documenter-dark .breadcrumb{font-size:1rem;white-space:nowrap}html.theme--documenter-dark .breadcrumb a{align-items:center;color:#1abc9c;display:flex;justify-content:center;padding:0 .75em}html.theme--documenter-dark .breadcrumb a:hover{color:#1dd2af}html.theme--documenter-dark .breadcrumb li{align-items:center;display:flex}html.theme--documenter-dark .breadcrumb li:first-child a{padding-left:0}html.theme--documenter-dark .breadcrumb li.is-active a{color:#f2f2f2;cursor:default;pointer-events:none}html.theme--documenter-dark .breadcrumb li+li::before{color:#8c9b9d;content:"\0002f"}html.theme--documenter-dark .breadcrumb ul,html.theme--documenter-dark .breadcrumb ol{align-items:flex-start;display:flex;flex-wrap:wrap;justify-content:flex-start}html.theme--documenter-dark .breadcrumb .icon:first-child{margin-right:.5em}html.theme--documenter-dark .breadcrumb .icon:last-child{margin-left:.5em}html.theme--documenter-dark .breadcrumb.is-centered ol,html.theme--documenter-dark .breadcrumb.is-centered ul{justify-content:center}html.theme--documenter-dark .breadcrumb.is-right ol,html.theme--documenter-dark .breadcrumb.is-right ul{justify-content:flex-end}html.theme--documenter-dark .breadcrumb.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.breadcrumb{font-size:.75rem}html.theme--documenter-dark .breadcrumb.is-medium{font-size:1.25rem}html.theme--documenter-dark .breadcrumb.is-large{font-size:1.5rem}html.theme--documenter-dark .breadcrumb.has-arrow-separator li+li::before{content:"\02192"}html.theme--documenter-dark .breadcrumb.has-bullet-separator li+li::before{content:"\02022"}html.theme--documenter-dark .breadcrumb.has-dot-separator li+li::before{content:"\000b7"}html.theme--documenter-dark .breadcrumb.has-succeeds-separator li+li::before{content:"\0227B"}html.theme--documenter-dark .card{background-color:#fff;border-radius:.25rem;box-shadow:#171717;color:#fff;max-width:100%;position:relative}html.theme--documenter-dark .card-footer:first-child,html.theme--documenter-dark .card-content:first-child,html.theme--documenter-dark .card-header:first-child{border-top-left-radius:.25rem;border-top-right-radius:.25rem}html.theme--documenter-dark .card-footer:last-child,html.theme--documenter-dark .card-content:last-child,html.theme--documenter-dark .card-header:last-child{border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem}html.theme--documenter-dark .card-header{background-color:rgba(0,0,0,0);align-items:stretch;box-shadow:0 0.125em 0.25em rgba(10,10,10,0.1);display:flex}html.theme--documenter-dark .card-header-title{align-items:center;color:#f2f2f2;display:flex;flex-grow:1;font-weight:700;padding:0.75rem 1rem}html.theme--documenter-dark .card-header-title.is-centered{justify-content:center}html.theme--documenter-dark .card-header-icon{-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;color:currentColor;font-family:inherit;font-size:1em;margin:0;padding:0;align-items:center;cursor:pointer;display:flex;justify-content:center;padding:0.75rem 1rem}html.theme--documenter-dark .card-image{display:block;position:relative}html.theme--documenter-dark .card-image:first-child img{border-top-left-radius:.25rem;border-top-right-radius:.25rem}html.theme--documenter-dark .card-image:last-child img{border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem}html.theme--documenter-dark .card-content{background-color:rgba(0,0,0,0);padding:1.5rem}html.theme--documenter-dark .card-footer{background-color:rgba(0,0,0,0);border-top:1px solid #ededed;align-items:stretch;display:flex}html.theme--documenter-dark .card-footer-item{align-items:center;display:flex;flex-basis:0;flex-grow:1;flex-shrink:0;justify-content:center;padding:.75rem}html.theme--documenter-dark .card-footer-item:not(:last-child){border-right:1px solid #ededed}html.theme--documenter-dark .card .media:not(:last-child){margin-bottom:1.5rem}html.theme--documenter-dark .dropdown{display:inline-flex;position:relative;vertical-align:top}html.theme--documenter-dark .dropdown.is-active .dropdown-menu,html.theme--documenter-dark .dropdown.is-hoverable:hover .dropdown-menu{display:block}html.theme--documenter-dark .dropdown.is-right .dropdown-menu{left:auto;right:0}html.theme--documenter-dark .dropdown.is-up .dropdown-menu{bottom:100%;padding-bottom:4px;padding-top:initial;top:auto}html.theme--documenter-dark .dropdown-menu{display:none;left:0;min-width:12rem;padding-top:4px;position:absolute;top:100%;z-index:20}html.theme--documenter-dark .dropdown-content{background-color:#282f2f;border-radius:.4em;box-shadow:#171717;padding-bottom:.5rem;padding-top:.5rem}html.theme--documenter-dark .dropdown-item{color:#fff;display:block;font-size:0.875rem;line-height:1.5;padding:0.375rem 1rem;position:relative}html.theme--documenter-dark a.dropdown-item,html.theme--documenter-dark button.dropdown-item{padding-right:3rem;text-align:inherit;white-space:nowrap;width:100%}html.theme--documenter-dark a.dropdown-item:hover,html.theme--documenter-dark button.dropdown-item:hover{background-color:#282f2f;color:#0a0a0a}html.theme--documenter-dark a.dropdown-item.is-active,html.theme--documenter-dark button.dropdown-item.is-active{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .dropdown-divider{background-color:#ededed;border:none;display:block;height:1px;margin:0.5rem 0}html.theme--documenter-dark .level{align-items:center;justify-content:space-between}html.theme--documenter-dark .level code{border-radius:.4em}html.theme--documenter-dark .level img{display:inline-block;vertical-align:top}html.theme--documenter-dark .level.is-mobile{display:flex}html.theme--documenter-dark .level.is-mobile .level-left,html.theme--documenter-dark .level.is-mobile .level-right{display:flex}html.theme--documenter-dark .level.is-mobile .level-left+.level-right{margin-top:0}html.theme--documenter-dark .level.is-mobile .level-item:not(:last-child){margin-bottom:0;margin-right:.75rem}html.theme--documenter-dark .level.is-mobile .level-item:not(.is-narrow){flex-grow:1}@media screen and (min-width: 769px),print{html.theme--documenter-dark .level{display:flex}html.theme--documenter-dark .level>.level-item:not(.is-narrow){flex-grow:1}}html.theme--documenter-dark .level-item{align-items:center;display:flex;flex-basis:auto;flex-grow:0;flex-shrink:0;justify-content:center}html.theme--documenter-dark .level-item .title,html.theme--documenter-dark .level-item .subtitle{margin-bottom:0}@media screen and (max-width: 768px){html.theme--documenter-dark .level-item:not(:last-child){margin-bottom:.75rem}}html.theme--documenter-dark .level-left,html.theme--documenter-dark .level-right{flex-basis:auto;flex-grow:0;flex-shrink:0}html.theme--documenter-dark .level-left .level-item.is-flexible,html.theme--documenter-dark .level-right .level-item.is-flexible{flex-grow:1}@media screen and (min-width: 769px),print{html.theme--documenter-dark .level-left .level-item:not(:last-child),html.theme--documenter-dark .level-right .level-item:not(:last-child){margin-right:.75rem}}html.theme--documenter-dark .level-left{align-items:center;justify-content:flex-start}@media screen and (max-width: 768px){html.theme--documenter-dark .level-left+.level-right{margin-top:1.5rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .level-left{display:flex}}html.theme--documenter-dark .level-right{align-items:center;justify-content:flex-end}@media screen and (min-width: 769px),print{html.theme--documenter-dark .level-right{display:flex}}html.theme--documenter-dark .media{align-items:flex-start;display:flex;text-align:inherit}html.theme--documenter-dark .media .content:not(:last-child){margin-bottom:.75rem}html.theme--documenter-dark .media .media{border-top:1px solid rgba(94,109,111,0.5);display:flex;padding-top:.75rem}html.theme--documenter-dark .media .media .content:not(:last-child),html.theme--documenter-dark .media .media .control:not(:last-child){margin-bottom:.5rem}html.theme--documenter-dark .media .media .media{padding-top:.5rem}html.theme--documenter-dark .media .media .media+.media{margin-top:.5rem}html.theme--documenter-dark .media+.media{border-top:1px solid rgba(94,109,111,0.5);margin-top:1rem;padding-top:1rem}html.theme--documenter-dark .media.is-large+.media{margin-top:1.5rem;padding-top:1.5rem}html.theme--documenter-dark .media-left,html.theme--documenter-dark .media-right{flex-basis:auto;flex-grow:0;flex-shrink:0}html.theme--documenter-dark .media-left{margin-right:1rem}html.theme--documenter-dark .media-right{margin-left:1rem}html.theme--documenter-dark .media-content{flex-basis:auto;flex-grow:1;flex-shrink:1;text-align:inherit}@media screen and (max-width: 768px){html.theme--documenter-dark .media-content{overflow-x:auto}}html.theme--documenter-dark .menu{font-size:1rem}html.theme--documenter-dark .menu.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.menu{font-size:.75rem}html.theme--documenter-dark .menu.is-medium{font-size:1.25rem}html.theme--documenter-dark .menu.is-large{font-size:1.5rem}html.theme--documenter-dark .menu-list{line-height:1.25}html.theme--documenter-dark .menu-list a{border-radius:3px;color:#fff;display:block;padding:0.5em 0.75em}html.theme--documenter-dark .menu-list a:hover{background-color:#282f2f;color:#f2f2f2}html.theme--documenter-dark .menu-list a.is-active{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .menu-list li ul{border-left:1px solid #5e6d6f;margin:.75em;padding-left:.75em}html.theme--documenter-dark .menu-label{color:#fff;font-size:.75em;letter-spacing:.1em;text-transform:uppercase}html.theme--documenter-dark .menu-label:not(:first-child){margin-top:1em}html.theme--documenter-dark .menu-label:not(:last-child){margin-bottom:1em}html.theme--documenter-dark .message{background-color:#282f2f;border-radius:.4em;font-size:1rem}html.theme--documenter-dark .message strong{color:currentColor}html.theme--documenter-dark .message a:not(.button):not(.tag):not(.dropdown-item){color:currentColor;text-decoration:underline}html.theme--documenter-dark .message.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.message{font-size:.75rem}html.theme--documenter-dark .message.is-medium{font-size:1.25rem}html.theme--documenter-dark .message.is-large{font-size:1.5rem}html.theme--documenter-dark .message.is-white{background-color:#fff}html.theme--documenter-dark .message.is-white .message-header{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .message.is-white .message-body{border-color:#fff}html.theme--documenter-dark .message.is-black{background-color:#fafafa}html.theme--documenter-dark .message.is-black .message-header{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .message.is-black .message-body{border-color:#0a0a0a}html.theme--documenter-dark .message.is-light{background-color:#f9fafb}html.theme--documenter-dark .message.is-light .message-header{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .message.is-light .message-body{border-color:#ecf0f1}html.theme--documenter-dark .message.is-dark,html.theme--documenter-dark .content kbd.message{background-color:#f9fafa}html.theme--documenter-dark .message.is-dark .message-header,html.theme--documenter-dark .content kbd.message .message-header{background-color:#282f2f;color:#fff}html.theme--documenter-dark .message.is-dark .message-body,html.theme--documenter-dark .content kbd.message .message-body{border-color:#282f2f}html.theme--documenter-dark .message.is-primary,html.theme--documenter-dark .docstring>section>a.message.docs-sourcelink{background-color:#f1f5f9}html.theme--documenter-dark .message.is-primary .message-header,html.theme--documenter-dark .docstring>section>a.message.docs-sourcelink .message-header{background-color:#375a7f;color:#fff}html.theme--documenter-dark .message.is-primary .message-body,html.theme--documenter-dark .docstring>section>a.message.docs-sourcelink .message-body{border-color:#375a7f;color:#4d7eb2}html.theme--documenter-dark .message.is-link{background-color:#edfdf9}html.theme--documenter-dark .message.is-link .message-header{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .message.is-link .message-body{border-color:#1abc9c;color:#15987e}html.theme--documenter-dark .message.is-info{background-color:#ebf7ff}html.theme--documenter-dark .message.is-info .message-header{background-color:#024c7d;color:#fff}html.theme--documenter-dark .message.is-info .message-body{border-color:#024c7d;color:#0e9dfb}html.theme--documenter-dark .message.is-success{background-color:#ebfff3}html.theme--documenter-dark .message.is-success .message-header{background-color:#008438;color:#fff}html.theme--documenter-dark .message.is-success .message-body{border-color:#008438;color:#00eb64}html.theme--documenter-dark .message.is-warning{background-color:#fffaeb}html.theme--documenter-dark .message.is-warning .message-header{background-color:#ad8100;color:#fff}html.theme--documenter-dark .message.is-warning .message-body{border-color:#ad8100;color:#d19c00}html.theme--documenter-dark .message.is-danger{background-color:#fdeeec}html.theme--documenter-dark .message.is-danger .message-header{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .message.is-danger .message-body{border-color:#9e1b0d;color:#ec311d}html.theme--documenter-dark .message-header{align-items:center;background-color:#fff;border-radius:.4em .4em 0 0;color:rgba(0,0,0,0.7);display:flex;font-weight:700;justify-content:space-between;line-height:1.25;padding:0.75em 1em;position:relative}html.theme--documenter-dark .message-header .delete{flex-grow:0;flex-shrink:0;margin-left:.75em}html.theme--documenter-dark .message-header+.message-body{border-width:0;border-top-left-radius:0;border-top-right-radius:0}html.theme--documenter-dark .message-body{border-color:#5e6d6f;border-radius:.4em;border-style:solid;border-width:0 0 0 4px;color:#fff;padding:1.25em 1.5em}html.theme--documenter-dark .message-body code,html.theme--documenter-dark .message-body pre{background-color:#fff}html.theme--documenter-dark .message-body pre code{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .modal{align-items:center;display:none;flex-direction:column;justify-content:center;overflow:hidden;position:fixed;z-index:40}html.theme--documenter-dark .modal.is-active{display:flex}html.theme--documenter-dark .modal-background{background-color:rgba(10,10,10,0.86)}html.theme--documenter-dark .modal-content,html.theme--documenter-dark .modal-card{margin:0 20px;max-height:calc(100vh - 160px);overflow:auto;position:relative;width:100%}@media screen and (min-width: 769px){html.theme--documenter-dark .modal-content,html.theme--documenter-dark .modal-card{margin:0 auto;max-height:calc(100vh - 40px);width:640px}}html.theme--documenter-dark .modal-close{background:none;height:40px;position:fixed;right:20px;top:20px;width:40px}html.theme--documenter-dark .modal-card{display:flex;flex-direction:column;max-height:calc(100vh - 40px);overflow:hidden;-ms-overflow-y:visible}html.theme--documenter-dark .modal-card-head,html.theme--documenter-dark .modal-card-foot{align-items:center;background-color:#282f2f;display:flex;flex-shrink:0;justify-content:flex-start;padding:20px;position:relative}html.theme--documenter-dark .modal-card-head{border-bottom:1px solid #5e6d6f;border-top-left-radius:8px;border-top-right-radius:8px}html.theme--documenter-dark .modal-card-title{color:#f2f2f2;flex-grow:1;flex-shrink:0;font-size:1.5rem;line-height:1}html.theme--documenter-dark .modal-card-foot{border-bottom-left-radius:8px;border-bottom-right-radius:8px;border-top:1px solid #5e6d6f}html.theme--documenter-dark .modal-card-foot .button:not(:last-child){margin-right:.5em}html.theme--documenter-dark .modal-card-body{-webkit-overflow-scrolling:touch;background-color:#fff;flex-grow:1;flex-shrink:1;overflow:auto;padding:20px}html.theme--documenter-dark .navbar{background-color:#375a7f;min-height:4rem;position:relative;z-index:30}html.theme--documenter-dark .navbar.is-white{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-white .navbar-brand .navbar-link{color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-white .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-white .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-white .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-white .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-white .navbar-brand .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-brand .navbar-link::after{border-color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-burger{color:#0a0a0a}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-white .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-white .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-white .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-white .navbar-end .navbar-link{color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-white .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-white .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-white .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-white .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-white .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-white .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-white .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-white .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-white .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-white .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-white .navbar-end .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-white .navbar-end .navbar-link::after{border-color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-white .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-white .navbar-item.has-dropdown.is-active .navbar-link{background-color:#f2f2f2;color:#0a0a0a}html.theme--documenter-dark .navbar.is-white .navbar-dropdown a.navbar-item.is-active{background-color:#fff;color:#0a0a0a}}html.theme--documenter-dark .navbar.is-black{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-black .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-black .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-black .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-black .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-black .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-black .navbar-brand .navbar-link.is-active{background-color:#000;color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-black .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-black .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-black .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-black .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-black .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-black .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-black .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-black .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-black .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-black .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-black .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-black .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-black .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-black .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-black .navbar-end .navbar-link.is-active{background-color:#000;color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-black .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-black .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-black .navbar-item.has-dropdown.is-active .navbar-link{background-color:#000;color:#fff}html.theme--documenter-dark .navbar.is-black .navbar-dropdown a.navbar-item.is-active{background-color:#0a0a0a;color:#fff}}html.theme--documenter-dark .navbar.is-light{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-light .navbar-brand .navbar-link{color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-light .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-light .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-light .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-light .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-light .navbar-brand .navbar-link.is-active{background-color:#dde4e6;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-brand .navbar-link::after{border-color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-burger{color:rgba(0,0,0,0.7)}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-light .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-light .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-light .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-light .navbar-end .navbar-link{color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-light .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-light .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-light .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-light .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-light .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-light .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-light .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-light .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-light .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-light .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-light .navbar-end .navbar-link.is-active{background-color:#dde4e6;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-light .navbar-end .navbar-link::after{border-color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-light .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-light .navbar-item.has-dropdown.is-active .navbar-link{background-color:#dde4e6;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .navbar.is-light .navbar-dropdown a.navbar-item.is-active{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}}html.theme--documenter-dark .navbar.is-dark,html.theme--documenter-dark .content kbd.navbar{background-color:#282f2f;color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-brand>.navbar-item,html.theme--documenter-dark .content kbd.navbar .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-dark .navbar-brand .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .content kbd.navbar .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-dark .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .content kbd.navbar .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-dark .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-dark .navbar-brand .navbar-link:focus,html.theme--documenter-dark .content kbd.navbar .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-dark .navbar-brand .navbar-link:hover,html.theme--documenter-dark .content kbd.navbar .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-dark .navbar-brand .navbar-link.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-brand .navbar-link.is-active{background-color:#1d2122;color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-brand .navbar-link::after,html.theme--documenter-dark .content kbd.navbar .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-burger,html.theme--documenter-dark .content kbd.navbar .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-dark .navbar-start>.navbar-item,html.theme--documenter-dark .content kbd.navbar .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-dark .navbar-start .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-dark .navbar-end>.navbar-item,html.theme--documenter-dark .content kbd.navbar .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-dark .navbar-end .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .content kbd.navbar .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-dark .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .content kbd.navbar .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-dark .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-dark .navbar-start .navbar-link:focus,html.theme--documenter-dark .content kbd.navbar .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-dark .navbar-start .navbar-link:hover,html.theme--documenter-dark .content kbd.navbar .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-dark .navbar-start .navbar-link.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-dark .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .content kbd.navbar .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-dark .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .content kbd.navbar .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-dark .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-dark .navbar-end .navbar-link:focus,html.theme--documenter-dark .content kbd.navbar .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-dark .navbar-end .navbar-link:hover,html.theme--documenter-dark .content kbd.navbar .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-dark .navbar-end .navbar-link.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-end .navbar-link.is-active{background-color:#1d2122;color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-start .navbar-link::after,html.theme--documenter-dark .content kbd.navbar .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-dark .navbar-end .navbar-link::after,html.theme--documenter-dark .content kbd.navbar .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-dark .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-dark .navbar-item.has-dropdown.is-active .navbar-link,html.theme--documenter-dark .content kbd.navbar .navbar-item.has-dropdown.is-active .navbar-link{background-color:#1d2122;color:#fff}html.theme--documenter-dark .navbar.is-dark .navbar-dropdown a.navbar-item.is-active,html.theme--documenter-dark .content kbd.navbar .navbar-dropdown a.navbar-item.is-active{background-color:#282f2f;color:#fff}}html.theme--documenter-dark .navbar.is-primary,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink{background-color:#375a7f;color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-brand>.navbar-item,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-primary .navbar-brand .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-primary .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-primary .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-primary .navbar-brand .navbar-link:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-primary .navbar-brand .navbar-link:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-primary .navbar-brand .navbar-link.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link.is-active{background-color:#2f4d6d;color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-brand .navbar-link::after,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-burger,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-primary .navbar-start>.navbar-item,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-primary .navbar-start .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-primary .navbar-end>.navbar-item,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-primary .navbar-end .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-primary .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-primary .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-primary .navbar-start .navbar-link:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-primary .navbar-start .navbar-link:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-primary .navbar-start .navbar-link.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-primary .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-primary .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-primary .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-primary .navbar-end .navbar-link:focus,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-primary .navbar-end .navbar-link:hover,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-primary .navbar-end .navbar-link.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link.is-active{background-color:#2f4d6d;color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-start .navbar-link::after,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-primary .navbar-end .navbar-link::after,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-primary .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-primary .navbar-item.has-dropdown.is-active .navbar-link,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown.is-active .navbar-link{background-color:#2f4d6d;color:#fff}html.theme--documenter-dark .navbar.is-primary .navbar-dropdown a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-dropdown a.navbar-item.is-active{background-color:#375a7f;color:#fff}}html.theme--documenter-dark .navbar.is-link{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-link .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-link .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-link .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-link .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-link .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-link .navbar-brand .navbar-link.is-active{background-color:#17a689;color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-link .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-link .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-link .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-link .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-link .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-link .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-link .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-link .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-link .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-link .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-link .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-link .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-link .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-link .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-link .navbar-end .navbar-link.is-active{background-color:#17a689;color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-link .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-link .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-link .navbar-item.has-dropdown.is-active .navbar-link{background-color:#17a689;color:#fff}html.theme--documenter-dark .navbar.is-link .navbar-dropdown a.navbar-item.is-active{background-color:#1abc9c;color:#fff}}html.theme--documenter-dark .navbar.is-info{background-color:#024c7d;color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-info .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-info .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-info .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-info .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-info .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-info .navbar-brand .navbar-link.is-active{background-color:#023d64;color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-info .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-info .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-info .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-info .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-info .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-info .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-info .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-info .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-info .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-info .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-info .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-info .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-info .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-info .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-info .navbar-end .navbar-link.is-active{background-color:#023d64;color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-info .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-info .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-info .navbar-item.has-dropdown.is-active .navbar-link{background-color:#023d64;color:#fff}html.theme--documenter-dark .navbar.is-info .navbar-dropdown a.navbar-item.is-active{background-color:#024c7d;color:#fff}}html.theme--documenter-dark .navbar.is-success{background-color:#008438;color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-success .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-success .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-success .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-success .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-success .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-success .navbar-brand .navbar-link.is-active{background-color:#006b2d;color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-success .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-success .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-success .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-success .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-success .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-success .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-success .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-success .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-success .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-success .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-success .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-success .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-success .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-success .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-success .navbar-end .navbar-link.is-active{background-color:#006b2d;color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-success .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-success .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-success .navbar-item.has-dropdown.is-active .navbar-link{background-color:#006b2d;color:#fff}html.theme--documenter-dark .navbar.is-success .navbar-dropdown a.navbar-item.is-active{background-color:#008438;color:#fff}}html.theme--documenter-dark .navbar.is-warning{background-color:#ad8100;color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-warning .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-warning .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-warning .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-warning .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-warning .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-warning .navbar-brand .navbar-link.is-active{background-color:#946e00;color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-warning .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-warning .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-warning .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-warning .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-warning .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-warning .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-warning .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-warning .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-warning .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-warning .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-warning .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-warning .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-warning .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-warning .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-warning .navbar-end .navbar-link.is-active{background-color:#946e00;color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-warning .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-warning .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-warning .navbar-item.has-dropdown.is-active .navbar-link{background-color:#946e00;color:#fff}html.theme--documenter-dark .navbar.is-warning .navbar-dropdown a.navbar-item.is-active{background-color:#ad8100;color:#fff}}html.theme--documenter-dark .navbar.is-danger{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-brand>.navbar-item,html.theme--documenter-dark .navbar.is-danger .navbar-brand .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-brand>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-danger .navbar-brand>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-danger .navbar-brand>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-danger .navbar-brand .navbar-link:focus,html.theme--documenter-dark .navbar.is-danger .navbar-brand .navbar-link:hover,html.theme--documenter-dark .navbar.is-danger .navbar-brand .navbar-link.is-active{background-color:#86170b;color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-brand .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-burger{color:#fff}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar.is-danger .navbar-start>.navbar-item,html.theme--documenter-dark .navbar.is-danger .navbar-start .navbar-link,html.theme--documenter-dark .navbar.is-danger .navbar-end>.navbar-item,html.theme--documenter-dark .navbar.is-danger .navbar-end .navbar-link{color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-start>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-danger .navbar-start>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-danger .navbar-start>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-danger .navbar-start .navbar-link:focus,html.theme--documenter-dark .navbar.is-danger .navbar-start .navbar-link:hover,html.theme--documenter-dark .navbar.is-danger .navbar-start .navbar-link.is-active,html.theme--documenter-dark .navbar.is-danger .navbar-end>a.navbar-item:focus,html.theme--documenter-dark .navbar.is-danger .navbar-end>a.navbar-item:hover,html.theme--documenter-dark .navbar.is-danger .navbar-end>a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-danger .navbar-end .navbar-link:focus,html.theme--documenter-dark .navbar.is-danger .navbar-end .navbar-link:hover,html.theme--documenter-dark .navbar.is-danger .navbar-end .navbar-link.is-active{background-color:#86170b;color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-start .navbar-link::after,html.theme--documenter-dark .navbar.is-danger .navbar-end .navbar-link::after{border-color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar.is-danger .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar.is-danger .navbar-item.has-dropdown.is-active .navbar-link{background-color:#86170b;color:#fff}html.theme--documenter-dark .navbar.is-danger .navbar-dropdown a.navbar-item.is-active{background-color:#9e1b0d;color:#fff}}html.theme--documenter-dark .navbar>.container{align-items:stretch;display:flex;min-height:4rem;width:100%}html.theme--documenter-dark .navbar.has-shadow{box-shadow:0 2px 0 0 #282f2f}html.theme--documenter-dark .navbar.is-fixed-bottom,html.theme--documenter-dark .navbar.is-fixed-top{left:0;position:fixed;right:0;z-index:30}html.theme--documenter-dark .navbar.is-fixed-bottom{bottom:0}html.theme--documenter-dark .navbar.is-fixed-bottom.has-shadow{box-shadow:0 -2px 0 0 #282f2f}html.theme--documenter-dark .navbar.is-fixed-top{top:0}html.theme--documenter-dark html.has-navbar-fixed-top,html.theme--documenter-dark body.has-navbar-fixed-top{padding-top:4rem}html.theme--documenter-dark html.has-navbar-fixed-bottom,html.theme--documenter-dark body.has-navbar-fixed-bottom{padding-bottom:4rem}html.theme--documenter-dark .navbar-brand,html.theme--documenter-dark .navbar-tabs{align-items:stretch;display:flex;flex-shrink:0;min-height:4rem}html.theme--documenter-dark .navbar-brand a.navbar-item:focus,html.theme--documenter-dark .navbar-brand a.navbar-item:hover{background-color:transparent}html.theme--documenter-dark .navbar-tabs{-webkit-overflow-scrolling:touch;max-width:100vw;overflow-x:auto;overflow-y:hidden}html.theme--documenter-dark .navbar-burger{color:#fff;-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;cursor:pointer;display:block;height:4rem;position:relative;width:4rem;margin-left:auto}html.theme--documenter-dark .navbar-burger span{background-color:currentColor;display:block;height:1px;left:calc(50% - 8px);position:absolute;transform-origin:center;transition-duration:86ms;transition-property:background-color, opacity, transform;transition-timing-function:ease-out;width:16px}html.theme--documenter-dark .navbar-burger span:nth-child(1){top:calc(50% - 6px)}html.theme--documenter-dark .navbar-burger span:nth-child(2){top:calc(50% - 1px)}html.theme--documenter-dark .navbar-burger span:nth-child(3){top:calc(50% + 4px)}html.theme--documenter-dark .navbar-burger:hover{background-color:rgba(0,0,0,0.05)}html.theme--documenter-dark .navbar-burger.is-active span:nth-child(1){transform:translateY(5px) rotate(45deg)}html.theme--documenter-dark .navbar-burger.is-active span:nth-child(2){opacity:0}html.theme--documenter-dark .navbar-burger.is-active span:nth-child(3){transform:translateY(-5px) rotate(-45deg)}html.theme--documenter-dark .navbar-menu{display:none}html.theme--documenter-dark .navbar-item,html.theme--documenter-dark .navbar-link{color:#fff;display:block;line-height:1.5;padding:0.5rem 0.75rem;position:relative}html.theme--documenter-dark .navbar-item .icon:only-child,html.theme--documenter-dark .navbar-link .icon:only-child{margin-left:-0.25rem;margin-right:-0.25rem}html.theme--documenter-dark a.navbar-item,html.theme--documenter-dark .navbar-link{cursor:pointer}html.theme--documenter-dark a.navbar-item:focus,html.theme--documenter-dark a.navbar-item:focus-within,html.theme--documenter-dark a.navbar-item:hover,html.theme--documenter-dark a.navbar-item.is-active,html.theme--documenter-dark .navbar-link:focus,html.theme--documenter-dark .navbar-link:focus-within,html.theme--documenter-dark .navbar-link:hover,html.theme--documenter-dark .navbar-link.is-active{background-color:rgba(0,0,0,0);color:#1abc9c}html.theme--documenter-dark .navbar-item{flex-grow:0;flex-shrink:0}html.theme--documenter-dark .navbar-item img{max-height:1.75rem}html.theme--documenter-dark .navbar-item.has-dropdown{padding:0}html.theme--documenter-dark .navbar-item.is-expanded{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .navbar-item.is-tab{border-bottom:1px solid transparent;min-height:4rem;padding-bottom:calc(0.5rem - 1px)}html.theme--documenter-dark .navbar-item.is-tab:focus,html.theme--documenter-dark .navbar-item.is-tab:hover{background-color:rgba(0,0,0,0);border-bottom-color:#1abc9c}html.theme--documenter-dark .navbar-item.is-tab.is-active{background-color:rgba(0,0,0,0);border-bottom-color:#1abc9c;border-bottom-style:solid;border-bottom-width:3px;color:#1abc9c;padding-bottom:calc(0.5rem - 3px)}html.theme--documenter-dark .navbar-content{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .navbar-link:not(.is-arrowless){padding-right:2.5em}html.theme--documenter-dark .navbar-link:not(.is-arrowless)::after{border-color:#fff;margin-top:-0.375em;right:1.125em}html.theme--documenter-dark .navbar-dropdown{font-size:0.875rem;padding-bottom:0.5rem;padding-top:0.5rem}html.theme--documenter-dark .navbar-dropdown .navbar-item{padding-left:1.5rem;padding-right:1.5rem}html.theme--documenter-dark .navbar-divider{background-color:rgba(0,0,0,0.2);border:none;display:none;height:2px;margin:0.5rem 0}@media screen and (max-width: 1055px){html.theme--documenter-dark .navbar>.container{display:block}html.theme--documenter-dark .navbar-brand .navbar-item,html.theme--documenter-dark .navbar-tabs .navbar-item{align-items:center;display:flex}html.theme--documenter-dark .navbar-link::after{display:none}html.theme--documenter-dark .navbar-menu{background-color:#375a7f;box-shadow:0 8px 16px rgba(10,10,10,0.1);padding:0.5rem 0}html.theme--documenter-dark .navbar-menu.is-active{display:block}html.theme--documenter-dark .navbar.is-fixed-bottom-touch,html.theme--documenter-dark .navbar.is-fixed-top-touch{left:0;position:fixed;right:0;z-index:30}html.theme--documenter-dark .navbar.is-fixed-bottom-touch{bottom:0}html.theme--documenter-dark .navbar.is-fixed-bottom-touch.has-shadow{box-shadow:0 -2px 3px rgba(10,10,10,0.1)}html.theme--documenter-dark .navbar.is-fixed-top-touch{top:0}html.theme--documenter-dark .navbar.is-fixed-top .navbar-menu,html.theme--documenter-dark .navbar.is-fixed-top-touch .navbar-menu{-webkit-overflow-scrolling:touch;max-height:calc(100vh - 4rem);overflow:auto}html.theme--documenter-dark html.has-navbar-fixed-top-touch,html.theme--documenter-dark body.has-navbar-fixed-top-touch{padding-top:4rem}html.theme--documenter-dark html.has-navbar-fixed-bottom-touch,html.theme--documenter-dark body.has-navbar-fixed-bottom-touch{padding-bottom:4rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .navbar,html.theme--documenter-dark .navbar-menu,html.theme--documenter-dark .navbar-start,html.theme--documenter-dark .navbar-end{align-items:stretch;display:flex}html.theme--documenter-dark .navbar{min-height:4rem}html.theme--documenter-dark .navbar.is-spaced{padding:1rem 2rem}html.theme--documenter-dark .navbar.is-spaced .navbar-start,html.theme--documenter-dark .navbar.is-spaced .navbar-end{align-items:center}html.theme--documenter-dark .navbar.is-spaced a.navbar-item,html.theme--documenter-dark .navbar.is-spaced .navbar-link{border-radius:.4em}html.theme--documenter-dark .navbar.is-transparent a.navbar-item:focus,html.theme--documenter-dark .navbar.is-transparent a.navbar-item:hover,html.theme--documenter-dark .navbar.is-transparent a.navbar-item.is-active,html.theme--documenter-dark .navbar.is-transparent .navbar-link:focus,html.theme--documenter-dark .navbar.is-transparent .navbar-link:hover,html.theme--documenter-dark .navbar.is-transparent .navbar-link.is-active{background-color:transparent !important}html.theme--documenter-dark .navbar.is-transparent .navbar-item.has-dropdown.is-active .navbar-link,html.theme--documenter-dark .navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:focus .navbar-link,html.theme--documenter-dark .navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:focus-within .navbar-link,html.theme--documenter-dark .navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:hover .navbar-link{background-color:transparent !important}html.theme--documenter-dark .navbar.is-transparent .navbar-dropdown a.navbar-item:focus,html.theme--documenter-dark .navbar.is-transparent .navbar-dropdown a.navbar-item:hover{background-color:rgba(0,0,0,0);color:#dbdee0}html.theme--documenter-dark .navbar.is-transparent .navbar-dropdown a.navbar-item.is-active{background-color:rgba(0,0,0,0);color:#1abc9c}html.theme--documenter-dark .navbar-burger{display:none}html.theme--documenter-dark .navbar-item,html.theme--documenter-dark .navbar-link{align-items:center;display:flex}html.theme--documenter-dark .navbar-item.has-dropdown{align-items:stretch}html.theme--documenter-dark .navbar-item.has-dropdown-up .navbar-link::after{transform:rotate(135deg) translate(0.25em, -0.25em)}html.theme--documenter-dark .navbar-item.has-dropdown-up .navbar-dropdown{border-bottom:1px solid rgba(0,0,0,0.2);border-radius:8px 8px 0 0;border-top:none;bottom:100%;box-shadow:0 -8px 8px rgba(10,10,10,0.1);top:auto}html.theme--documenter-dark .navbar-item.is-active .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:focus .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:focus-within .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:hover .navbar-dropdown{display:block}.navbar.is-spaced html.theme--documenter-dark .navbar-item.is-active .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-active .navbar-dropdown.is-boxed,.navbar.is-spaced html.theme--documenter-dark .navbar-item.is-hoverable:focus .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:focus .navbar-dropdown.is-boxed,.navbar.is-spaced html.theme--documenter-dark .navbar-item.is-hoverable:focus-within .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:focus-within .navbar-dropdown.is-boxed,.navbar.is-spaced html.theme--documenter-dark .navbar-item.is-hoverable:hover .navbar-dropdown,html.theme--documenter-dark .navbar-item.is-hoverable:hover .navbar-dropdown.is-boxed{opacity:1;pointer-events:auto;transform:translateY(0)}html.theme--documenter-dark .navbar-menu{flex-grow:1;flex-shrink:0}html.theme--documenter-dark .navbar-start{justify-content:flex-start;margin-right:auto}html.theme--documenter-dark .navbar-end{justify-content:flex-end;margin-left:auto}html.theme--documenter-dark .navbar-dropdown{background-color:#375a7f;border-bottom-left-radius:8px;border-bottom-right-radius:8px;border-top:1px solid rgba(0,0,0,0.2);box-shadow:0 8px 8px rgba(10,10,10,0.1);display:none;font-size:0.875rem;left:0;min-width:100%;position:absolute;top:100%;z-index:20}html.theme--documenter-dark .navbar-dropdown .navbar-item{padding:0.375rem 1rem;white-space:nowrap}html.theme--documenter-dark .navbar-dropdown a.navbar-item{padding-right:3rem}html.theme--documenter-dark .navbar-dropdown a.navbar-item:focus,html.theme--documenter-dark .navbar-dropdown a.navbar-item:hover{background-color:rgba(0,0,0,0);color:#dbdee0}html.theme--documenter-dark .navbar-dropdown a.navbar-item.is-active{background-color:rgba(0,0,0,0);color:#1abc9c}.navbar.is-spaced html.theme--documenter-dark .navbar-dropdown,html.theme--documenter-dark .navbar-dropdown.is-boxed{border-radius:8px;border-top:none;box-shadow:0 8px 8px rgba(10,10,10,0.1), 0 0 0 1px rgba(10,10,10,0.1);display:block;opacity:0;pointer-events:none;top:calc(100% + (-4px));transform:translateY(-5px);transition-duration:86ms;transition-property:opacity, transform}html.theme--documenter-dark .navbar-dropdown.is-right{left:auto;right:0}html.theme--documenter-dark .navbar-divider{display:block}html.theme--documenter-dark .navbar>.container .navbar-brand,html.theme--documenter-dark .container>.navbar .navbar-brand{margin-left:-.75rem}html.theme--documenter-dark .navbar>.container .navbar-menu,html.theme--documenter-dark .container>.navbar .navbar-menu{margin-right:-.75rem}html.theme--documenter-dark .navbar.is-fixed-bottom-desktop,html.theme--documenter-dark .navbar.is-fixed-top-desktop{left:0;position:fixed;right:0;z-index:30}html.theme--documenter-dark .navbar.is-fixed-bottom-desktop{bottom:0}html.theme--documenter-dark .navbar.is-fixed-bottom-desktop.has-shadow{box-shadow:0 -2px 3px rgba(10,10,10,0.1)}html.theme--documenter-dark .navbar.is-fixed-top-desktop{top:0}html.theme--documenter-dark html.has-navbar-fixed-top-desktop,html.theme--documenter-dark body.has-navbar-fixed-top-desktop{padding-top:4rem}html.theme--documenter-dark html.has-navbar-fixed-bottom-desktop,html.theme--documenter-dark body.has-navbar-fixed-bottom-desktop{padding-bottom:4rem}html.theme--documenter-dark html.has-spaced-navbar-fixed-top,html.theme--documenter-dark body.has-spaced-navbar-fixed-top{padding-top:6rem}html.theme--documenter-dark html.has-spaced-navbar-fixed-bottom,html.theme--documenter-dark body.has-spaced-navbar-fixed-bottom{padding-bottom:6rem}html.theme--documenter-dark a.navbar-item.is-active,html.theme--documenter-dark .navbar-link.is-active{color:#1abc9c}html.theme--documenter-dark a.navbar-item.is-active:not(:focus):not(:hover),html.theme--documenter-dark .navbar-link.is-active:not(:focus):not(:hover){background-color:rgba(0,0,0,0)}html.theme--documenter-dark .navbar-item.has-dropdown:focus .navbar-link,html.theme--documenter-dark .navbar-item.has-dropdown:hover .navbar-link,html.theme--documenter-dark .navbar-item.has-dropdown.is-active .navbar-link{background-color:rgba(0,0,0,0)}}html.theme--documenter-dark .hero.is-fullheight-with-navbar{min-height:calc(100vh - 4rem)}html.theme--documenter-dark .pagination{font-size:1rem;margin:-.25rem}html.theme--documenter-dark .pagination.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.pagination{font-size:.75rem}html.theme--documenter-dark .pagination.is-medium{font-size:1.25rem}html.theme--documenter-dark .pagination.is-large{font-size:1.5rem}html.theme--documenter-dark .pagination.is-rounded .pagination-previous,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.pagination .pagination-previous,html.theme--documenter-dark .pagination.is-rounded .pagination-next,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.pagination .pagination-next{padding-left:1em;padding-right:1em;border-radius:9999px}html.theme--documenter-dark .pagination.is-rounded .pagination-link,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.pagination .pagination-link{border-radius:9999px}html.theme--documenter-dark .pagination,html.theme--documenter-dark .pagination-list{align-items:center;display:flex;justify-content:center;text-align:center}html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-ellipsis{font-size:1em;justify-content:center;margin:.25rem;padding-left:.5em;padding-right:.5em;text-align:center}html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-link{border-color:#5e6d6f;color:#1abc9c;min-width:2.5em}html.theme--documenter-dark .pagination-previous:hover,html.theme--documenter-dark .pagination-next:hover,html.theme--documenter-dark .pagination-link:hover{border-color:#8c9b9d;color:#1dd2af}html.theme--documenter-dark .pagination-previous:focus,html.theme--documenter-dark .pagination-next:focus,html.theme--documenter-dark .pagination-link:focus{border-color:#8c9b9d}html.theme--documenter-dark .pagination-previous:active,html.theme--documenter-dark .pagination-next:active,html.theme--documenter-dark .pagination-link:active{box-shadow:inset 0 1px 2px rgba(10,10,10,0.2)}html.theme--documenter-dark .pagination-previous[disabled],html.theme--documenter-dark .pagination-previous.is-disabled,html.theme--documenter-dark .pagination-next[disabled],html.theme--documenter-dark .pagination-next.is-disabled,html.theme--documenter-dark .pagination-link[disabled],html.theme--documenter-dark .pagination-link.is-disabled{background-color:#5e6d6f;border-color:#5e6d6f;box-shadow:none;color:#fff;opacity:0.5}html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next{padding-left:.75em;padding-right:.75em;white-space:nowrap}html.theme--documenter-dark .pagination-link.is-current{background-color:#1abc9c;border-color:#1abc9c;color:#fff}html.theme--documenter-dark .pagination-ellipsis{color:#8c9b9d;pointer-events:none}html.theme--documenter-dark .pagination-list{flex-wrap:wrap}html.theme--documenter-dark .pagination-list li{list-style:none}@media screen and (max-width: 768px){html.theme--documenter-dark .pagination{flex-wrap:wrap}html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .pagination-list li{flex-grow:1;flex-shrink:1}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .pagination-list{flex-grow:1;flex-shrink:1;justify-content:flex-start;order:1}html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-ellipsis{margin-bottom:0;margin-top:0}html.theme--documenter-dark .pagination-previous{order:2}html.theme--documenter-dark .pagination-next{order:3}html.theme--documenter-dark .pagination{justify-content:space-between;margin-bottom:0;margin-top:0}html.theme--documenter-dark .pagination.is-centered .pagination-previous{order:1}html.theme--documenter-dark .pagination.is-centered .pagination-list{justify-content:center;order:2}html.theme--documenter-dark .pagination.is-centered .pagination-next{order:3}html.theme--documenter-dark .pagination.is-right .pagination-previous{order:1}html.theme--documenter-dark .pagination.is-right .pagination-next{order:2}html.theme--documenter-dark .pagination.is-right .pagination-list{justify-content:flex-end;order:3}}html.theme--documenter-dark .panel{border-radius:8px;box-shadow:#171717;font-size:1rem}html.theme--documenter-dark .panel:not(:last-child){margin-bottom:1.5rem}html.theme--documenter-dark .panel.is-white .panel-heading{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .panel.is-white .panel-tabs a.is-active{border-bottom-color:#fff}html.theme--documenter-dark .panel.is-white .panel-block.is-active .panel-icon{color:#fff}html.theme--documenter-dark .panel.is-black .panel-heading{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .panel.is-black .panel-tabs a.is-active{border-bottom-color:#0a0a0a}html.theme--documenter-dark .panel.is-black .panel-block.is-active .panel-icon{color:#0a0a0a}html.theme--documenter-dark .panel.is-light .panel-heading{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .panel.is-light .panel-tabs a.is-active{border-bottom-color:#ecf0f1}html.theme--documenter-dark .panel.is-light .panel-block.is-active .panel-icon{color:#ecf0f1}html.theme--documenter-dark .panel.is-dark .panel-heading,html.theme--documenter-dark .content kbd.panel .panel-heading{background-color:#282f2f;color:#fff}html.theme--documenter-dark .panel.is-dark .panel-tabs a.is-active,html.theme--documenter-dark .content kbd.panel .panel-tabs a.is-active{border-bottom-color:#282f2f}html.theme--documenter-dark .panel.is-dark .panel-block.is-active .panel-icon,html.theme--documenter-dark .content kbd.panel .panel-block.is-active .panel-icon{color:#282f2f}html.theme--documenter-dark .panel.is-primary .panel-heading,html.theme--documenter-dark .docstring>section>a.panel.docs-sourcelink .panel-heading{background-color:#375a7f;color:#fff}html.theme--documenter-dark .panel.is-primary .panel-tabs a.is-active,html.theme--documenter-dark .docstring>section>a.panel.docs-sourcelink .panel-tabs a.is-active{border-bottom-color:#375a7f}html.theme--documenter-dark .panel.is-primary .panel-block.is-active .panel-icon,html.theme--documenter-dark .docstring>section>a.panel.docs-sourcelink .panel-block.is-active .panel-icon{color:#375a7f}html.theme--documenter-dark .panel.is-link .panel-heading{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .panel.is-link .panel-tabs a.is-active{border-bottom-color:#1abc9c}html.theme--documenter-dark .panel.is-link .panel-block.is-active .panel-icon{color:#1abc9c}html.theme--documenter-dark .panel.is-info .panel-heading{background-color:#024c7d;color:#fff}html.theme--documenter-dark .panel.is-info .panel-tabs a.is-active{border-bottom-color:#024c7d}html.theme--documenter-dark .panel.is-info .panel-block.is-active .panel-icon{color:#024c7d}html.theme--documenter-dark .panel.is-success .panel-heading{background-color:#008438;color:#fff}html.theme--documenter-dark .panel.is-success .panel-tabs a.is-active{border-bottom-color:#008438}html.theme--documenter-dark .panel.is-success .panel-block.is-active .panel-icon{color:#008438}html.theme--documenter-dark .panel.is-warning .panel-heading{background-color:#ad8100;color:#fff}html.theme--documenter-dark .panel.is-warning .panel-tabs a.is-active{border-bottom-color:#ad8100}html.theme--documenter-dark .panel.is-warning .panel-block.is-active .panel-icon{color:#ad8100}html.theme--documenter-dark .panel.is-danger .panel-heading{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .panel.is-danger .panel-tabs a.is-active{border-bottom-color:#9e1b0d}html.theme--documenter-dark .panel.is-danger .panel-block.is-active .panel-icon{color:#9e1b0d}html.theme--documenter-dark .panel-tabs:not(:last-child),html.theme--documenter-dark .panel-block:not(:last-child){border-bottom:1px solid #ededed}html.theme--documenter-dark .panel-heading{background-color:#343c3d;border-radius:8px 8px 0 0;color:#f2f2f2;font-size:1.25em;font-weight:700;line-height:1.25;padding:0.75em 1em}html.theme--documenter-dark .panel-tabs{align-items:flex-end;display:flex;font-size:.875em;justify-content:center}html.theme--documenter-dark .panel-tabs a{border-bottom:1px solid #5e6d6f;margin-bottom:-1px;padding:0.5em}html.theme--documenter-dark .panel-tabs a.is-active{border-bottom-color:#343c3d;color:#17a689}html.theme--documenter-dark .panel-list a{color:#fff}html.theme--documenter-dark .panel-list a:hover{color:#1abc9c}html.theme--documenter-dark .panel-block{align-items:center;color:#f2f2f2;display:flex;justify-content:flex-start;padding:0.5em 0.75em}html.theme--documenter-dark .panel-block input[type="checkbox"]{margin-right:.75em}html.theme--documenter-dark .panel-block>.control{flex-grow:1;flex-shrink:1;width:100%}html.theme--documenter-dark .panel-block.is-wrapped{flex-wrap:wrap}html.theme--documenter-dark .panel-block.is-active{border-left-color:#1abc9c;color:#17a689}html.theme--documenter-dark .panel-block.is-active .panel-icon{color:#1abc9c}html.theme--documenter-dark .panel-block:last-child{border-bottom-left-radius:8px;border-bottom-right-radius:8px}html.theme--documenter-dark a.panel-block,html.theme--documenter-dark label.panel-block{cursor:pointer}html.theme--documenter-dark a.panel-block:hover,html.theme--documenter-dark label.panel-block:hover{background-color:#282f2f}html.theme--documenter-dark .panel-icon{display:inline-block;font-size:14px;height:1em;line-height:1em;text-align:center;vertical-align:top;width:1em;color:#fff;margin-right:.75em}html.theme--documenter-dark .panel-icon .fa{font-size:inherit;line-height:inherit}html.theme--documenter-dark .tabs{-webkit-overflow-scrolling:touch;align-items:stretch;display:flex;font-size:1rem;justify-content:space-between;overflow:hidden;overflow-x:auto;white-space:nowrap}html.theme--documenter-dark .tabs a{align-items:center;border-bottom-color:#5e6d6f;border-bottom-style:solid;border-bottom-width:1px;color:#fff;display:flex;justify-content:center;margin-bottom:-1px;padding:0.5em 1em;vertical-align:top}html.theme--documenter-dark .tabs a:hover{border-bottom-color:#f2f2f2;color:#f2f2f2}html.theme--documenter-dark .tabs li{display:block}html.theme--documenter-dark .tabs li.is-active a{border-bottom-color:#1abc9c;color:#1abc9c}html.theme--documenter-dark .tabs ul{align-items:center;border-bottom-color:#5e6d6f;border-bottom-style:solid;border-bottom-width:1px;display:flex;flex-grow:1;flex-shrink:0;justify-content:flex-start}html.theme--documenter-dark .tabs ul.is-left{padding-right:0.75em}html.theme--documenter-dark .tabs ul.is-center{flex:none;justify-content:center;padding-left:0.75em;padding-right:0.75em}html.theme--documenter-dark .tabs ul.is-right{justify-content:flex-end;padding-left:0.75em}html.theme--documenter-dark .tabs .icon:first-child{margin-right:.5em}html.theme--documenter-dark .tabs .icon:last-child{margin-left:.5em}html.theme--documenter-dark .tabs.is-centered ul{justify-content:center}html.theme--documenter-dark .tabs.is-right ul{justify-content:flex-end}html.theme--documenter-dark .tabs.is-boxed a{border:1px solid transparent;border-radius:.4em .4em 0 0}html.theme--documenter-dark .tabs.is-boxed a:hover{background-color:#282f2f;border-bottom-color:#5e6d6f}html.theme--documenter-dark .tabs.is-boxed li.is-active a{background-color:#fff;border-color:#5e6d6f;border-bottom-color:rgba(0,0,0,0) !important}html.theme--documenter-dark .tabs.is-fullwidth li{flex-grow:1;flex-shrink:0}html.theme--documenter-dark .tabs.is-toggle a{border-color:#5e6d6f;border-style:solid;border-width:1px;margin-bottom:0;position:relative}html.theme--documenter-dark .tabs.is-toggle a:hover{background-color:#282f2f;border-color:#8c9b9d;z-index:2}html.theme--documenter-dark .tabs.is-toggle li+li{margin-left:-1px}html.theme--documenter-dark .tabs.is-toggle li:first-child a{border-top-left-radius:.4em;border-bottom-left-radius:.4em}html.theme--documenter-dark .tabs.is-toggle li:last-child a{border-top-right-radius:.4em;border-bottom-right-radius:.4em}html.theme--documenter-dark .tabs.is-toggle li.is-active a{background-color:#1abc9c;border-color:#1abc9c;color:#fff;z-index:1}html.theme--documenter-dark .tabs.is-toggle ul{border-bottom:none}html.theme--documenter-dark .tabs.is-toggle.is-toggle-rounded li:first-child a{border-bottom-left-radius:9999px;border-top-left-radius:9999px;padding-left:1.25em}html.theme--documenter-dark .tabs.is-toggle.is-toggle-rounded li:last-child a{border-bottom-right-radius:9999px;border-top-right-radius:9999px;padding-right:1.25em}html.theme--documenter-dark .tabs.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.tabs{font-size:.75rem}html.theme--documenter-dark .tabs.is-medium{font-size:1.25rem}html.theme--documenter-dark .tabs.is-large{font-size:1.5rem}html.theme--documenter-dark .column{display:block;flex-basis:0;flex-grow:1;flex-shrink:1;padding:.75rem}.columns.is-mobile>html.theme--documenter-dark .column.is-narrow{flex:none;width:unset}.columns.is-mobile>html.theme--documenter-dark .column.is-full{flex:none;width:100%}.columns.is-mobile>html.theme--documenter-dark .column.is-three-quarters{flex:none;width:75%}.columns.is-mobile>html.theme--documenter-dark .column.is-two-thirds{flex:none;width:66.6666%}.columns.is-mobile>html.theme--documenter-dark .column.is-half{flex:none;width:50%}.columns.is-mobile>html.theme--documenter-dark .column.is-one-third{flex:none;width:33.3333%}.columns.is-mobile>html.theme--documenter-dark .column.is-one-quarter{flex:none;width:25%}.columns.is-mobile>html.theme--documenter-dark .column.is-one-fifth{flex:none;width:20%}.columns.is-mobile>html.theme--documenter-dark .column.is-two-fifths{flex:none;width:40%}.columns.is-mobile>html.theme--documenter-dark .column.is-three-fifths{flex:none;width:60%}.columns.is-mobile>html.theme--documenter-dark .column.is-four-fifths{flex:none;width:80%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-three-quarters{margin-left:75%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-two-thirds{margin-left:66.6666%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-half{margin-left:50%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-one-third{margin-left:33.3333%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-one-quarter{margin-left:25%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-one-fifth{margin-left:20%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-two-fifths{margin-left:40%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-three-fifths{margin-left:60%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-four-fifths{margin-left:80%}.columns.is-mobile>html.theme--documenter-dark .column.is-0{flex:none;width:0%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-0{margin-left:0%}.columns.is-mobile>html.theme--documenter-dark .column.is-1{flex:none;width:8.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-1{margin-left:8.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-2{flex:none;width:16.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-2{margin-left:16.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-3{flex:none;width:25%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-3{margin-left:25%}.columns.is-mobile>html.theme--documenter-dark .column.is-4{flex:none;width:33.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-4{margin-left:33.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-5{flex:none;width:41.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-5{margin-left:41.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-6{flex:none;width:50%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-6{margin-left:50%}.columns.is-mobile>html.theme--documenter-dark .column.is-7{flex:none;width:58.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-7{margin-left:58.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-8{flex:none;width:66.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-8{margin-left:66.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-9{flex:none;width:75%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-9{margin-left:75%}.columns.is-mobile>html.theme--documenter-dark .column.is-10{flex:none;width:83.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-10{margin-left:83.33333337%}.columns.is-mobile>html.theme--documenter-dark .column.is-11{flex:none;width:91.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-11{margin-left:91.66666674%}.columns.is-mobile>html.theme--documenter-dark .column.is-12{flex:none;width:100%}.columns.is-mobile>html.theme--documenter-dark .column.is-offset-12{margin-left:100%}@media screen and (max-width: 768px){html.theme--documenter-dark .column.is-narrow-mobile{flex:none;width:unset}html.theme--documenter-dark .column.is-full-mobile{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters-mobile{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds-mobile{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half-mobile{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third-mobile{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter-mobile{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth-mobile{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths-mobile{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths-mobile{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths-mobile{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters-mobile{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds-mobile{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half-mobile{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third-mobile{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter-mobile{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth-mobile{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths-mobile{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths-mobile{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths-mobile{margin-left:80%}html.theme--documenter-dark .column.is-0-mobile{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0-mobile{margin-left:0%}html.theme--documenter-dark .column.is-1-mobile{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1-mobile{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2-mobile{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2-mobile{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3-mobile{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3-mobile{margin-left:25%}html.theme--documenter-dark .column.is-4-mobile{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4-mobile{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5-mobile{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5-mobile{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6-mobile{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6-mobile{margin-left:50%}html.theme--documenter-dark .column.is-7-mobile{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7-mobile{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8-mobile{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8-mobile{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9-mobile{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9-mobile{margin-left:75%}html.theme--documenter-dark .column.is-10-mobile{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10-mobile{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11-mobile{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11-mobile{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12-mobile{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12-mobile{margin-left:100%}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .column.is-narrow,html.theme--documenter-dark .column.is-narrow-tablet{flex:none;width:unset}html.theme--documenter-dark .column.is-full,html.theme--documenter-dark .column.is-full-tablet{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters,html.theme--documenter-dark .column.is-three-quarters-tablet{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds,html.theme--documenter-dark .column.is-two-thirds-tablet{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half,html.theme--documenter-dark .column.is-half-tablet{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third,html.theme--documenter-dark .column.is-one-third-tablet{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter,html.theme--documenter-dark .column.is-one-quarter-tablet{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth,html.theme--documenter-dark .column.is-one-fifth-tablet{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths,html.theme--documenter-dark .column.is-two-fifths-tablet{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths,html.theme--documenter-dark .column.is-three-fifths-tablet{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths,html.theme--documenter-dark .column.is-four-fifths-tablet{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters,html.theme--documenter-dark .column.is-offset-three-quarters-tablet{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds,html.theme--documenter-dark .column.is-offset-two-thirds-tablet{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half,html.theme--documenter-dark .column.is-offset-half-tablet{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third,html.theme--documenter-dark .column.is-offset-one-third-tablet{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter,html.theme--documenter-dark .column.is-offset-one-quarter-tablet{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth,html.theme--documenter-dark .column.is-offset-one-fifth-tablet{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths,html.theme--documenter-dark .column.is-offset-two-fifths-tablet{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths,html.theme--documenter-dark .column.is-offset-three-fifths-tablet{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths,html.theme--documenter-dark .column.is-offset-four-fifths-tablet{margin-left:80%}html.theme--documenter-dark .column.is-0,html.theme--documenter-dark .column.is-0-tablet{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0,html.theme--documenter-dark .column.is-offset-0-tablet{margin-left:0%}html.theme--documenter-dark .column.is-1,html.theme--documenter-dark .column.is-1-tablet{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1,html.theme--documenter-dark .column.is-offset-1-tablet{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2,html.theme--documenter-dark .column.is-2-tablet{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2,html.theme--documenter-dark .column.is-offset-2-tablet{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3,html.theme--documenter-dark .column.is-3-tablet{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3,html.theme--documenter-dark .column.is-offset-3-tablet{margin-left:25%}html.theme--documenter-dark .column.is-4,html.theme--documenter-dark .column.is-4-tablet{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4,html.theme--documenter-dark .column.is-offset-4-tablet{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5,html.theme--documenter-dark .column.is-5-tablet{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5,html.theme--documenter-dark .column.is-offset-5-tablet{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6,html.theme--documenter-dark .column.is-6-tablet{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6,html.theme--documenter-dark .column.is-offset-6-tablet{margin-left:50%}html.theme--documenter-dark .column.is-7,html.theme--documenter-dark .column.is-7-tablet{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7,html.theme--documenter-dark .column.is-offset-7-tablet{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8,html.theme--documenter-dark .column.is-8-tablet{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8,html.theme--documenter-dark .column.is-offset-8-tablet{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9,html.theme--documenter-dark .column.is-9-tablet{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9,html.theme--documenter-dark .column.is-offset-9-tablet{margin-left:75%}html.theme--documenter-dark .column.is-10,html.theme--documenter-dark .column.is-10-tablet{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10,html.theme--documenter-dark .column.is-offset-10-tablet{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11,html.theme--documenter-dark .column.is-11-tablet{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11,html.theme--documenter-dark .column.is-offset-11-tablet{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12,html.theme--documenter-dark .column.is-12-tablet{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12,html.theme--documenter-dark .column.is-offset-12-tablet{margin-left:100%}}@media screen and (max-width: 1055px){html.theme--documenter-dark .column.is-narrow-touch{flex:none;width:unset}html.theme--documenter-dark .column.is-full-touch{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters-touch{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds-touch{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half-touch{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third-touch{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter-touch{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth-touch{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths-touch{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths-touch{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths-touch{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters-touch{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds-touch{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half-touch{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third-touch{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter-touch{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth-touch{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths-touch{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths-touch{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths-touch{margin-left:80%}html.theme--documenter-dark .column.is-0-touch{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0-touch{margin-left:0%}html.theme--documenter-dark .column.is-1-touch{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1-touch{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2-touch{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2-touch{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3-touch{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3-touch{margin-left:25%}html.theme--documenter-dark .column.is-4-touch{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4-touch{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5-touch{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5-touch{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6-touch{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6-touch{margin-left:50%}html.theme--documenter-dark .column.is-7-touch{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7-touch{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8-touch{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8-touch{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9-touch{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9-touch{margin-left:75%}html.theme--documenter-dark .column.is-10-touch{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10-touch{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11-touch{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11-touch{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12-touch{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12-touch{margin-left:100%}}@media screen and (min-width: 1056px){html.theme--documenter-dark .column.is-narrow-desktop{flex:none;width:unset}html.theme--documenter-dark .column.is-full-desktop{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters-desktop{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds-desktop{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half-desktop{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third-desktop{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter-desktop{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth-desktop{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths-desktop{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths-desktop{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths-desktop{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters-desktop{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds-desktop{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half-desktop{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third-desktop{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter-desktop{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth-desktop{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths-desktop{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths-desktop{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths-desktop{margin-left:80%}html.theme--documenter-dark .column.is-0-desktop{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0-desktop{margin-left:0%}html.theme--documenter-dark .column.is-1-desktop{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1-desktop{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2-desktop{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2-desktop{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3-desktop{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3-desktop{margin-left:25%}html.theme--documenter-dark .column.is-4-desktop{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4-desktop{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5-desktop{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5-desktop{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6-desktop{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6-desktop{margin-left:50%}html.theme--documenter-dark .column.is-7-desktop{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7-desktop{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8-desktop{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8-desktop{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9-desktop{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9-desktop{margin-left:75%}html.theme--documenter-dark .column.is-10-desktop{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10-desktop{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11-desktop{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11-desktop{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12-desktop{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12-desktop{margin-left:100%}}@media screen and (min-width: 1216px){html.theme--documenter-dark .column.is-narrow-widescreen{flex:none;width:unset}html.theme--documenter-dark .column.is-full-widescreen{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters-widescreen{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds-widescreen{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half-widescreen{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third-widescreen{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter-widescreen{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth-widescreen{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths-widescreen{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths-widescreen{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths-widescreen{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters-widescreen{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds-widescreen{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half-widescreen{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third-widescreen{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter-widescreen{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth-widescreen{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths-widescreen{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths-widescreen{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths-widescreen{margin-left:80%}html.theme--documenter-dark .column.is-0-widescreen{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0-widescreen{margin-left:0%}html.theme--documenter-dark .column.is-1-widescreen{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1-widescreen{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2-widescreen{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2-widescreen{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3-widescreen{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3-widescreen{margin-left:25%}html.theme--documenter-dark .column.is-4-widescreen{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4-widescreen{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5-widescreen{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5-widescreen{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6-widescreen{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6-widescreen{margin-left:50%}html.theme--documenter-dark .column.is-7-widescreen{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7-widescreen{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8-widescreen{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8-widescreen{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9-widescreen{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9-widescreen{margin-left:75%}html.theme--documenter-dark .column.is-10-widescreen{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10-widescreen{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11-widescreen{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11-widescreen{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12-widescreen{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12-widescreen{margin-left:100%}}@media screen and (min-width: 1408px){html.theme--documenter-dark .column.is-narrow-fullhd{flex:none;width:unset}html.theme--documenter-dark .column.is-full-fullhd{flex:none;width:100%}html.theme--documenter-dark .column.is-three-quarters-fullhd{flex:none;width:75%}html.theme--documenter-dark .column.is-two-thirds-fullhd{flex:none;width:66.6666%}html.theme--documenter-dark .column.is-half-fullhd{flex:none;width:50%}html.theme--documenter-dark .column.is-one-third-fullhd{flex:none;width:33.3333%}html.theme--documenter-dark .column.is-one-quarter-fullhd{flex:none;width:25%}html.theme--documenter-dark .column.is-one-fifth-fullhd{flex:none;width:20%}html.theme--documenter-dark .column.is-two-fifths-fullhd{flex:none;width:40%}html.theme--documenter-dark .column.is-three-fifths-fullhd{flex:none;width:60%}html.theme--documenter-dark .column.is-four-fifths-fullhd{flex:none;width:80%}html.theme--documenter-dark .column.is-offset-three-quarters-fullhd{margin-left:75%}html.theme--documenter-dark .column.is-offset-two-thirds-fullhd{margin-left:66.6666%}html.theme--documenter-dark .column.is-offset-half-fullhd{margin-left:50%}html.theme--documenter-dark .column.is-offset-one-third-fullhd{margin-left:33.3333%}html.theme--documenter-dark .column.is-offset-one-quarter-fullhd{margin-left:25%}html.theme--documenter-dark .column.is-offset-one-fifth-fullhd{margin-left:20%}html.theme--documenter-dark .column.is-offset-two-fifths-fullhd{margin-left:40%}html.theme--documenter-dark .column.is-offset-three-fifths-fullhd{margin-left:60%}html.theme--documenter-dark .column.is-offset-four-fifths-fullhd{margin-left:80%}html.theme--documenter-dark .column.is-0-fullhd{flex:none;width:0%}html.theme--documenter-dark .column.is-offset-0-fullhd{margin-left:0%}html.theme--documenter-dark .column.is-1-fullhd{flex:none;width:8.33333337%}html.theme--documenter-dark .column.is-offset-1-fullhd{margin-left:8.33333337%}html.theme--documenter-dark .column.is-2-fullhd{flex:none;width:16.66666674%}html.theme--documenter-dark .column.is-offset-2-fullhd{margin-left:16.66666674%}html.theme--documenter-dark .column.is-3-fullhd{flex:none;width:25%}html.theme--documenter-dark .column.is-offset-3-fullhd{margin-left:25%}html.theme--documenter-dark .column.is-4-fullhd{flex:none;width:33.33333337%}html.theme--documenter-dark .column.is-offset-4-fullhd{margin-left:33.33333337%}html.theme--documenter-dark .column.is-5-fullhd{flex:none;width:41.66666674%}html.theme--documenter-dark .column.is-offset-5-fullhd{margin-left:41.66666674%}html.theme--documenter-dark .column.is-6-fullhd{flex:none;width:50%}html.theme--documenter-dark .column.is-offset-6-fullhd{margin-left:50%}html.theme--documenter-dark .column.is-7-fullhd{flex:none;width:58.33333337%}html.theme--documenter-dark .column.is-offset-7-fullhd{margin-left:58.33333337%}html.theme--documenter-dark .column.is-8-fullhd{flex:none;width:66.66666674%}html.theme--documenter-dark .column.is-offset-8-fullhd{margin-left:66.66666674%}html.theme--documenter-dark .column.is-9-fullhd{flex:none;width:75%}html.theme--documenter-dark .column.is-offset-9-fullhd{margin-left:75%}html.theme--documenter-dark .column.is-10-fullhd{flex:none;width:83.33333337%}html.theme--documenter-dark .column.is-offset-10-fullhd{margin-left:83.33333337%}html.theme--documenter-dark .column.is-11-fullhd{flex:none;width:91.66666674%}html.theme--documenter-dark .column.is-offset-11-fullhd{margin-left:91.66666674%}html.theme--documenter-dark .column.is-12-fullhd{flex:none;width:100%}html.theme--documenter-dark .column.is-offset-12-fullhd{margin-left:100%}}html.theme--documenter-dark .columns{margin-left:-.75rem;margin-right:-.75rem;margin-top:-.75rem}html.theme--documenter-dark .columns:last-child{margin-bottom:-.75rem}html.theme--documenter-dark .columns:not(:last-child){margin-bottom:calc(1.5rem - .75rem)}html.theme--documenter-dark .columns.is-centered{justify-content:center}html.theme--documenter-dark .columns.is-gapless{margin-left:0;margin-right:0;margin-top:0}html.theme--documenter-dark .columns.is-gapless>.column{margin:0;padding:0 !important}html.theme--documenter-dark .columns.is-gapless:not(:last-child){margin-bottom:1.5rem}html.theme--documenter-dark .columns.is-gapless:last-child{margin-bottom:0}html.theme--documenter-dark .columns.is-mobile{display:flex}html.theme--documenter-dark .columns.is-multiline{flex-wrap:wrap}html.theme--documenter-dark .columns.is-vcentered{align-items:center}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns:not(.is-desktop){display:flex}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-desktop{display:flex}}html.theme--documenter-dark .columns.is-variable{--columnGap: 0.75rem;margin-left:calc(-1 * var(--columnGap));margin-right:calc(-1 * var(--columnGap))}html.theme--documenter-dark .columns.is-variable>.column{padding-left:var(--columnGap);padding-right:var(--columnGap)}html.theme--documenter-dark .columns.is-variable.is-0{--columnGap: 0rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-0-mobile{--columnGap: 0rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-0-tablet{--columnGap: 0rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-0-tablet-only{--columnGap: 0rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-0-touch{--columnGap: 0rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-0-desktop{--columnGap: 0rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-0-desktop-only{--columnGap: 0rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-0-widescreen{--columnGap: 0rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-0-widescreen-only{--columnGap: 0rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-0-fullhd{--columnGap: 0rem}}html.theme--documenter-dark .columns.is-variable.is-1{--columnGap: .25rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-1-mobile{--columnGap: .25rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-1-tablet{--columnGap: .25rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-1-tablet-only{--columnGap: .25rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-1-touch{--columnGap: .25rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-1-desktop{--columnGap: .25rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-1-desktop-only{--columnGap: .25rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-1-widescreen{--columnGap: .25rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-1-widescreen-only{--columnGap: .25rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-1-fullhd{--columnGap: .25rem}}html.theme--documenter-dark .columns.is-variable.is-2{--columnGap: .5rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-2-mobile{--columnGap: .5rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-2-tablet{--columnGap: .5rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-2-tablet-only{--columnGap: .5rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-2-touch{--columnGap: .5rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-2-desktop{--columnGap: .5rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-2-desktop-only{--columnGap: .5rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-2-widescreen{--columnGap: .5rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-2-widescreen-only{--columnGap: .5rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-2-fullhd{--columnGap: .5rem}}html.theme--documenter-dark .columns.is-variable.is-3{--columnGap: .75rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-3-mobile{--columnGap: .75rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-3-tablet{--columnGap: .75rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-3-tablet-only{--columnGap: .75rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-3-touch{--columnGap: .75rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-3-desktop{--columnGap: .75rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-3-desktop-only{--columnGap: .75rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-3-widescreen{--columnGap: .75rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-3-widescreen-only{--columnGap: .75rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-3-fullhd{--columnGap: .75rem}}html.theme--documenter-dark .columns.is-variable.is-4{--columnGap: 1rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-4-mobile{--columnGap: 1rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-4-tablet{--columnGap: 1rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-4-tablet-only{--columnGap: 1rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-4-touch{--columnGap: 1rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-4-desktop{--columnGap: 1rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-4-desktop-only{--columnGap: 1rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-4-widescreen{--columnGap: 1rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-4-widescreen-only{--columnGap: 1rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-4-fullhd{--columnGap: 1rem}}html.theme--documenter-dark .columns.is-variable.is-5{--columnGap: 1.25rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-5-mobile{--columnGap: 1.25rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-5-tablet{--columnGap: 1.25rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-5-tablet-only{--columnGap: 1.25rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-5-touch{--columnGap: 1.25rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-5-desktop{--columnGap: 1.25rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-5-desktop-only{--columnGap: 1.25rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-5-widescreen{--columnGap: 1.25rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-5-widescreen-only{--columnGap: 1.25rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-5-fullhd{--columnGap: 1.25rem}}html.theme--documenter-dark .columns.is-variable.is-6{--columnGap: 1.5rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-6-mobile{--columnGap: 1.5rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-6-tablet{--columnGap: 1.5rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-6-tablet-only{--columnGap: 1.5rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-6-touch{--columnGap: 1.5rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-6-desktop{--columnGap: 1.5rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-6-desktop-only{--columnGap: 1.5rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-6-widescreen{--columnGap: 1.5rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-6-widescreen-only{--columnGap: 1.5rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-6-fullhd{--columnGap: 1.5rem}}html.theme--documenter-dark .columns.is-variable.is-7{--columnGap: 1.75rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-7-mobile{--columnGap: 1.75rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-7-tablet{--columnGap: 1.75rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-7-tablet-only{--columnGap: 1.75rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-7-touch{--columnGap: 1.75rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-7-desktop{--columnGap: 1.75rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-7-desktop-only{--columnGap: 1.75rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-7-widescreen{--columnGap: 1.75rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-7-widescreen-only{--columnGap: 1.75rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-7-fullhd{--columnGap: 1.75rem}}html.theme--documenter-dark .columns.is-variable.is-8{--columnGap: 2rem}@media screen and (max-width: 768px){html.theme--documenter-dark .columns.is-variable.is-8-mobile{--columnGap: 2rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .columns.is-variable.is-8-tablet{--columnGap: 2rem}}@media screen and (min-width: 769px) and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-8-tablet-only{--columnGap: 2rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark .columns.is-variable.is-8-touch{--columnGap: 2rem}}@media screen and (min-width: 1056px){html.theme--documenter-dark .columns.is-variable.is-8-desktop{--columnGap: 2rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){html.theme--documenter-dark .columns.is-variable.is-8-desktop-only{--columnGap: 2rem}}@media screen and (min-width: 1216px){html.theme--documenter-dark .columns.is-variable.is-8-widescreen{--columnGap: 2rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){html.theme--documenter-dark .columns.is-variable.is-8-widescreen-only{--columnGap: 2rem}}@media screen and (min-width: 1408px){html.theme--documenter-dark .columns.is-variable.is-8-fullhd{--columnGap: 2rem}}html.theme--documenter-dark .tile{align-items:stretch;display:block;flex-basis:0;flex-grow:1;flex-shrink:1;min-height:min-content}html.theme--documenter-dark .tile.is-ancestor{margin-left:-.75rem;margin-right:-.75rem;margin-top:-.75rem}html.theme--documenter-dark .tile.is-ancestor:last-child{margin-bottom:-.75rem}html.theme--documenter-dark .tile.is-ancestor:not(:last-child){margin-bottom:.75rem}html.theme--documenter-dark .tile.is-child{margin:0 !important}html.theme--documenter-dark .tile.is-parent{padding:.75rem}html.theme--documenter-dark .tile.is-vertical{flex-direction:column}html.theme--documenter-dark .tile.is-vertical>.tile.is-child:not(:last-child){margin-bottom:1.5rem !important}@media screen and (min-width: 769px),print{html.theme--documenter-dark .tile:not(.is-child){display:flex}html.theme--documenter-dark .tile.is-1{flex:none;width:8.33333337%}html.theme--documenter-dark .tile.is-2{flex:none;width:16.66666674%}html.theme--documenter-dark .tile.is-3{flex:none;width:25%}html.theme--documenter-dark .tile.is-4{flex:none;width:33.33333337%}html.theme--documenter-dark .tile.is-5{flex:none;width:41.66666674%}html.theme--documenter-dark .tile.is-6{flex:none;width:50%}html.theme--documenter-dark .tile.is-7{flex:none;width:58.33333337%}html.theme--documenter-dark .tile.is-8{flex:none;width:66.66666674%}html.theme--documenter-dark .tile.is-9{flex:none;width:75%}html.theme--documenter-dark .tile.is-10{flex:none;width:83.33333337%}html.theme--documenter-dark .tile.is-11{flex:none;width:91.66666674%}html.theme--documenter-dark .tile.is-12{flex:none;width:100%}}html.theme--documenter-dark .hero{align-items:stretch;display:flex;flex-direction:column;justify-content:space-between}html.theme--documenter-dark .hero .navbar{background:none}html.theme--documenter-dark .hero .tabs ul{border-bottom:none}html.theme--documenter-dark .hero.is-white{background-color:#fff;color:#0a0a0a}html.theme--documenter-dark .hero.is-white a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-white strong{color:inherit}html.theme--documenter-dark .hero.is-white .title{color:#0a0a0a}html.theme--documenter-dark .hero.is-white .subtitle{color:rgba(10,10,10,0.9)}html.theme--documenter-dark .hero.is-white .subtitle a:not(.button),html.theme--documenter-dark .hero.is-white .subtitle strong{color:#0a0a0a}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-white .navbar-menu{background-color:#fff}}html.theme--documenter-dark .hero.is-white .navbar-item,html.theme--documenter-dark .hero.is-white .navbar-link{color:rgba(10,10,10,0.7)}html.theme--documenter-dark .hero.is-white a.navbar-item:hover,html.theme--documenter-dark .hero.is-white a.navbar-item.is-active,html.theme--documenter-dark .hero.is-white .navbar-link:hover,html.theme--documenter-dark .hero.is-white .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}html.theme--documenter-dark .hero.is-white .tabs a{color:#0a0a0a;opacity:0.9}html.theme--documenter-dark .hero.is-white .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-white .tabs li.is-active a{color:#fff !important;opacity:1}html.theme--documenter-dark .hero.is-white .tabs.is-boxed a,html.theme--documenter-dark .hero.is-white .tabs.is-toggle a{color:#0a0a0a}html.theme--documenter-dark .hero.is-white .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-white .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-white .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-white .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-white .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-white .tabs.is-toggle li.is-active a:hover{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}html.theme--documenter-dark .hero.is-white.is-bold{background-image:linear-gradient(141deg, #e8e3e4 0%, #fff 71%, #fff 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-white.is-bold .navbar-menu{background-image:linear-gradient(141deg, #e8e3e4 0%, #fff 71%, #fff 100%)}}html.theme--documenter-dark .hero.is-black{background-color:#0a0a0a;color:#fff}html.theme--documenter-dark .hero.is-black a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-black strong{color:inherit}html.theme--documenter-dark .hero.is-black .title{color:#fff}html.theme--documenter-dark .hero.is-black .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-black .subtitle a:not(.button),html.theme--documenter-dark .hero.is-black .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-black .navbar-menu{background-color:#0a0a0a}}html.theme--documenter-dark .hero.is-black .navbar-item,html.theme--documenter-dark .hero.is-black .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-black a.navbar-item:hover,html.theme--documenter-dark .hero.is-black a.navbar-item.is-active,html.theme--documenter-dark .hero.is-black .navbar-link:hover,html.theme--documenter-dark .hero.is-black .navbar-link.is-active{background-color:#000;color:#fff}html.theme--documenter-dark .hero.is-black .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-black .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-black .tabs li.is-active a{color:#0a0a0a !important;opacity:1}html.theme--documenter-dark .hero.is-black .tabs.is-boxed a,html.theme--documenter-dark .hero.is-black .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-black .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-black .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-black .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-black .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-black .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-black .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#0a0a0a}html.theme--documenter-dark .hero.is-black.is-bold{background-image:linear-gradient(141deg, #000 0%, #0a0a0a 71%, #181616 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-black.is-bold .navbar-menu{background-image:linear-gradient(141deg, #000 0%, #0a0a0a 71%, #181616 100%)}}html.theme--documenter-dark .hero.is-light{background-color:#ecf0f1;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .hero.is-light a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-light strong{color:inherit}html.theme--documenter-dark .hero.is-light .title{color:rgba(0,0,0,0.7)}html.theme--documenter-dark .hero.is-light .subtitle{color:rgba(0,0,0,0.9)}html.theme--documenter-dark .hero.is-light .subtitle a:not(.button),html.theme--documenter-dark .hero.is-light .subtitle strong{color:rgba(0,0,0,0.7)}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-light .navbar-menu{background-color:#ecf0f1}}html.theme--documenter-dark .hero.is-light .navbar-item,html.theme--documenter-dark .hero.is-light .navbar-link{color:rgba(0,0,0,0.7)}html.theme--documenter-dark .hero.is-light a.navbar-item:hover,html.theme--documenter-dark .hero.is-light a.navbar-item.is-active,html.theme--documenter-dark .hero.is-light .navbar-link:hover,html.theme--documenter-dark .hero.is-light .navbar-link.is-active{background-color:#dde4e6;color:rgba(0,0,0,0.7)}html.theme--documenter-dark .hero.is-light .tabs a{color:rgba(0,0,0,0.7);opacity:0.9}html.theme--documenter-dark .hero.is-light .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-light .tabs li.is-active a{color:#ecf0f1 !important;opacity:1}html.theme--documenter-dark .hero.is-light .tabs.is-boxed a,html.theme--documenter-dark .hero.is-light .tabs.is-toggle a{color:rgba(0,0,0,0.7)}html.theme--documenter-dark .hero.is-light .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-light .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-light .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-light .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-light .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-light .tabs.is-toggle li.is-active a:hover{background-color:rgba(0,0,0,0.7);border-color:rgba(0,0,0,0.7);color:#ecf0f1}html.theme--documenter-dark .hero.is-light.is-bold{background-image:linear-gradient(141deg, #cadfe0 0%, #ecf0f1 71%, #fafbfc 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-light.is-bold .navbar-menu{background-image:linear-gradient(141deg, #cadfe0 0%, #ecf0f1 71%, #fafbfc 100%)}}html.theme--documenter-dark .hero.is-dark,html.theme--documenter-dark .content kbd.hero{background-color:#282f2f;color:#fff}html.theme--documenter-dark .hero.is-dark a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .content kbd.hero a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-dark strong,html.theme--documenter-dark .content kbd.hero strong{color:inherit}html.theme--documenter-dark .hero.is-dark .title,html.theme--documenter-dark .content kbd.hero .title{color:#fff}html.theme--documenter-dark .hero.is-dark .subtitle,html.theme--documenter-dark .content kbd.hero .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-dark .subtitle a:not(.button),html.theme--documenter-dark .content kbd.hero .subtitle a:not(.button),html.theme--documenter-dark .hero.is-dark .subtitle strong,html.theme--documenter-dark .content kbd.hero .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-dark .navbar-menu,html.theme--documenter-dark .content kbd.hero .navbar-menu{background-color:#282f2f}}html.theme--documenter-dark .hero.is-dark .navbar-item,html.theme--documenter-dark .content kbd.hero .navbar-item,html.theme--documenter-dark .hero.is-dark .navbar-link,html.theme--documenter-dark .content kbd.hero .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-dark a.navbar-item:hover,html.theme--documenter-dark .content kbd.hero a.navbar-item:hover,html.theme--documenter-dark .hero.is-dark a.navbar-item.is-active,html.theme--documenter-dark .content kbd.hero a.navbar-item.is-active,html.theme--documenter-dark .hero.is-dark .navbar-link:hover,html.theme--documenter-dark .content kbd.hero .navbar-link:hover,html.theme--documenter-dark .hero.is-dark .navbar-link.is-active,html.theme--documenter-dark .content kbd.hero .navbar-link.is-active{background-color:#1d2122;color:#fff}html.theme--documenter-dark .hero.is-dark .tabs a,html.theme--documenter-dark .content kbd.hero .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-dark .tabs a:hover,html.theme--documenter-dark .content kbd.hero .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-dark .tabs li.is-active a,html.theme--documenter-dark .content kbd.hero .tabs li.is-active a{color:#282f2f !important;opacity:1}html.theme--documenter-dark .hero.is-dark .tabs.is-boxed a,html.theme--documenter-dark .content kbd.hero .tabs.is-boxed a,html.theme--documenter-dark .hero.is-dark .tabs.is-toggle a,html.theme--documenter-dark .content kbd.hero .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-dark .tabs.is-boxed a:hover,html.theme--documenter-dark .content kbd.hero .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-dark .tabs.is-toggle a:hover,html.theme--documenter-dark .content kbd.hero .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-dark .tabs.is-boxed li.is-active a,html.theme--documenter-dark .content kbd.hero .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-dark .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-dark .tabs.is-toggle li.is-active a,html.theme--documenter-dark .content kbd.hero .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-dark .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#282f2f}html.theme--documenter-dark .hero.is-dark.is-bold,html.theme--documenter-dark .content kbd.hero.is-bold{background-image:linear-gradient(141deg, #0f1615 0%, #282f2f 71%, #313c40 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-dark.is-bold .navbar-menu,html.theme--documenter-dark .content kbd.hero.is-bold .navbar-menu{background-image:linear-gradient(141deg, #0f1615 0%, #282f2f 71%, #313c40 100%)}}html.theme--documenter-dark .hero.is-primary,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink{background-color:#375a7f;color:#fff}html.theme--documenter-dark .hero.is-primary a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-primary strong,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink strong{color:inherit}html.theme--documenter-dark .hero.is-primary .title,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .title{color:#fff}html.theme--documenter-dark .hero.is-primary .subtitle,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-primary .subtitle a:not(.button),html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .subtitle a:not(.button),html.theme--documenter-dark .hero.is-primary .subtitle strong,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-primary .navbar-menu,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar-menu{background-color:#375a7f}}html.theme--documenter-dark .hero.is-primary .navbar-item,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar-item,html.theme--documenter-dark .hero.is-primary .navbar-link,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-primary a.navbar-item:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink a.navbar-item:hover,html.theme--documenter-dark .hero.is-primary a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink a.navbar-item.is-active,html.theme--documenter-dark .hero.is-primary .navbar-link:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar-link:hover,html.theme--documenter-dark .hero.is-primary .navbar-link.is-active,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar-link.is-active{background-color:#2f4d6d;color:#fff}html.theme--documenter-dark .hero.is-primary .tabs a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-primary .tabs a:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-primary .tabs li.is-active a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs li.is-active a{color:#375a7f !important;opacity:1}html.theme--documenter-dark .hero.is-primary .tabs.is-boxed a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-boxed a,html.theme--documenter-dark .hero.is-primary .tabs.is-toggle a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-primary .tabs.is-boxed a:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-primary .tabs.is-toggle a:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-primary .tabs.is-boxed li.is-active a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-primary .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-primary .tabs.is-toggle li.is-active a,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-primary .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#375a7f}html.theme--documenter-dark .hero.is-primary.is-bold,html.theme--documenter-dark .docstring>section>a.hero.is-bold.docs-sourcelink{background-image:linear-gradient(141deg, #214b62 0%, #375a7f 71%, #3a5796 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-primary.is-bold .navbar-menu,html.theme--documenter-dark .docstring>section>a.hero.is-bold.docs-sourcelink .navbar-menu{background-image:linear-gradient(141deg, #214b62 0%, #375a7f 71%, #3a5796 100%)}}html.theme--documenter-dark .hero.is-link{background-color:#1abc9c;color:#fff}html.theme--documenter-dark .hero.is-link a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-link strong{color:inherit}html.theme--documenter-dark .hero.is-link .title{color:#fff}html.theme--documenter-dark .hero.is-link .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-link .subtitle a:not(.button),html.theme--documenter-dark .hero.is-link .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-link .navbar-menu{background-color:#1abc9c}}html.theme--documenter-dark .hero.is-link .navbar-item,html.theme--documenter-dark .hero.is-link .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-link a.navbar-item:hover,html.theme--documenter-dark .hero.is-link a.navbar-item.is-active,html.theme--documenter-dark .hero.is-link .navbar-link:hover,html.theme--documenter-dark .hero.is-link .navbar-link.is-active{background-color:#17a689;color:#fff}html.theme--documenter-dark .hero.is-link .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-link .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-link .tabs li.is-active a{color:#1abc9c !important;opacity:1}html.theme--documenter-dark .hero.is-link .tabs.is-boxed a,html.theme--documenter-dark .hero.is-link .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-link .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-link .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-link .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-link .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-link .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-link .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#1abc9c}html.theme--documenter-dark .hero.is-link.is-bold{background-image:linear-gradient(141deg, #0c9764 0%, #1abc9c 71%, #17d8d2 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-link.is-bold .navbar-menu{background-image:linear-gradient(141deg, #0c9764 0%, #1abc9c 71%, #17d8d2 100%)}}html.theme--documenter-dark .hero.is-info{background-color:#024c7d;color:#fff}html.theme--documenter-dark .hero.is-info a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-info strong{color:inherit}html.theme--documenter-dark .hero.is-info .title{color:#fff}html.theme--documenter-dark .hero.is-info .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-info .subtitle a:not(.button),html.theme--documenter-dark .hero.is-info .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-info .navbar-menu{background-color:#024c7d}}html.theme--documenter-dark .hero.is-info .navbar-item,html.theme--documenter-dark .hero.is-info .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-info a.navbar-item:hover,html.theme--documenter-dark .hero.is-info a.navbar-item.is-active,html.theme--documenter-dark .hero.is-info .navbar-link:hover,html.theme--documenter-dark .hero.is-info .navbar-link.is-active{background-color:#023d64;color:#fff}html.theme--documenter-dark .hero.is-info .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-info .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-info .tabs li.is-active a{color:#024c7d !important;opacity:1}html.theme--documenter-dark .hero.is-info .tabs.is-boxed a,html.theme--documenter-dark .hero.is-info .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-info .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-info .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-info .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-info .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-info .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-info .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#024c7d}html.theme--documenter-dark .hero.is-info.is-bold{background-image:linear-gradient(141deg, #003a4c 0%, #024c7d 71%, #004299 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-info.is-bold .navbar-menu{background-image:linear-gradient(141deg, #003a4c 0%, #024c7d 71%, #004299 100%)}}html.theme--documenter-dark .hero.is-success{background-color:#008438;color:#fff}html.theme--documenter-dark .hero.is-success a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-success strong{color:inherit}html.theme--documenter-dark .hero.is-success .title{color:#fff}html.theme--documenter-dark .hero.is-success .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-success .subtitle a:not(.button),html.theme--documenter-dark .hero.is-success .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-success .navbar-menu{background-color:#008438}}html.theme--documenter-dark .hero.is-success .navbar-item,html.theme--documenter-dark .hero.is-success .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-success a.navbar-item:hover,html.theme--documenter-dark .hero.is-success a.navbar-item.is-active,html.theme--documenter-dark .hero.is-success .navbar-link:hover,html.theme--documenter-dark .hero.is-success .navbar-link.is-active{background-color:#006b2d;color:#fff}html.theme--documenter-dark .hero.is-success .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-success .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-success .tabs li.is-active a{color:#008438 !important;opacity:1}html.theme--documenter-dark .hero.is-success .tabs.is-boxed a,html.theme--documenter-dark .hero.is-success .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-success .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-success .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-success .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-success .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-success .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-success .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#008438}html.theme--documenter-dark .hero.is-success.is-bold{background-image:linear-gradient(141deg, #005115 0%, #008438 71%, #009e5d 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-success.is-bold .navbar-menu{background-image:linear-gradient(141deg, #005115 0%, #008438 71%, #009e5d 100%)}}html.theme--documenter-dark .hero.is-warning{background-color:#ad8100;color:#fff}html.theme--documenter-dark .hero.is-warning a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-warning strong{color:inherit}html.theme--documenter-dark .hero.is-warning .title{color:#fff}html.theme--documenter-dark .hero.is-warning .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-warning .subtitle a:not(.button),html.theme--documenter-dark .hero.is-warning .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-warning .navbar-menu{background-color:#ad8100}}html.theme--documenter-dark .hero.is-warning .navbar-item,html.theme--documenter-dark .hero.is-warning .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-warning a.navbar-item:hover,html.theme--documenter-dark .hero.is-warning a.navbar-item.is-active,html.theme--documenter-dark .hero.is-warning .navbar-link:hover,html.theme--documenter-dark .hero.is-warning .navbar-link.is-active{background-color:#946e00;color:#fff}html.theme--documenter-dark .hero.is-warning .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-warning .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-warning .tabs li.is-active a{color:#ad8100 !important;opacity:1}html.theme--documenter-dark .hero.is-warning .tabs.is-boxed a,html.theme--documenter-dark .hero.is-warning .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-warning .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-warning .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-warning .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-warning .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-warning .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-warning .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#ad8100}html.theme--documenter-dark .hero.is-warning.is-bold{background-image:linear-gradient(141deg, #7a4700 0%, #ad8100 71%, #c7b500 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-warning.is-bold .navbar-menu{background-image:linear-gradient(141deg, #7a4700 0%, #ad8100 71%, #c7b500 100%)}}html.theme--documenter-dark .hero.is-danger{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .hero.is-danger a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),html.theme--documenter-dark .hero.is-danger strong{color:inherit}html.theme--documenter-dark .hero.is-danger .title{color:#fff}html.theme--documenter-dark .hero.is-danger .subtitle{color:rgba(255,255,255,0.9)}html.theme--documenter-dark .hero.is-danger .subtitle a:not(.button),html.theme--documenter-dark .hero.is-danger .subtitle strong{color:#fff}@media screen and (max-width: 1055px){html.theme--documenter-dark .hero.is-danger .navbar-menu{background-color:#9e1b0d}}html.theme--documenter-dark .hero.is-danger .navbar-item,html.theme--documenter-dark .hero.is-danger .navbar-link{color:rgba(255,255,255,0.7)}html.theme--documenter-dark .hero.is-danger a.navbar-item:hover,html.theme--documenter-dark .hero.is-danger a.navbar-item.is-active,html.theme--documenter-dark .hero.is-danger .navbar-link:hover,html.theme--documenter-dark .hero.is-danger .navbar-link.is-active{background-color:#86170b;color:#fff}html.theme--documenter-dark .hero.is-danger .tabs a{color:#fff;opacity:0.9}html.theme--documenter-dark .hero.is-danger .tabs a:hover{opacity:1}html.theme--documenter-dark .hero.is-danger .tabs li.is-active a{color:#9e1b0d !important;opacity:1}html.theme--documenter-dark .hero.is-danger .tabs.is-boxed a,html.theme--documenter-dark .hero.is-danger .tabs.is-toggle a{color:#fff}html.theme--documenter-dark .hero.is-danger .tabs.is-boxed a:hover,html.theme--documenter-dark .hero.is-danger .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}html.theme--documenter-dark .hero.is-danger .tabs.is-boxed li.is-active a,html.theme--documenter-dark .hero.is-danger .tabs.is-boxed li.is-active a:hover,html.theme--documenter-dark .hero.is-danger .tabs.is-toggle li.is-active a,html.theme--documenter-dark .hero.is-danger .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#9e1b0d}html.theme--documenter-dark .hero.is-danger.is-bold{background-image:linear-gradient(141deg, #75030b 0%, #9e1b0d 71%, #ba380a 100%)}@media screen and (max-width: 768px){html.theme--documenter-dark .hero.is-danger.is-bold .navbar-menu{background-image:linear-gradient(141deg, #75030b 0%, #9e1b0d 71%, #ba380a 100%)}}html.theme--documenter-dark .hero.is-small .hero-body,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.hero .hero-body{padding:1.5rem}@media screen and (min-width: 769px),print{html.theme--documenter-dark .hero.is-medium .hero-body{padding:9rem 4.5rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .hero.is-large .hero-body{padding:18rem 6rem}}html.theme--documenter-dark .hero.is-halfheight .hero-body,html.theme--documenter-dark .hero.is-fullheight .hero-body,html.theme--documenter-dark .hero.is-fullheight-with-navbar .hero-body{align-items:center;display:flex}html.theme--documenter-dark .hero.is-halfheight .hero-body>.container,html.theme--documenter-dark .hero.is-fullheight .hero-body>.container,html.theme--documenter-dark .hero.is-fullheight-with-navbar .hero-body>.container{flex-grow:1;flex-shrink:1}html.theme--documenter-dark .hero.is-halfheight{min-height:50vh}html.theme--documenter-dark .hero.is-fullheight{min-height:100vh}html.theme--documenter-dark .hero-video{overflow:hidden}html.theme--documenter-dark .hero-video video{left:50%;min-height:100%;min-width:100%;position:absolute;top:50%;transform:translate3d(-50%, -50%, 0)}html.theme--documenter-dark .hero-video.is-transparent{opacity:0.3}@media screen and (max-width: 768px){html.theme--documenter-dark .hero-video{display:none}}html.theme--documenter-dark .hero-buttons{margin-top:1.5rem}@media screen and (max-width: 768px){html.theme--documenter-dark .hero-buttons .button{display:flex}html.theme--documenter-dark .hero-buttons .button:not(:last-child){margin-bottom:0.75rem}}@media screen and (min-width: 769px),print{html.theme--documenter-dark .hero-buttons{display:flex;justify-content:center}html.theme--documenter-dark .hero-buttons .button:not(:last-child){margin-right:1.5rem}}html.theme--documenter-dark .hero-head,html.theme--documenter-dark .hero-foot{flex-grow:0;flex-shrink:0}html.theme--documenter-dark .hero-body{flex-grow:1;flex-shrink:0;padding:3rem 1.5rem}@media screen and (min-width: 769px),print{html.theme--documenter-dark .hero-body{padding:3rem 3rem}}html.theme--documenter-dark .section{padding:3rem 1.5rem}@media screen and (min-width: 1056px){html.theme--documenter-dark .section{padding:3rem 3rem}html.theme--documenter-dark .section.is-medium{padding:9rem 4.5rem}html.theme--documenter-dark .section.is-large{padding:18rem 6rem}}html.theme--documenter-dark .footer{background-color:#282f2f;padding:3rem 1.5rem 6rem}html.theme--documenter-dark hr{height:1px}html.theme--documenter-dark h6{text-transform:uppercase;letter-spacing:0.5px}html.theme--documenter-dark .hero{background-color:#343c3d}html.theme--documenter-dark a{transition:all 200ms ease}html.theme--documenter-dark .button{transition:all 200ms ease;border-width:1px;color:#fff}html.theme--documenter-dark .button.is-active,html.theme--documenter-dark .button.is-focused,html.theme--documenter-dark .button:active,html.theme--documenter-dark .button:focus{box-shadow:0 0 0 2px rgba(140,155,157,0.5)}html.theme--documenter-dark .button.is-white.is-hovered,html.theme--documenter-dark .button.is-white:hover{background-color:#fff}html.theme--documenter-dark .button.is-white.is-active,html.theme--documenter-dark .button.is-white.is-focused,html.theme--documenter-dark .button.is-white:active,html.theme--documenter-dark .button.is-white:focus{border-color:#fff;box-shadow:0 0 0 2px rgba(255,255,255,0.5)}html.theme--documenter-dark .button.is-black.is-hovered,html.theme--documenter-dark .button.is-black:hover{background-color:#1d1d1d}html.theme--documenter-dark .button.is-black.is-active,html.theme--documenter-dark .button.is-black.is-focused,html.theme--documenter-dark .button.is-black:active,html.theme--documenter-dark .button.is-black:focus{border-color:#0a0a0a;box-shadow:0 0 0 2px rgba(10,10,10,0.5)}html.theme--documenter-dark .button.is-light.is-hovered,html.theme--documenter-dark .button.is-light:hover{background-color:#fff}html.theme--documenter-dark .button.is-light.is-active,html.theme--documenter-dark .button.is-light.is-focused,html.theme--documenter-dark .button.is-light:active,html.theme--documenter-dark .button.is-light:focus{border-color:#ecf0f1;box-shadow:0 0 0 2px rgba(236,240,241,0.5)}html.theme--documenter-dark .button.is-dark.is-hovered,html.theme--documenter-dark .content kbd.button.is-hovered,html.theme--documenter-dark .button.is-dark:hover,html.theme--documenter-dark .content kbd.button:hover{background-color:#3a4344}html.theme--documenter-dark .button.is-dark.is-active,html.theme--documenter-dark .content kbd.button.is-active,html.theme--documenter-dark .button.is-dark.is-focused,html.theme--documenter-dark .content kbd.button.is-focused,html.theme--documenter-dark .button.is-dark:active,html.theme--documenter-dark .content kbd.button:active,html.theme--documenter-dark .button.is-dark:focus,html.theme--documenter-dark .content kbd.button:focus{border-color:#282f2f;box-shadow:0 0 0 2px rgba(40,47,47,0.5)}html.theme--documenter-dark .button.is-primary.is-hovered,html.theme--documenter-dark .docstring>section>a.button.is-hovered.docs-sourcelink,html.theme--documenter-dark .button.is-primary:hover,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:hover{background-color:#436d9a}html.theme--documenter-dark .button.is-primary.is-active,html.theme--documenter-dark .docstring>section>a.button.is-active.docs-sourcelink,html.theme--documenter-dark .button.is-primary.is-focused,html.theme--documenter-dark .docstring>section>a.button.is-focused.docs-sourcelink,html.theme--documenter-dark .button.is-primary:active,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:active,html.theme--documenter-dark .button.is-primary:focus,html.theme--documenter-dark .docstring>section>a.button.docs-sourcelink:focus{border-color:#375a7f;box-shadow:0 0 0 2px rgba(55,90,127,0.5)}html.theme--documenter-dark .button.is-link.is-hovered,html.theme--documenter-dark .button.is-link:hover{background-color:#1fdeb8}html.theme--documenter-dark .button.is-link.is-active,html.theme--documenter-dark .button.is-link.is-focused,html.theme--documenter-dark .button.is-link:active,html.theme--documenter-dark .button.is-link:focus{border-color:#1abc9c;box-shadow:0 0 0 2px rgba(26,188,156,0.5)}html.theme--documenter-dark .button.is-info.is-hovered,html.theme--documenter-dark .button.is-info:hover{background-color:#0363a3}html.theme--documenter-dark .button.is-info.is-active,html.theme--documenter-dark .button.is-info.is-focused,html.theme--documenter-dark .button.is-info:active,html.theme--documenter-dark .button.is-info:focus{border-color:#024c7d;box-shadow:0 0 0 2px rgba(2,76,125,0.5)}html.theme--documenter-dark .button.is-success.is-hovered,html.theme--documenter-dark .button.is-success:hover{background-color:#00aa48}html.theme--documenter-dark .button.is-success.is-active,html.theme--documenter-dark .button.is-success.is-focused,html.theme--documenter-dark .button.is-success:active,html.theme--documenter-dark .button.is-success:focus{border-color:#008438;box-shadow:0 0 0 2px rgba(0,132,56,0.5)}html.theme--documenter-dark .button.is-warning.is-hovered,html.theme--documenter-dark .button.is-warning:hover{background-color:#d39e00}html.theme--documenter-dark .button.is-warning.is-active,html.theme--documenter-dark .button.is-warning.is-focused,html.theme--documenter-dark .button.is-warning:active,html.theme--documenter-dark .button.is-warning:focus{border-color:#ad8100;box-shadow:0 0 0 2px rgba(173,129,0,0.5)}html.theme--documenter-dark .button.is-danger.is-hovered,html.theme--documenter-dark .button.is-danger:hover{background-color:#c12110}html.theme--documenter-dark .button.is-danger.is-active,html.theme--documenter-dark .button.is-danger.is-focused,html.theme--documenter-dark .button.is-danger:active,html.theme--documenter-dark .button.is-danger:focus{border-color:#9e1b0d;box-shadow:0 0 0 2px rgba(158,27,13,0.5)}html.theme--documenter-dark .label{color:#dbdee0}html.theme--documenter-dark .button,html.theme--documenter-dark .control.has-icons-left .icon,html.theme--documenter-dark .control.has-icons-right .icon,html.theme--documenter-dark .input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark .pagination-ellipsis,html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-previous,html.theme--documenter-dark .select,html.theme--documenter-dark .select select,html.theme--documenter-dark .textarea{height:2.5em}html.theme--documenter-dark .input,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark .textarea{transition:all 200ms ease;box-shadow:none;border-width:1px;padding-left:1em;padding-right:1em}html.theme--documenter-dark .select:after,html.theme--documenter-dark .select select{border-width:1px}html.theme--documenter-dark .control.has-addons .button,html.theme--documenter-dark .control.has-addons .input,html.theme--documenter-dark .control.has-addons #documenter .docs-sidebar form.docs-search>input,html.theme--documenter-dark #documenter .docs-sidebar .control.has-addons form.docs-search>input,html.theme--documenter-dark .control.has-addons .select{margin-right:-1px}html.theme--documenter-dark .notification{background-color:#343c3d}html.theme--documenter-dark .card{box-shadow:none;border:1px solid #343c3d;background-color:#282f2f;border-radius:.4em}html.theme--documenter-dark .card .card-image img{border-radius:.4em .4em 0 0}html.theme--documenter-dark .card .card-header{box-shadow:none;background-color:rgba(18,18,18,0.2);border-radius:.4em .4em 0 0}html.theme--documenter-dark .card .card-footer{background-color:rgba(18,18,18,0.2)}html.theme--documenter-dark .card .card-footer,html.theme--documenter-dark .card .card-footer-item{border-width:1px;border-color:#343c3d}html.theme--documenter-dark .notification.is-white a:not(.button){color:#0a0a0a;text-decoration:underline}html.theme--documenter-dark .notification.is-black a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-light a:not(.button){color:rgba(0,0,0,0.7);text-decoration:underline}html.theme--documenter-dark .notification.is-dark a:not(.button),html.theme--documenter-dark .content kbd.notification a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-primary a:not(.button),html.theme--documenter-dark .docstring>section>a.notification.docs-sourcelink a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-link a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-info a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-success a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-warning a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .notification.is-danger a:not(.button){color:#fff;text-decoration:underline}html.theme--documenter-dark .tag,html.theme--documenter-dark .content kbd,html.theme--documenter-dark .docstring>section>a.docs-sourcelink{border-radius:.4em}html.theme--documenter-dark .menu-list a{transition:all 300ms ease}html.theme--documenter-dark .modal-card-body{background-color:#282f2f}html.theme--documenter-dark .modal-card-foot,html.theme--documenter-dark .modal-card-head{border-color:#343c3d}html.theme--documenter-dark .message-header{font-weight:700;background-color:#343c3d;color:#fff}html.theme--documenter-dark .message-body{border-width:1px;border-color:#343c3d}html.theme--documenter-dark .navbar{border-radius:.4em}html.theme--documenter-dark .navbar.is-transparent{background:none}html.theme--documenter-dark .navbar.is-primary .navbar-dropdown a.navbar-item.is-active,html.theme--documenter-dark .docstring>section>a.navbar.docs-sourcelink .navbar-dropdown a.navbar-item.is-active{background-color:#1abc9c}@media screen and (max-width: 1055px){html.theme--documenter-dark .navbar .navbar-menu{background-color:#375a7f;border-radius:0 0 .4em .4em}}html.theme--documenter-dark .hero .navbar,html.theme--documenter-dark body>.navbar{border-radius:0}html.theme--documenter-dark .pagination-link,html.theme--documenter-dark .pagination-next,html.theme--documenter-dark .pagination-previous{border-width:1px}html.theme--documenter-dark .panel-block,html.theme--documenter-dark .panel-heading,html.theme--documenter-dark .panel-tabs{border-width:1px}html.theme--documenter-dark .panel-block:first-child,html.theme--documenter-dark .panel-heading:first-child,html.theme--documenter-dark .panel-tabs:first-child{border-top-width:1px}html.theme--documenter-dark .panel-heading{font-weight:700}html.theme--documenter-dark .panel-tabs a{border-width:1px;margin-bottom:-1px}html.theme--documenter-dark .panel-tabs a.is-active{border-bottom-color:#17a689}html.theme--documenter-dark .panel-block:hover{color:#1dd2af}html.theme--documenter-dark .panel-block:hover .panel-icon{color:#1dd2af}html.theme--documenter-dark .panel-block.is-active .panel-icon{color:#17a689}html.theme--documenter-dark .tabs a{border-bottom-width:1px;margin-bottom:-1px}html.theme--documenter-dark .tabs ul{border-bottom-width:1px}html.theme--documenter-dark .tabs.is-boxed a{border-width:1px}html.theme--documenter-dark .tabs.is-boxed li.is-active a{background-color:#1f2424}html.theme--documenter-dark .tabs.is-toggle li a{border-width:1px;margin-bottom:0}html.theme--documenter-dark .tabs.is-toggle li+li{margin-left:-1px}html.theme--documenter-dark .hero.is-white .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-black .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-light .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-dark .navbar .navbar-dropdown .navbar-item:hover,html.theme--documenter-dark .content kbd.hero .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-primary .navbar .navbar-dropdown .navbar-item:hover,html.theme--documenter-dark .docstring>section>a.hero.docs-sourcelink .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-link .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-info .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-success .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-warning .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark .hero.is-danger .navbar .navbar-dropdown .navbar-item:hover{background-color:rgba(0,0,0,0)}html.theme--documenter-dark h1 .docs-heading-anchor,html.theme--documenter-dark h1 .docs-heading-anchor:hover,html.theme--documenter-dark h1 .docs-heading-anchor:visited,html.theme--documenter-dark h2 .docs-heading-anchor,html.theme--documenter-dark h2 .docs-heading-anchor:hover,html.theme--documenter-dark h2 .docs-heading-anchor:visited,html.theme--documenter-dark h3 .docs-heading-anchor,html.theme--documenter-dark h3 .docs-heading-anchor:hover,html.theme--documenter-dark h3 .docs-heading-anchor:visited,html.theme--documenter-dark h4 .docs-heading-anchor,html.theme--documenter-dark h4 .docs-heading-anchor:hover,html.theme--documenter-dark h4 .docs-heading-anchor:visited,html.theme--documenter-dark h5 .docs-heading-anchor,html.theme--documenter-dark h5 .docs-heading-anchor:hover,html.theme--documenter-dark h5 .docs-heading-anchor:visited,html.theme--documenter-dark h6 .docs-heading-anchor,html.theme--documenter-dark h6 .docs-heading-anchor:hover,html.theme--documenter-dark h6 .docs-heading-anchor:visited{color:#f2f2f2}html.theme--documenter-dark h1 .docs-heading-anchor-permalink,html.theme--documenter-dark h2 .docs-heading-anchor-permalink,html.theme--documenter-dark h3 .docs-heading-anchor-permalink,html.theme--documenter-dark h4 .docs-heading-anchor-permalink,html.theme--documenter-dark h5 .docs-heading-anchor-permalink,html.theme--documenter-dark h6 .docs-heading-anchor-permalink{visibility:hidden;vertical-align:middle;margin-left:0.5em;font-size:0.7rem}html.theme--documenter-dark h1 .docs-heading-anchor-permalink::before,html.theme--documenter-dark h2 .docs-heading-anchor-permalink::before,html.theme--documenter-dark h3 .docs-heading-anchor-permalink::before,html.theme--documenter-dark h4 .docs-heading-anchor-permalink::before,html.theme--documenter-dark h5 .docs-heading-anchor-permalink::before,html.theme--documenter-dark h6 .docs-heading-anchor-permalink::before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f0c1"}html.theme--documenter-dark h1:hover .docs-heading-anchor-permalink,html.theme--documenter-dark h2:hover .docs-heading-anchor-permalink,html.theme--documenter-dark h3:hover .docs-heading-anchor-permalink,html.theme--documenter-dark h4:hover .docs-heading-anchor-permalink,html.theme--documenter-dark h5:hover .docs-heading-anchor-permalink,html.theme--documenter-dark h6:hover .docs-heading-anchor-permalink{visibility:visible}html.theme--documenter-dark .docs-light-only{display:none !important}html.theme--documenter-dark pre{position:relative;overflow:hidden}html.theme--documenter-dark pre code,html.theme--documenter-dark pre code.hljs{padding:0 .75rem !important;overflow:auto;display:block}html.theme--documenter-dark pre code:first-of-type,html.theme--documenter-dark pre code.hljs:first-of-type{padding-top:0.5rem !important}html.theme--documenter-dark pre code:last-of-type,html.theme--documenter-dark pre code.hljs:last-of-type{padding-bottom:0.5rem !important}html.theme--documenter-dark pre .copy-button{opacity:0.2;transition:opacity 0.2s;position:absolute;right:0em;top:0em;padding:0.5em;width:2.5em;height:2.5em;background:transparent;border:none;font-family:"Font Awesome 6 Free";color:#fff;cursor:pointer;text-align:center}html.theme--documenter-dark pre .copy-button:focus,html.theme--documenter-dark pre .copy-button:hover{opacity:1;background:rgba(255,255,255,0.1);color:#1abc9c}html.theme--documenter-dark pre .copy-button.success{color:#259a12;opacity:1}html.theme--documenter-dark pre .copy-button.error{color:#cb3c33;opacity:1}html.theme--documenter-dark pre:hover .copy-button{opacity:1}html.theme--documenter-dark .admonition{background-color:#282f2f;border-style:solid;border-width:1px;border-color:#5e6d6f;border-radius:.4em;font-size:1rem}html.theme--documenter-dark .admonition strong{color:currentColor}html.theme--documenter-dark .admonition.is-small,html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input.admonition{font-size:.75rem}html.theme--documenter-dark .admonition.is-medium{font-size:1.25rem}html.theme--documenter-dark .admonition.is-large{font-size:1.5rem}html.theme--documenter-dark .admonition.is-default{background-color:#282f2f;border-color:#5e6d6f}html.theme--documenter-dark .admonition.is-default>.admonition-header{background-color:#5e6d6f;color:#fff}html.theme--documenter-dark .admonition.is-default>.admonition-body{color:#fff}html.theme--documenter-dark .admonition.is-info{background-color:#282f2f;border-color:#024c7d}html.theme--documenter-dark .admonition.is-info>.admonition-header{background-color:#024c7d;color:#fff}html.theme--documenter-dark .admonition.is-info>.admonition-body{color:#fff}html.theme--documenter-dark .admonition.is-success{background-color:#282f2f;border-color:#008438}html.theme--documenter-dark .admonition.is-success>.admonition-header{background-color:#008438;color:#fff}html.theme--documenter-dark .admonition.is-success>.admonition-body{color:#fff}html.theme--documenter-dark .admonition.is-warning{background-color:#282f2f;border-color:#ad8100}html.theme--documenter-dark .admonition.is-warning>.admonition-header{background-color:#ad8100;color:#fff}html.theme--documenter-dark .admonition.is-warning>.admonition-body{color:#fff}html.theme--documenter-dark .admonition.is-danger{background-color:#282f2f;border-color:#9e1b0d}html.theme--documenter-dark .admonition.is-danger>.admonition-header{background-color:#9e1b0d;color:#fff}html.theme--documenter-dark .admonition.is-danger>.admonition-body{color:#fff}html.theme--documenter-dark .admonition.is-compat{background-color:#282f2f;border-color:#137886}html.theme--documenter-dark .admonition.is-compat>.admonition-header{background-color:#137886;color:#fff}html.theme--documenter-dark .admonition.is-compat>.admonition-body{color:#fff}html.theme--documenter-dark .admonition-header{color:#fff;background-color:#5e6d6f;align-items:center;font-weight:700;justify-content:space-between;line-height:1.25;padding:0.5rem .75rem;position:relative}html.theme--documenter-dark .admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;margin-right:.75rem;content:"\f06a"}html.theme--documenter-dark details.admonition.is-details>.admonition-header{list-style:none}html.theme--documenter-dark details.admonition.is-details>.admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f055"}html.theme--documenter-dark details.admonition.is-details[open]>.admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f056"}html.theme--documenter-dark .admonition-body{color:#fff;padding:0.5rem .75rem}html.theme--documenter-dark .admonition-body pre{background-color:#282f2f}html.theme--documenter-dark .admonition-body code{background-color:rgba(255,255,255,0.05)}html.theme--documenter-dark .docstring{margin-bottom:1em;background-color:rgba(0,0,0,0);border:1px solid #5e6d6f;box-shadow:none;max-width:100%}html.theme--documenter-dark .docstring>header{cursor:pointer;display:flex;flex-grow:1;align-items:stretch;padding:0.5rem .75rem;background-color:#282f2f;box-shadow:0 0.125em 0.25em rgba(10,10,10,0.1);box-shadow:none;border-bottom:1px solid #5e6d6f;overflow:auto}html.theme--documenter-dark .docstring>header code{background-color:transparent}html.theme--documenter-dark .docstring>header .docstring-article-toggle-button{min-width:1.1rem;padding:0.2rem 0.2rem 0.2rem 0}html.theme--documenter-dark .docstring>header .docstring-binding{margin-right:0.3em}html.theme--documenter-dark .docstring>header .docstring-category{margin-left:0.3em}html.theme--documenter-dark .docstring>section{position:relative;padding:.75rem .75rem;border-bottom:1px solid #5e6d6f}html.theme--documenter-dark .docstring>section:last-child{border-bottom:none}html.theme--documenter-dark .docstring>section>a.docs-sourcelink{transition:opacity 0.3s;opacity:0;position:absolute;right:.375rem;bottom:.375rem}html.theme--documenter-dark .docstring>section>a.docs-sourcelink:focus{opacity:1 !important}html.theme--documenter-dark .docstring:hover>section>a.docs-sourcelink{opacity:0.2}html.theme--documenter-dark .docstring:focus-within>section>a.docs-sourcelink{opacity:0.2}html.theme--documenter-dark .docstring>section:hover a.docs-sourcelink{opacity:1}html.theme--documenter-dark .documenter-example-output{background-color:#1f2424}html.theme--documenter-dark .outdated-warning-overlay{position:fixed;top:0;left:0;right:0;box-shadow:0 0 10px rgba(0,0,0,0.3);z-index:999;background-color:#282f2f;color:#fff;border-bottom:3px solid #9e1b0d;padding:10px 35px;text-align:center;font-size:15px}html.theme--documenter-dark .outdated-warning-overlay .outdated-warning-closer{position:absolute;top:calc(50% - 10px);right:18px;cursor:pointer;width:12px}html.theme--documenter-dark .outdated-warning-overlay a{color:#1abc9c}html.theme--documenter-dark .outdated-warning-overlay a:hover{color:#1dd2af}html.theme--documenter-dark .content pre{border:1px solid #5e6d6f}html.theme--documenter-dark .content code{font-weight:inherit}html.theme--documenter-dark .content a code{color:#1abc9c}html.theme--documenter-dark .content h1 code,html.theme--documenter-dark .content h2 code,html.theme--documenter-dark .content h3 code,html.theme--documenter-dark .content h4 code,html.theme--documenter-dark .content h5 code,html.theme--documenter-dark .content h6 code{color:#f2f2f2}html.theme--documenter-dark .content table{display:block;width:initial;max-width:100%;overflow-x:auto}html.theme--documenter-dark .content blockquote>ul:first-child,html.theme--documenter-dark .content blockquote>ol:first-child,html.theme--documenter-dark .content .admonition-body>ul:first-child,html.theme--documenter-dark .content .admonition-body>ol:first-child{margin-top:0}html.theme--documenter-dark pre,html.theme--documenter-dark code{font-variant-ligatures:no-contextual}html.theme--documenter-dark .breadcrumb a.is-disabled{cursor:default;pointer-events:none}html.theme--documenter-dark .breadcrumb a.is-disabled,html.theme--documenter-dark .breadcrumb a.is-disabled:hover{color:#f2f2f2}html.theme--documenter-dark .hljs{background:initial !important}html.theme--documenter-dark .katex .katex-mathml{top:0;right:0}html.theme--documenter-dark .katex-display,html.theme--documenter-dark mjx-container,html.theme--documenter-dark .MathJax_Display{margin:0.5em 0 !important}html.theme--documenter-dark html{-moz-osx-font-smoothing:auto;-webkit-font-smoothing:auto}html.theme--documenter-dark li.no-marker{list-style:none}html.theme--documenter-dark #documenter .docs-main>article{overflow-wrap:break-word}html.theme--documenter-dark #documenter .docs-main>article .math-container{overflow-x:auto;overflow-y:hidden}@media screen and (min-width: 1056px){html.theme--documenter-dark #documenter .docs-main{max-width:52rem;margin-left:20rem;padding-right:1rem}}@media screen and (max-width: 1055px){html.theme--documenter-dark #documenter .docs-main{width:100%}html.theme--documenter-dark #documenter .docs-main>article{max-width:52rem;margin-left:auto;margin-right:auto;margin-bottom:1rem;padding:0 1rem}html.theme--documenter-dark #documenter .docs-main>header,html.theme--documenter-dark #documenter .docs-main>nav{max-width:100%;width:100%;margin:0}}html.theme--documenter-dark #documenter .docs-main header.docs-navbar{background-color:#1f2424;border-bottom:1px solid #5e6d6f;z-index:2;min-height:4rem;margin-bottom:1rem;display:flex}html.theme--documenter-dark #documenter .docs-main header.docs-navbar .breadcrumb{flex-grow:1;overflow-x:hidden}html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-sidebar-button{display:block;font-size:1.5rem;padding-bottom:0.1rem;margin-right:1rem}html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-right{display:flex;white-space:nowrap;gap:1rem;align-items:center}html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-right .docs-icon,html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-right .docs-label{display:inline-block}html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-right .docs-label{padding:0;margin-left:0.3em}@media screen and (max-width: 1055px){html.theme--documenter-dark #documenter .docs-main header.docs-navbar .docs-right .docs-navbar-link{margin-left:0.4rem;margin-right:0.4rem}}html.theme--documenter-dark #documenter .docs-main header.docs-navbar>*{margin:auto 0}@media screen and (max-width: 1055px){html.theme--documenter-dark #documenter .docs-main header.docs-navbar{position:sticky;top:0;padding:0 1rem;transition-property:top, box-shadow;-webkit-transition-property:top, box-shadow;transition-duration:0.3s;-webkit-transition-duration:0.3s}html.theme--documenter-dark #documenter .docs-main header.docs-navbar.headroom--not-top{box-shadow:.2rem 0rem .4rem #171717;transition-duration:0.7s;-webkit-transition-duration:0.7s}html.theme--documenter-dark #documenter .docs-main header.docs-navbar.headroom--unpinned.headroom--not-top.headroom--not-bottom{top:-4.5rem;transition-duration:0.7s;-webkit-transition-duration:0.7s}}html.theme--documenter-dark #documenter .docs-main section.footnotes{border-top:1px solid #5e6d6f}html.theme--documenter-dark #documenter .docs-main section.footnotes li .tag:first-child,html.theme--documenter-dark #documenter .docs-main section.footnotes li .docstring>section>a.docs-sourcelink:first-child,html.theme--documenter-dark #documenter .docs-main section.footnotes li .content kbd:first-child,html.theme--documenter-dark .content #documenter .docs-main section.footnotes li kbd:first-child{margin-right:1em;margin-bottom:0.4em}html.theme--documenter-dark #documenter .docs-main .docs-footer{display:flex;flex-wrap:wrap;margin-left:0;margin-right:0;border-top:1px solid #5e6d6f;padding-top:1rem;padding-bottom:1rem}@media screen and (max-width: 1055px){html.theme--documenter-dark #documenter .docs-main .docs-footer{padding-left:1rem;padding-right:1rem}}html.theme--documenter-dark #documenter .docs-main .docs-footer .docs-footer-nextpage,html.theme--documenter-dark #documenter .docs-main .docs-footer .docs-footer-prevpage{flex-grow:1}html.theme--documenter-dark #documenter .docs-main .docs-footer .docs-footer-nextpage{text-align:right}html.theme--documenter-dark #documenter .docs-main .docs-footer .flexbox-break{flex-basis:100%;height:0}html.theme--documenter-dark #documenter .docs-main .docs-footer .footer-message{font-size:0.8em;margin:0.5em auto 0 auto;text-align:center}html.theme--documenter-dark #documenter .docs-sidebar{display:flex;flex-direction:column;color:#fff;background-color:#282f2f;border-right:1px solid #5e6d6f;padding:0;flex:0 0 18rem;z-index:5;font-size:1rem;position:fixed;left:-18rem;width:18rem;height:100%;transition:left 0.3s}html.theme--documenter-dark #documenter .docs-sidebar.visible{left:0;box-shadow:.4rem 0rem .8rem #171717}@media screen and (min-width: 1056px){html.theme--documenter-dark #documenter .docs-sidebar.visible{box-shadow:none}}@media screen and (min-width: 1056px){html.theme--documenter-dark #documenter .docs-sidebar{left:0;top:0}}html.theme--documenter-dark #documenter .docs-sidebar .docs-logo{margin-top:1rem;padding:0 1rem}html.theme--documenter-dark #documenter .docs-sidebar .docs-logo>img{max-height:6rem;margin:auto}html.theme--documenter-dark #documenter .docs-sidebar .docs-package-name{flex-shrink:0;font-size:1.5rem;font-weight:700;text-align:center;white-space:nowrap;overflow:hidden;padding:0.5rem 0}html.theme--documenter-dark #documenter .docs-sidebar .docs-package-name .docs-autofit{max-width:16.2rem}html.theme--documenter-dark #documenter .docs-sidebar .docs-package-name a,html.theme--documenter-dark #documenter .docs-sidebar .docs-package-name a:hover{color:#fff}html.theme--documenter-dark #documenter .docs-sidebar .docs-version-selector{border-top:1px solid #5e6d6f;display:none;padding:0.5rem}html.theme--documenter-dark #documenter .docs-sidebar .docs-version-selector.visible{display:flex}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu{flex-grow:1;user-select:none;border-top:1px solid #5e6d6f;padding-bottom:1.5rem}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu>li>.tocitem{font-weight:bold}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu>li li{font-size:.95rem;margin-left:1em;border-left:1px solid #5e6d6f}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu input.collapse-toggle{display:none}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu ul.collapsed{display:none}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu input:checked~ul.collapsed{display:block}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu label.tocitem{display:flex}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu label.tocitem .docs-label{flex-grow:2}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu label.tocitem .docs-chevron{display:inline-block;font-style:normal;font-variant:normal;text-rendering:auto;line-height:1;font-size:.75rem;margin-left:1rem;margin-top:auto;margin-bottom:auto}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu label.tocitem .docs-chevron::before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f054"}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu input:checked~label.tocitem .docs-chevron::before{content:"\f078"}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu .tocitem{display:block;padding:0.5rem 0.5rem}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu .tocitem,html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu .tocitem:hover{color:#fff;background:#282f2f}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu a.tocitem:hover,html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu label.tocitem:hover{color:#fff;background-color:#32393a}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu li.is-active{border-top:1px solid #5e6d6f;border-bottom:1px solid #5e6d6f;background-color:#1f2424}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu li.is-active .tocitem,html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu li.is-active .tocitem:hover{background-color:#1f2424;color:#fff}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu li.is-active ul.internal .tocitem:hover{background-color:#32393a;color:#fff}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu>li.is-active:first-child{border-top:none}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu ul.internal{margin:0 0.5rem 0.5rem;border-top:1px solid #5e6d6f}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu ul.internal li{font-size:.85rem;border-left:none;margin-left:0;margin-top:0.5rem}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu ul.internal .tocitem{width:100%;padding:0}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu ul.internal .tocitem::before{content:"⚬";margin-right:0.4em}html.theme--documenter-dark #documenter .docs-sidebar form.docs-search{margin:auto;margin-top:0.5rem;margin-bottom:0.5rem}html.theme--documenter-dark #documenter .docs-sidebar form.docs-search>input{width:14.4rem}html.theme--documenter-dark #documenter .docs-sidebar #documenter-search-query{color:#868c98;width:14.4rem;box-shadow:inset 0 1px 2px rgba(10,10,10,0.1)}@media screen and (min-width: 1056px){html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu{overflow-y:auto;-webkit-overflow-scroll:touch}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar{width:.3rem;background:none}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar-thumb{border-radius:5px 0px 0px 5px;background:#3b4445}html.theme--documenter-dark #documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar-thumb:hover{background:#4e5a5c}}@media screen and (max-width: 1055px){html.theme--documenter-dark #documenter .docs-sidebar{overflow-y:auto;-webkit-overflow-scroll:touch}html.theme--documenter-dark #documenter .docs-sidebar::-webkit-scrollbar{width:.3rem;background:none}html.theme--documenter-dark #documenter .docs-sidebar::-webkit-scrollbar-thumb{border-radius:5px 0px 0px 5px;background:#3b4445}html.theme--documenter-dark #documenter .docs-sidebar::-webkit-scrollbar-thumb:hover{background:#4e5a5c}}html.theme--documenter-dark kbd.search-modal-key-hints{border-radius:0.25rem;border:1px solid rgba(245,245,245,0.6);box-shadow:0 2px 0 1px rgba(245,245,245,0.6);cursor:default;font-size:0.9rem;line-height:1.5;min-width:0.75rem;text-align:center;padding:0.1rem 0.3rem;position:relative;top:-1px}html.theme--documenter-dark .search-min-width-50{min-width:50%}html.theme--documenter-dark .search-min-height-100{min-height:100%}html.theme--documenter-dark .search-modal-card-body{max-height:calc(100vh - 15rem)}html.theme--documenter-dark .search-result-link{border-radius:0.7em;transition:all 300ms}html.theme--documenter-dark .search-result-link:hover,html.theme--documenter-dark .search-result-link:focus{background-color:rgba(0,128,128,0.1)}html.theme--documenter-dark .search-result-link .property-search-result-badge,html.theme--documenter-dark .search-result-link .search-filter{transition:all 300ms}html.theme--documenter-dark .property-search-result-badge,html.theme--documenter-dark .search-filter{padding:0.15em 0.5em;font-size:0.8em;font-style:italic;text-transform:none !important;line-height:1.5;color:#f5f5f5;background-color:rgba(51,65,85,0.501961);border-radius:0.6rem}html.theme--documenter-dark .search-result-link:hover .property-search-result-badge,html.theme--documenter-dark .search-result-link:hover .search-filter,html.theme--documenter-dark .search-result-link:focus .property-search-result-badge,html.theme--documenter-dark .search-result-link:focus .search-filter{color:#333;background-color:#f1f5f9}html.theme--documenter-dark .search-filter{color:#333;background-color:#f5f5f5;transition:all 300ms}html.theme--documenter-dark .search-filter:hover,html.theme--documenter-dark .search-filter:focus{color:#333}html.theme--documenter-dark .search-filter-selected{color:#f5f5f5;background-color:rgba(139,0,139,0.5)}html.theme--documenter-dark .search-filter-selected:hover,html.theme--documenter-dark .search-filter-selected:focus{color:#f5f5f5}html.theme--documenter-dark .search-result-highlight{background-color:#ffdd57;color:black}html.theme--documenter-dark .search-divider{border-bottom:1px solid #5e6d6f}html.theme--documenter-dark .search-result-title{width:85%;color:#f5f5f5}html.theme--documenter-dark .search-result-code-title{font-size:0.875rem;font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace}html.theme--documenter-dark #search-modal .modal-card-body::-webkit-scrollbar,html.theme--documenter-dark #search-modal .filter-tabs::-webkit-scrollbar{height:10px;width:10px;background-color:transparent}html.theme--documenter-dark #search-modal .modal-card-body::-webkit-scrollbar-thumb,html.theme--documenter-dark #search-modal .filter-tabs::-webkit-scrollbar-thumb{background-color:gray;border-radius:1rem}html.theme--documenter-dark #search-modal .modal-card-body::-webkit-scrollbar-track,html.theme--documenter-dark #search-modal .filter-tabs::-webkit-scrollbar-track{-webkit-box-shadow:inset 0 0 6px rgba(0,0,0,0.6);background-color:transparent}html.theme--documenter-dark .w-100{width:100%}html.theme--documenter-dark .gap-2{gap:0.5rem}html.theme--documenter-dark .gap-4{gap:1rem}html.theme--documenter-dark .gap-8{gap:2rem}html.theme--documenter-dark{background-color:#1f2424;font-size:16px;min-width:300px;overflow-x:auto;overflow-y:scroll;text-rendering:optimizeLegibility;text-size-adjust:100%}html.theme--documenter-dark .ansi span.sgr1{font-weight:bolder}html.theme--documenter-dark .ansi span.sgr2{font-weight:lighter}html.theme--documenter-dark .ansi span.sgr3{font-style:italic}html.theme--documenter-dark .ansi span.sgr4{text-decoration:underline}html.theme--documenter-dark .ansi span.sgr7{color:#1f2424;background-color:#fff}html.theme--documenter-dark .ansi span.sgr8{color:transparent}html.theme--documenter-dark .ansi span.sgr8 span{color:transparent}html.theme--documenter-dark .ansi span.sgr9{text-decoration:line-through}html.theme--documenter-dark .ansi span.sgr30{color:#242424}html.theme--documenter-dark .ansi span.sgr31{color:#f6705f}html.theme--documenter-dark .ansi span.sgr32{color:#4fb43a}html.theme--documenter-dark .ansi span.sgr33{color:#f4c72f}html.theme--documenter-dark .ansi span.sgr34{color:#7587f0}html.theme--documenter-dark .ansi span.sgr35{color:#bc89d3}html.theme--documenter-dark .ansi span.sgr36{color:#49b6ca}html.theme--documenter-dark .ansi span.sgr37{color:#b3bdbe}html.theme--documenter-dark .ansi span.sgr40{background-color:#242424}html.theme--documenter-dark .ansi span.sgr41{background-color:#f6705f}html.theme--documenter-dark .ansi span.sgr42{background-color:#4fb43a}html.theme--documenter-dark .ansi span.sgr43{background-color:#f4c72f}html.theme--documenter-dark .ansi span.sgr44{background-color:#7587f0}html.theme--documenter-dark .ansi span.sgr45{background-color:#bc89d3}html.theme--documenter-dark .ansi span.sgr46{background-color:#49b6ca}html.theme--documenter-dark .ansi span.sgr47{background-color:#b3bdbe}html.theme--documenter-dark .ansi span.sgr90{color:#92a0a2}html.theme--documenter-dark .ansi span.sgr91{color:#ff8674}html.theme--documenter-dark .ansi span.sgr92{color:#79d462}html.theme--documenter-dark .ansi span.sgr93{color:#ffe76b}html.theme--documenter-dark .ansi span.sgr94{color:#8a98ff}html.theme--documenter-dark .ansi span.sgr95{color:#d2a4e6}html.theme--documenter-dark .ansi span.sgr96{color:#6bc8db}html.theme--documenter-dark .ansi span.sgr97{color:#ecf0f1}html.theme--documenter-dark .ansi span.sgr100{background-color:#92a0a2}html.theme--documenter-dark .ansi span.sgr101{background-color:#ff8674}html.theme--documenter-dark .ansi span.sgr102{background-color:#79d462}html.theme--documenter-dark .ansi span.sgr103{background-color:#ffe76b}html.theme--documenter-dark .ansi span.sgr104{background-color:#8a98ff}html.theme--documenter-dark .ansi span.sgr105{background-color:#d2a4e6}html.theme--documenter-dark .ansi span.sgr106{background-color:#6bc8db}html.theme--documenter-dark .ansi span.sgr107{background-color:#ecf0f1}html.theme--documenter-dark code.language-julia-repl>span.hljs-meta{color:#4fb43a;font-weight:bolder}html.theme--documenter-dark .hljs{background:#2b2b2b;color:#f8f8f2}html.theme--documenter-dark .hljs-comment,html.theme--documenter-dark .hljs-quote{color:#d4d0ab}html.theme--documenter-dark .hljs-variable,html.theme--documenter-dark .hljs-template-variable,html.theme--documenter-dark .hljs-tag,html.theme--documenter-dark .hljs-name,html.theme--documenter-dark .hljs-selector-id,html.theme--documenter-dark .hljs-selector-class,html.theme--documenter-dark .hljs-regexp,html.theme--documenter-dark .hljs-deletion{color:#ffa07a}html.theme--documenter-dark .hljs-number,html.theme--documenter-dark .hljs-built_in,html.theme--documenter-dark .hljs-literal,html.theme--documenter-dark .hljs-type,html.theme--documenter-dark .hljs-params,html.theme--documenter-dark .hljs-meta,html.theme--documenter-dark .hljs-link{color:#f5ab35}html.theme--documenter-dark .hljs-attribute{color:#ffd700}html.theme--documenter-dark .hljs-string,html.theme--documenter-dark .hljs-symbol,html.theme--documenter-dark .hljs-bullet,html.theme--documenter-dark .hljs-addition{color:#abe338}html.theme--documenter-dark .hljs-title,html.theme--documenter-dark .hljs-section{color:#00e0e0}html.theme--documenter-dark .hljs-keyword,html.theme--documenter-dark .hljs-selector-tag{color:#dcc6e0}html.theme--documenter-dark .hljs-emphasis{font-style:italic}html.theme--documenter-dark .hljs-strong{font-weight:bold}@media screen and (-ms-high-contrast: active){html.theme--documenter-dark .hljs-addition,html.theme--documenter-dark .hljs-attribute,html.theme--documenter-dark .hljs-built_in,html.theme--documenter-dark .hljs-bullet,html.theme--documenter-dark .hljs-comment,html.theme--documenter-dark .hljs-link,html.theme--documenter-dark .hljs-literal,html.theme--documenter-dark .hljs-meta,html.theme--documenter-dark .hljs-number,html.theme--documenter-dark .hljs-params,html.theme--documenter-dark .hljs-string,html.theme--documenter-dark .hljs-symbol,html.theme--documenter-dark .hljs-type,html.theme--documenter-dark .hljs-quote{color:highlight}html.theme--documenter-dark .hljs-keyword,html.theme--documenter-dark .hljs-selector-tag{font-weight:bold}}html.theme--documenter-dark .hljs-subst{color:#f8f8f2}html.theme--documenter-dark .search-result-link{border-radius:0.7em;transition:all 300ms}html.theme--documenter-dark .search-result-link:hover,html.theme--documenter-dark .search-result-link:focus{background-color:rgba(0,128,128,0.1)}html.theme--documenter-dark .search-result-link .property-search-result-badge,html.theme--documenter-dark .search-result-link .search-filter{transition:all 300ms}html.theme--documenter-dark .search-result-link:hover .property-search-result-badge,html.theme--documenter-dark .search-result-link:hover .search-filter,html.theme--documenter-dark .search-result-link:focus .property-search-result-badge,html.theme--documenter-dark .search-result-link:focus .search-filter{color:#333 !important;background-color:#f1f5f9 !important}html.theme--documenter-dark .property-search-result-badge,html.theme--documenter-dark .search-filter{padding:0.15em 0.5em;font-size:0.8em;font-style:italic;text-transform:none !important;line-height:1.5;color:whitesmoke;background-color:#33415580;border-radius:0.6rem}html.theme--documenter-dark .search-result-title{color:whitesmoke}html.theme--documenter-dark .search-result-highlight{background-color:greenyellow;color:black}html.theme--documenter-dark .search-divider{border-bottom:1px solid #5e6d6f50}html.theme--documenter-dark .w-100{width:100%}html.theme--documenter-dark .gap-2{gap:0.5rem}html.theme--documenter-dark .gap-4{gap:1rem} diff --git a/previews/PR663/assets/themes/documenter-light.css b/previews/PR663/assets/themes/documenter-light.css new file mode 100644 index 000000000..2f168c77b --- /dev/null +++ b/previews/PR663/assets/themes/documenter-light.css @@ -0,0 +1,9 @@ +.pagination-previous,.pagination-next,.pagination-link,.pagination-ellipsis,.file-cta,.file-name,.select select,.textarea,.input,#documenter .docs-sidebar form.docs-search>input,.button{-moz-appearance:none;-webkit-appearance:none;align-items:center;border:1px solid transparent;border-radius:4px;box-shadow:none;display:inline-flex;font-size:1rem;height:2.5em;justify-content:flex-start;line-height:1.5;padding-bottom:calc(0.5em - 1px);padding-left:calc(0.75em - 1px);padding-right:calc(0.75em - 1px);padding-top:calc(0.5em - 1px);position:relative;vertical-align:top}.pagination-previous:focus,.pagination-next:focus,.pagination-link:focus,.pagination-ellipsis:focus,.file-cta:focus,.file-name:focus,.select select:focus,.textarea:focus,.input:focus,#documenter .docs-sidebar form.docs-search>input:focus,.button:focus,.is-focused.pagination-previous,.is-focused.pagination-next,.is-focused.pagination-link,.is-focused.pagination-ellipsis,.is-focused.file-cta,.is-focused.file-name,.select select.is-focused,.is-focused.textarea,.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-focused.button,.pagination-previous:active,.pagination-next:active,.pagination-link:active,.pagination-ellipsis:active,.file-cta:active,.file-name:active,.select select:active,.textarea:active,.input:active,#documenter .docs-sidebar form.docs-search>input:active,.button:active,.is-active.pagination-previous,.is-active.pagination-next,.is-active.pagination-link,.is-active.pagination-ellipsis,.is-active.file-cta,.is-active.file-name,.select select.is-active,.is-active.textarea,.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active,.is-active.button{outline:none}.pagination-previous[disabled],.pagination-next[disabled],.pagination-link[disabled],.pagination-ellipsis[disabled],.file-cta[disabled],.file-name[disabled],.select select[disabled],.textarea[disabled],.input[disabled],#documenter .docs-sidebar form.docs-search>input[disabled],.button[disabled],fieldset[disabled] .pagination-previous,fieldset[disabled] .pagination-next,fieldset[disabled] .pagination-link,fieldset[disabled] .pagination-ellipsis,fieldset[disabled] .file-cta,fieldset[disabled] .file-name,fieldset[disabled] .select select,.select fieldset[disabled] select,fieldset[disabled] .textarea,fieldset[disabled] .input,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input,fieldset[disabled] .button{cursor:not-allowed}.tabs,.pagination-previous,.pagination-next,.pagination-link,.pagination-ellipsis,.breadcrumb,.file,.button,.is-unselectable{-webkit-touch-callout:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.navbar-link:not(.is-arrowless)::after,.select:not(.is-multiple):not(.is-loading)::after{border:3px solid rgba(0,0,0,0);border-radius:2px;border-right:0;border-top:0;content:" ";display:block;height:0.625em;margin-top:-0.4375em;pointer-events:none;position:absolute;top:50%;transform:rotate(-45deg);transform-origin:center;width:0.625em}.admonition:not(:last-child),.tabs:not(:last-child),.pagination:not(:last-child),.message:not(:last-child),.level:not(:last-child),.breadcrumb:not(:last-child),.block:not(:last-child),.title:not(:last-child),.subtitle:not(:last-child),.table-container:not(:last-child),.table:not(:last-child),.progress:not(:last-child),.notification:not(:last-child),.content:not(:last-child),.box:not(:last-child){margin-bottom:1.5rem}.modal-close,.delete{-webkit-touch-callout:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;-moz-appearance:none;-webkit-appearance:none;background-color:rgba(10,10,10,0.2);border:none;border-radius:9999px;cursor:pointer;pointer-events:auto;display:inline-block;flex-grow:0;flex-shrink:0;font-size:0;height:20px;max-height:20px;max-width:20px;min-height:20px;min-width:20px;outline:none;position:relative;vertical-align:top;width:20px}.modal-close::before,.delete::before,.modal-close::after,.delete::after{background-color:#fff;content:"";display:block;left:50%;position:absolute;top:50%;transform:translateX(-50%) translateY(-50%) rotate(45deg);transform-origin:center center}.modal-close::before,.delete::before{height:2px;width:50%}.modal-close::after,.delete::after{height:50%;width:2px}.modal-close:hover,.delete:hover,.modal-close:focus,.delete:focus{background-color:rgba(10,10,10,0.3)}.modal-close:active,.delete:active{background-color:rgba(10,10,10,0.4)}.is-small.modal-close,#documenter .docs-sidebar form.docs-search>input.modal-close,.is-small.delete,#documenter .docs-sidebar form.docs-search>input.delete{height:16px;max-height:16px;max-width:16px;min-height:16px;min-width:16px;width:16px}.is-medium.modal-close,.is-medium.delete{height:24px;max-height:24px;max-width:24px;min-height:24px;min-width:24px;width:24px}.is-large.modal-close,.is-large.delete{height:32px;max-height:32px;max-width:32px;min-height:32px;min-width:32px;width:32px}.control.is-loading::after,.select.is-loading::after,.loader,.button.is-loading::after{animation:spinAround 500ms infinite linear;border:2px solid #dbdbdb;border-radius:9999px;border-right-color:transparent;border-top-color:transparent;content:"";display:block;height:1em;position:relative;width:1em}.hero-video,.modal-background,.modal,.image.is-square img,#documenter .docs-sidebar .docs-logo>img.is-square img,.image.is-square .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-square .has-ratio,.image.is-1by1 img,#documenter .docs-sidebar .docs-logo>img.is-1by1 img,.image.is-1by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by1 .has-ratio,.image.is-5by4 img,#documenter .docs-sidebar .docs-logo>img.is-5by4 img,.image.is-5by4 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-5by4 .has-ratio,.image.is-4by3 img,#documenter .docs-sidebar .docs-logo>img.is-4by3 img,.image.is-4by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-4by3 .has-ratio,.image.is-3by2 img,#documenter .docs-sidebar .docs-logo>img.is-3by2 img,.image.is-3by2 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by2 .has-ratio,.image.is-5by3 img,#documenter .docs-sidebar .docs-logo>img.is-5by3 img,.image.is-5by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-5by3 .has-ratio,.image.is-16by9 img,#documenter .docs-sidebar .docs-logo>img.is-16by9 img,.image.is-16by9 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-16by9 .has-ratio,.image.is-2by1 img,#documenter .docs-sidebar .docs-logo>img.is-2by1 img,.image.is-2by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-2by1 .has-ratio,.image.is-3by1 img,#documenter .docs-sidebar .docs-logo>img.is-3by1 img,.image.is-3by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by1 .has-ratio,.image.is-4by5 img,#documenter .docs-sidebar .docs-logo>img.is-4by5 img,.image.is-4by5 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-4by5 .has-ratio,.image.is-3by4 img,#documenter .docs-sidebar .docs-logo>img.is-3by4 img,.image.is-3by4 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by4 .has-ratio,.image.is-2by3 img,#documenter .docs-sidebar .docs-logo>img.is-2by3 img,.image.is-2by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-2by3 .has-ratio,.image.is-3by5 img,#documenter .docs-sidebar .docs-logo>img.is-3by5 img,.image.is-3by5 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by5 .has-ratio,.image.is-9by16 img,#documenter .docs-sidebar .docs-logo>img.is-9by16 img,.image.is-9by16 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-9by16 .has-ratio,.image.is-1by2 img,#documenter .docs-sidebar .docs-logo>img.is-1by2 img,.image.is-1by2 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by2 .has-ratio,.image.is-1by3 img,#documenter .docs-sidebar .docs-logo>img.is-1by3 img,.image.is-1by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by3 .has-ratio,.is-overlay{bottom:0;left:0;position:absolute;right:0;top:0}.navbar-burger{-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;color:currentColor;font-family:inherit;font-size:1em;margin:0;padding:0}.has-text-white{color:#fff !important}a.has-text-white:hover,a.has-text-white:focus{color:#e6e6e6 !important}.has-background-white{background-color:#fff !important}.has-text-black{color:#0a0a0a !important}a.has-text-black:hover,a.has-text-black:focus{color:#000 !important}.has-background-black{background-color:#0a0a0a !important}.has-text-light{color:#f5f5f5 !important}a.has-text-light:hover,a.has-text-light:focus{color:#dbdbdb !important}.has-background-light{background-color:#f5f5f5 !important}.has-text-dark{color:#363636 !important}a.has-text-dark:hover,a.has-text-dark:focus{color:#1c1c1c !important}.has-background-dark{background-color:#363636 !important}.has-text-primary{color:#4eb5de !important}a.has-text-primary:hover,a.has-text-primary:focus{color:#27a1d2 !important}.has-background-primary{background-color:#4eb5de !important}.has-text-primary-light{color:#eef8fc !important}a.has-text-primary-light:hover,a.has-text-primary-light:focus{color:#c3e6f4 !important}.has-background-primary-light{background-color:#eef8fc !important}.has-text-primary-dark{color:#1a6d8e !important}a.has-text-primary-dark:hover,a.has-text-primary-dark:focus{color:#228eb9 !important}.has-background-primary-dark{background-color:#1a6d8e !important}.has-text-link{color:#2e63b8 !important}a.has-text-link:hover,a.has-text-link:focus{color:#244d8f !important}.has-background-link{background-color:#2e63b8 !important}.has-text-link-light{color:#eff3fb !important}a.has-text-link-light:hover,a.has-text-link-light:focus{color:#c6d6f1 !important}.has-background-link-light{background-color:#eff3fb !important}.has-text-link-dark{color:#3169c4 !important}a.has-text-link-dark:hover,a.has-text-link-dark:focus{color:#5485d4 !important}.has-background-link-dark{background-color:#3169c4 !important}.has-text-info{color:#209cee !important}a.has-text-info:hover,a.has-text-info:focus{color:#1081cb !important}.has-background-info{background-color:#209cee !important}.has-text-info-light{color:#ecf7fe !important}a.has-text-info-light:hover,a.has-text-info-light:focus{color:#bde2fa !important}.has-background-info-light{background-color:#ecf7fe !important}.has-text-info-dark{color:#0e72b4 !important}a.has-text-info-dark:hover,a.has-text-info-dark:focus{color:#1190e3 !important}.has-background-info-dark{background-color:#0e72b4 !important}.has-text-success{color:#22c35b !important}a.has-text-success:hover,a.has-text-success:focus{color:#1a9847 !important}.has-background-success{background-color:#22c35b !important}.has-text-success-light{color:#eefcf3 !important}a.has-text-success-light:hover,a.has-text-success-light:focus{color:#c2f4d4 !important}.has-background-success-light{background-color:#eefcf3 !important}.has-text-success-dark{color:#198f43 !important}a.has-text-success-dark:hover,a.has-text-success-dark:focus{color:#21bb57 !important}.has-background-success-dark{background-color:#198f43 !important}.has-text-warning{color:#ffdd57 !important}a.has-text-warning:hover,a.has-text-warning:focus{color:#ffd324 !important}.has-background-warning{background-color:#ffdd57 !important}.has-text-warning-light{color:#fffbeb !important}a.has-text-warning-light:hover,a.has-text-warning-light:focus{color:#fff1b8 !important}.has-background-warning-light{background-color:#fffbeb !important}.has-text-warning-dark{color:#947600 !important}a.has-text-warning-dark:hover,a.has-text-warning-dark:focus{color:#c79f00 !important}.has-background-warning-dark{background-color:#947600 !important}.has-text-danger{color:#da0b00 !important}a.has-text-danger:hover,a.has-text-danger:focus{color:#a70800 !important}.has-background-danger{background-color:#da0b00 !important}.has-text-danger-light{color:#ffeceb !important}a.has-text-danger-light:hover,a.has-text-danger-light:focus{color:#ffbbb8 !important}.has-background-danger-light{background-color:#ffeceb !important}.has-text-danger-dark{color:#f50c00 !important}a.has-text-danger-dark:hover,a.has-text-danger-dark:focus{color:#ff3429 !important}.has-background-danger-dark{background-color:#f50c00 !important}.has-text-black-bis{color:#121212 !important}.has-background-black-bis{background-color:#121212 !important}.has-text-black-ter{color:#242424 !important}.has-background-black-ter{background-color:#242424 !important}.has-text-grey-darker{color:#363636 !important}.has-background-grey-darker{background-color:#363636 !important}.has-text-grey-dark{color:#4a4a4a !important}.has-background-grey-dark{background-color:#4a4a4a !important}.has-text-grey{color:#6b6b6b !important}.has-background-grey{background-color:#6b6b6b !important}.has-text-grey-light{color:#b5b5b5 !important}.has-background-grey-light{background-color:#b5b5b5 !important}.has-text-grey-lighter{color:#dbdbdb !important}.has-background-grey-lighter{background-color:#dbdbdb !important}.has-text-white-ter{color:#f5f5f5 !important}.has-background-white-ter{background-color:#f5f5f5 !important}.has-text-white-bis{color:#fafafa !important}.has-background-white-bis{background-color:#fafafa !important}.is-flex-direction-row{flex-direction:row !important}.is-flex-direction-row-reverse{flex-direction:row-reverse !important}.is-flex-direction-column{flex-direction:column !important}.is-flex-direction-column-reverse{flex-direction:column-reverse !important}.is-flex-wrap-nowrap{flex-wrap:nowrap !important}.is-flex-wrap-wrap{flex-wrap:wrap !important}.is-flex-wrap-wrap-reverse{flex-wrap:wrap-reverse !important}.is-justify-content-flex-start{justify-content:flex-start !important}.is-justify-content-flex-end{justify-content:flex-end !important}.is-justify-content-center{justify-content:center !important}.is-justify-content-space-between{justify-content:space-between !important}.is-justify-content-space-around{justify-content:space-around !important}.is-justify-content-space-evenly{justify-content:space-evenly !important}.is-justify-content-start{justify-content:start !important}.is-justify-content-end{justify-content:end !important}.is-justify-content-left{justify-content:left !important}.is-justify-content-right{justify-content:right !important}.is-align-content-flex-start{align-content:flex-start !important}.is-align-content-flex-end{align-content:flex-end !important}.is-align-content-center{align-content:center !important}.is-align-content-space-between{align-content:space-between !important}.is-align-content-space-around{align-content:space-around !important}.is-align-content-space-evenly{align-content:space-evenly !important}.is-align-content-stretch{align-content:stretch !important}.is-align-content-start{align-content:start !important}.is-align-content-end{align-content:end !important}.is-align-content-baseline{align-content:baseline !important}.is-align-items-stretch{align-items:stretch !important}.is-align-items-flex-start{align-items:flex-start !important}.is-align-items-flex-end{align-items:flex-end !important}.is-align-items-center{align-items:center !important}.is-align-items-baseline{align-items:baseline !important}.is-align-items-start{align-items:start !important}.is-align-items-end{align-items:end !important}.is-align-items-self-start{align-items:self-start !important}.is-align-items-self-end{align-items:self-end !important}.is-align-self-auto{align-self:auto !important}.is-align-self-flex-start{align-self:flex-start !important}.is-align-self-flex-end{align-self:flex-end !important}.is-align-self-center{align-self:center !important}.is-align-self-baseline{align-self:baseline !important}.is-align-self-stretch{align-self:stretch !important}.is-flex-grow-0{flex-grow:0 !important}.is-flex-grow-1{flex-grow:1 !important}.is-flex-grow-2{flex-grow:2 !important}.is-flex-grow-3{flex-grow:3 !important}.is-flex-grow-4{flex-grow:4 !important}.is-flex-grow-5{flex-grow:5 !important}.is-flex-shrink-0{flex-shrink:0 !important}.is-flex-shrink-1{flex-shrink:1 !important}.is-flex-shrink-2{flex-shrink:2 !important}.is-flex-shrink-3{flex-shrink:3 !important}.is-flex-shrink-4{flex-shrink:4 !important}.is-flex-shrink-5{flex-shrink:5 !important}.is-clearfix::after{clear:both;content:" ";display:table}.is-pulled-left{float:left !important}.is-pulled-right{float:right !important}.is-radiusless{border-radius:0 !important}.is-shadowless{box-shadow:none !important}.is-clickable{cursor:pointer !important;pointer-events:all !important}.is-clipped{overflow:hidden !important}.is-relative{position:relative !important}.is-marginless{margin:0 !important}.is-paddingless{padding:0 !important}.m-0{margin:0 !important}.mt-0{margin-top:0 !important}.mr-0{margin-right:0 !important}.mb-0{margin-bottom:0 !important}.ml-0{margin-left:0 !important}.mx-0{margin-left:0 !important;margin-right:0 !important}.my-0{margin-top:0 !important;margin-bottom:0 !important}.m-1{margin:.25rem !important}.mt-1{margin-top:.25rem !important}.mr-1{margin-right:.25rem !important}.mb-1{margin-bottom:.25rem !important}.ml-1{margin-left:.25rem !important}.mx-1{margin-left:.25rem !important;margin-right:.25rem !important}.my-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.m-2{margin:.5rem !important}.mt-2{margin-top:.5rem !important}.mr-2{margin-right:.5rem !important}.mb-2{margin-bottom:.5rem !important}.ml-2{margin-left:.5rem !important}.mx-2{margin-left:.5rem !important;margin-right:.5rem !important}.my-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.m-3{margin:.75rem !important}.mt-3{margin-top:.75rem !important}.mr-3{margin-right:.75rem !important}.mb-3{margin-bottom:.75rem !important}.ml-3{margin-left:.75rem !important}.mx-3{margin-left:.75rem !important;margin-right:.75rem !important}.my-3{margin-top:.75rem !important;margin-bottom:.75rem !important}.m-4{margin:1rem !important}.mt-4{margin-top:1rem !important}.mr-4{margin-right:1rem !important}.mb-4{margin-bottom:1rem !important}.ml-4{margin-left:1rem !important}.mx-4{margin-left:1rem !important;margin-right:1rem !important}.my-4{margin-top:1rem !important;margin-bottom:1rem !important}.m-5{margin:1.5rem !important}.mt-5{margin-top:1.5rem !important}.mr-5{margin-right:1.5rem !important}.mb-5{margin-bottom:1.5rem !important}.ml-5{margin-left:1.5rem !important}.mx-5{margin-left:1.5rem !important;margin-right:1.5rem !important}.my-5{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.m-6{margin:3rem !important}.mt-6{margin-top:3rem !important}.mr-6{margin-right:3rem !important}.mb-6{margin-bottom:3rem !important}.ml-6{margin-left:3rem !important}.mx-6{margin-left:3rem !important;margin-right:3rem !important}.my-6{margin-top:3rem !important;margin-bottom:3rem !important}.m-auto{margin:auto !important}.mt-auto{margin-top:auto !important}.mr-auto{margin-right:auto !important}.mb-auto{margin-bottom:auto !important}.ml-auto{margin-left:auto !important}.mx-auto{margin-left:auto !important;margin-right:auto !important}.my-auto{margin-top:auto !important;margin-bottom:auto !important}.p-0{padding:0 !important}.pt-0{padding-top:0 !important}.pr-0{padding-right:0 !important}.pb-0{padding-bottom:0 !important}.pl-0{padding-left:0 !important}.px-0{padding-left:0 !important;padding-right:0 !important}.py-0{padding-top:0 !important;padding-bottom:0 !important}.p-1{padding:.25rem !important}.pt-1{padding-top:.25rem !important}.pr-1{padding-right:.25rem !important}.pb-1{padding-bottom:.25rem !important}.pl-1{padding-left:.25rem !important}.px-1{padding-left:.25rem !important;padding-right:.25rem !important}.py-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.p-2{padding:.5rem !important}.pt-2{padding-top:.5rem !important}.pr-2{padding-right:.5rem !important}.pb-2{padding-bottom:.5rem !important}.pl-2{padding-left:.5rem !important}.px-2{padding-left:.5rem !important;padding-right:.5rem !important}.py-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.p-3{padding:.75rem !important}.pt-3{padding-top:.75rem !important}.pr-3{padding-right:.75rem !important}.pb-3{padding-bottom:.75rem !important}.pl-3{padding-left:.75rem !important}.px-3{padding-left:.75rem !important;padding-right:.75rem !important}.py-3{padding-top:.75rem !important;padding-bottom:.75rem !important}.p-4{padding:1rem !important}.pt-4{padding-top:1rem !important}.pr-4{padding-right:1rem !important}.pb-4{padding-bottom:1rem !important}.pl-4{padding-left:1rem !important}.px-4{padding-left:1rem !important;padding-right:1rem !important}.py-4{padding-top:1rem !important;padding-bottom:1rem !important}.p-5{padding:1.5rem !important}.pt-5{padding-top:1.5rem !important}.pr-5{padding-right:1.5rem !important}.pb-5{padding-bottom:1.5rem !important}.pl-5{padding-left:1.5rem !important}.px-5{padding-left:1.5rem !important;padding-right:1.5rem !important}.py-5{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.p-6{padding:3rem !important}.pt-6{padding-top:3rem !important}.pr-6{padding-right:3rem !important}.pb-6{padding-bottom:3rem !important}.pl-6{padding-left:3rem !important}.px-6{padding-left:3rem !important;padding-right:3rem !important}.py-6{padding-top:3rem !important;padding-bottom:3rem !important}.p-auto{padding:auto !important}.pt-auto{padding-top:auto !important}.pr-auto{padding-right:auto !important}.pb-auto{padding-bottom:auto !important}.pl-auto{padding-left:auto !important}.px-auto{padding-left:auto !important;padding-right:auto !important}.py-auto{padding-top:auto !important;padding-bottom:auto !important}.is-size-1{font-size:3rem !important}.is-size-2{font-size:2.5rem !important}.is-size-3{font-size:2rem !important}.is-size-4{font-size:1.5rem !important}.is-size-5{font-size:1.25rem !important}.is-size-6{font-size:1rem !important}.is-size-7,.docstring>section>a.docs-sourcelink{font-size:.75rem !important}@media screen and (max-width: 768px){.is-size-1-mobile{font-size:3rem !important}.is-size-2-mobile{font-size:2.5rem !important}.is-size-3-mobile{font-size:2rem !important}.is-size-4-mobile{font-size:1.5rem !important}.is-size-5-mobile{font-size:1.25rem !important}.is-size-6-mobile{font-size:1rem !important}.is-size-7-mobile{font-size:.75rem !important}}@media screen and (min-width: 769px),print{.is-size-1-tablet{font-size:3rem !important}.is-size-2-tablet{font-size:2.5rem !important}.is-size-3-tablet{font-size:2rem !important}.is-size-4-tablet{font-size:1.5rem !important}.is-size-5-tablet{font-size:1.25rem !important}.is-size-6-tablet{font-size:1rem !important}.is-size-7-tablet{font-size:.75rem !important}}@media screen and (max-width: 1055px){.is-size-1-touch{font-size:3rem !important}.is-size-2-touch{font-size:2.5rem !important}.is-size-3-touch{font-size:2rem !important}.is-size-4-touch{font-size:1.5rem !important}.is-size-5-touch{font-size:1.25rem !important}.is-size-6-touch{font-size:1rem !important}.is-size-7-touch{font-size:.75rem !important}}@media screen and (min-width: 1056px){.is-size-1-desktop{font-size:3rem !important}.is-size-2-desktop{font-size:2.5rem !important}.is-size-3-desktop{font-size:2rem !important}.is-size-4-desktop{font-size:1.5rem !important}.is-size-5-desktop{font-size:1.25rem !important}.is-size-6-desktop{font-size:1rem !important}.is-size-7-desktop{font-size:.75rem !important}}@media screen and (min-width: 1216px){.is-size-1-widescreen{font-size:3rem !important}.is-size-2-widescreen{font-size:2.5rem !important}.is-size-3-widescreen{font-size:2rem !important}.is-size-4-widescreen{font-size:1.5rem !important}.is-size-5-widescreen{font-size:1.25rem !important}.is-size-6-widescreen{font-size:1rem !important}.is-size-7-widescreen{font-size:.75rem !important}}@media screen and (min-width: 1408px){.is-size-1-fullhd{font-size:3rem !important}.is-size-2-fullhd{font-size:2.5rem !important}.is-size-3-fullhd{font-size:2rem !important}.is-size-4-fullhd{font-size:1.5rem !important}.is-size-5-fullhd{font-size:1.25rem !important}.is-size-6-fullhd{font-size:1rem !important}.is-size-7-fullhd{font-size:.75rem !important}}.has-text-centered{text-align:center !important}.has-text-justified{text-align:justify !important}.has-text-left{text-align:left !important}.has-text-right{text-align:right !important}@media screen and (max-width: 768px){.has-text-centered-mobile{text-align:center !important}}@media screen and (min-width: 769px),print{.has-text-centered-tablet{text-align:center !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-centered-tablet-only{text-align:center !important}}@media screen and (max-width: 1055px){.has-text-centered-touch{text-align:center !important}}@media screen and (min-width: 1056px){.has-text-centered-desktop{text-align:center !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-centered-desktop-only{text-align:center !important}}@media screen and (min-width: 1216px){.has-text-centered-widescreen{text-align:center !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-centered-widescreen-only{text-align:center !important}}@media screen and (min-width: 1408px){.has-text-centered-fullhd{text-align:center !important}}@media screen and (max-width: 768px){.has-text-justified-mobile{text-align:justify !important}}@media screen and (min-width: 769px),print{.has-text-justified-tablet{text-align:justify !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-justified-tablet-only{text-align:justify !important}}@media screen and (max-width: 1055px){.has-text-justified-touch{text-align:justify !important}}@media screen and (min-width: 1056px){.has-text-justified-desktop{text-align:justify !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-justified-desktop-only{text-align:justify !important}}@media screen and (min-width: 1216px){.has-text-justified-widescreen{text-align:justify !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-justified-widescreen-only{text-align:justify !important}}@media screen and (min-width: 1408px){.has-text-justified-fullhd{text-align:justify !important}}@media screen and (max-width: 768px){.has-text-left-mobile{text-align:left !important}}@media screen and (min-width: 769px),print{.has-text-left-tablet{text-align:left !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-left-tablet-only{text-align:left !important}}@media screen and (max-width: 1055px){.has-text-left-touch{text-align:left !important}}@media screen and (min-width: 1056px){.has-text-left-desktop{text-align:left !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-left-desktop-only{text-align:left !important}}@media screen and (min-width: 1216px){.has-text-left-widescreen{text-align:left !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-left-widescreen-only{text-align:left !important}}@media screen and (min-width: 1408px){.has-text-left-fullhd{text-align:left !important}}@media screen and (max-width: 768px){.has-text-right-mobile{text-align:right !important}}@media screen and (min-width: 769px),print{.has-text-right-tablet{text-align:right !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.has-text-right-tablet-only{text-align:right !important}}@media screen and (max-width: 1055px){.has-text-right-touch{text-align:right !important}}@media screen and (min-width: 1056px){.has-text-right-desktop{text-align:right !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.has-text-right-desktop-only{text-align:right !important}}@media screen and (min-width: 1216px){.has-text-right-widescreen{text-align:right !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.has-text-right-widescreen-only{text-align:right !important}}@media screen and (min-width: 1408px){.has-text-right-fullhd{text-align:right !important}}.is-capitalized{text-transform:capitalize !important}.is-lowercase{text-transform:lowercase !important}.is-uppercase{text-transform:uppercase !important}.is-italic{font-style:italic !important}.is-underlined{text-decoration:underline !important}.has-text-weight-light{font-weight:300 !important}.has-text-weight-normal{font-weight:400 !important}.has-text-weight-medium{font-weight:500 !important}.has-text-weight-semibold{font-weight:600 !important}.has-text-weight-bold{font-weight:700 !important}.is-family-primary{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-secondary{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-sans-serif{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif !important}.is-family-monospace{font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace !important}.is-family-code{font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace !important}.is-block{display:block !important}@media screen and (max-width: 768px){.is-block-mobile{display:block !important}}@media screen and (min-width: 769px),print{.is-block-tablet{display:block !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-block-tablet-only{display:block !important}}@media screen and (max-width: 1055px){.is-block-touch{display:block !important}}@media screen and (min-width: 1056px){.is-block-desktop{display:block !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-block-desktop-only{display:block !important}}@media screen and (min-width: 1216px){.is-block-widescreen{display:block !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-block-widescreen-only{display:block !important}}@media screen and (min-width: 1408px){.is-block-fullhd{display:block !important}}.is-flex{display:flex !important}@media screen and (max-width: 768px){.is-flex-mobile{display:flex !important}}@media screen and (min-width: 769px),print{.is-flex-tablet{display:flex !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-flex-tablet-only{display:flex !important}}@media screen and (max-width: 1055px){.is-flex-touch{display:flex !important}}@media screen and (min-width: 1056px){.is-flex-desktop{display:flex !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-flex-desktop-only{display:flex !important}}@media screen and (min-width: 1216px){.is-flex-widescreen{display:flex !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-flex-widescreen-only{display:flex !important}}@media screen and (min-width: 1408px){.is-flex-fullhd{display:flex !important}}.is-inline{display:inline !important}@media screen and (max-width: 768px){.is-inline-mobile{display:inline !important}}@media screen and (min-width: 769px),print{.is-inline-tablet{display:inline !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-tablet-only{display:inline !important}}@media screen and (max-width: 1055px){.is-inline-touch{display:inline !important}}@media screen and (min-width: 1056px){.is-inline-desktop{display:inline !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-desktop-only{display:inline !important}}@media screen and (min-width: 1216px){.is-inline-widescreen{display:inline !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-widescreen-only{display:inline !important}}@media screen and (min-width: 1408px){.is-inline-fullhd{display:inline !important}}.is-inline-block{display:inline-block !important}@media screen and (max-width: 768px){.is-inline-block-mobile{display:inline-block !important}}@media screen and (min-width: 769px),print{.is-inline-block-tablet{display:inline-block !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-block-tablet-only{display:inline-block !important}}@media screen and (max-width: 1055px){.is-inline-block-touch{display:inline-block !important}}@media screen and (min-width: 1056px){.is-inline-block-desktop{display:inline-block !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-block-desktop-only{display:inline-block !important}}@media screen and (min-width: 1216px){.is-inline-block-widescreen{display:inline-block !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-block-widescreen-only{display:inline-block !important}}@media screen and (min-width: 1408px){.is-inline-block-fullhd{display:inline-block !important}}.is-inline-flex{display:inline-flex !important}@media screen and (max-width: 768px){.is-inline-flex-mobile{display:inline-flex !important}}@media screen and (min-width: 769px),print{.is-inline-flex-tablet{display:inline-flex !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-inline-flex-tablet-only{display:inline-flex !important}}@media screen and (max-width: 1055px){.is-inline-flex-touch{display:inline-flex !important}}@media screen and (min-width: 1056px){.is-inline-flex-desktop{display:inline-flex !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-inline-flex-desktop-only{display:inline-flex !important}}@media screen and (min-width: 1216px){.is-inline-flex-widescreen{display:inline-flex !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-inline-flex-widescreen-only{display:inline-flex !important}}@media screen and (min-width: 1408px){.is-inline-flex-fullhd{display:inline-flex !important}}.is-hidden{display:none !important}.is-sr-only{border:none !important;clip:rect(0, 0, 0, 0) !important;height:0.01em !important;overflow:hidden !important;padding:0 !important;position:absolute !important;white-space:nowrap !important;width:0.01em !important}@media screen and (max-width: 768px){.is-hidden-mobile{display:none !important}}@media screen and (min-width: 769px),print{.is-hidden-tablet{display:none !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-hidden-tablet-only{display:none !important}}@media screen and (max-width: 1055px){.is-hidden-touch{display:none !important}}@media screen and (min-width: 1056px){.is-hidden-desktop{display:none !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-hidden-desktop-only{display:none !important}}@media screen and (min-width: 1216px){.is-hidden-widescreen{display:none !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-hidden-widescreen-only{display:none !important}}@media screen and (min-width: 1408px){.is-hidden-fullhd{display:none !important}}.is-invisible{visibility:hidden !important}@media screen and (max-width: 768px){.is-invisible-mobile{visibility:hidden !important}}@media screen and (min-width: 769px),print{.is-invisible-tablet{visibility:hidden !important}}@media screen and (min-width: 769px) and (max-width: 1055px){.is-invisible-tablet-only{visibility:hidden !important}}@media screen and (max-width: 1055px){.is-invisible-touch{visibility:hidden !important}}@media screen and (min-width: 1056px){.is-invisible-desktop{visibility:hidden !important}}@media screen and (min-width: 1056px) and (max-width: 1215px){.is-invisible-desktop-only{visibility:hidden !important}}@media screen and (min-width: 1216px){.is-invisible-widescreen{visibility:hidden !important}}@media screen and (min-width: 1216px) and (max-width: 1407px){.is-invisible-widescreen-only{visibility:hidden !important}}@media screen and (min-width: 1408px){.is-invisible-fullhd{visibility:hidden !important}}/*! minireset.css v0.0.6 | MIT License | github.com/jgthms/minireset.css */html,body,p,ol,ul,li,dl,dt,dd,blockquote,figure,fieldset,legend,textarea,pre,iframe,hr,h1,h2,h3,h4,h5,h6{margin:0;padding:0}h1,h2,h3,h4,h5,h6{font-size:100%;font-weight:normal}ul{list-style:none}button,input,select,textarea{margin:0}html{box-sizing:border-box}*,*::before,*::after{box-sizing:inherit}img,video{height:auto;max-width:100%}iframe{border:0}table{border-collapse:collapse;border-spacing:0}td,th{padding:0}td:not([align]),th:not([align]){text-align:inherit}html{background-color:#fff;font-size:16px;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased;min-width:300px;overflow-x:auto;overflow-y:scroll;text-rendering:optimizeLegibility;text-size-adjust:100%}article,aside,figure,footer,header,hgroup,section{display:block}body,button,input,optgroup,select,textarea{font-family:"Lato Medium",-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue","Helvetica","Arial",sans-serif}code,pre{-moz-osx-font-smoothing:auto;-webkit-font-smoothing:auto;font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace}body{color:#222;font-size:1em;font-weight:400;line-height:1.5}a{color:#2e63b8;cursor:pointer;text-decoration:none}a strong{color:currentColor}a:hover{color:#363636}code{background-color:rgba(0,0,0,0.05);color:#000;font-size:.875em;font-weight:normal;padding:.1em}hr{background-color:#f5f5f5;border:none;display:block;height:2px;margin:1.5rem 0}img{height:auto;max-width:100%}input[type="checkbox"],input[type="radio"]{vertical-align:baseline}small{font-size:.875em}span{font-style:inherit;font-weight:inherit}strong{color:#222;font-weight:700}fieldset{border:none}pre{-webkit-overflow-scrolling:touch;background-color:#f5f5f5;color:#222;font-size:.875em;overflow-x:auto;padding:1.25rem 1.5rem;white-space:pre;word-wrap:normal}pre code{background-color:transparent;color:currentColor;font-size:1em;padding:0}table td,table th{vertical-align:top}table td:not([align]),table th:not([align]){text-align:inherit}table th{color:#222}@keyframes spinAround{from{transform:rotate(0deg)}to{transform:rotate(359deg)}}.box{background-color:#fff;border-radius:6px;box-shadow:#bbb;color:#222;display:block;padding:1.25rem}a.box:hover,a.box:focus{box-shadow:0 0.5em 1em -0.125em rgba(10,10,10,0.1),0 0 0 1px #2e63b8}a.box:active{box-shadow:inset 0 1px 2px rgba(10,10,10,0.2),0 0 0 1px #2e63b8}.button{background-color:#fff;border-color:#dbdbdb;border-width:1px;color:#222;cursor:pointer;justify-content:center;padding-bottom:calc(0.5em - 1px);padding-left:1em;padding-right:1em;padding-top:calc(0.5em - 1px);text-align:center;white-space:nowrap}.button strong{color:inherit}.button .icon,.button .icon.is-small,.button #documenter .docs-sidebar form.docs-search>input.icon,#documenter .docs-sidebar .button form.docs-search>input.icon,.button .icon.is-medium,.button .icon.is-large{height:1.5em;width:1.5em}.button .icon:first-child:not(:last-child){margin-left:calc(-0.5em - 1px);margin-right:.25em}.button .icon:last-child:not(:first-child){margin-left:.25em;margin-right:calc(-0.5em - 1px)}.button .icon:first-child:last-child{margin-left:calc(-0.5em - 1px);margin-right:calc(-0.5em - 1px)}.button:hover,.button.is-hovered{border-color:#b5b5b5;color:#363636}.button:focus,.button.is-focused{border-color:#3c5dcd;color:#363636}.button:focus:not(:active),.button.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(46,99,184,0.25)}.button:active,.button.is-active{border-color:#4a4a4a;color:#363636}.button.is-text{background-color:transparent;border-color:transparent;color:#222;text-decoration:underline}.button.is-text:hover,.button.is-text.is-hovered,.button.is-text:focus,.button.is-text.is-focused{background-color:#f5f5f5;color:#222}.button.is-text:active,.button.is-text.is-active{background-color:#e8e8e8;color:#222}.button.is-text[disabled],fieldset[disabled] .button.is-text{background-color:transparent;border-color:transparent;box-shadow:none}.button.is-ghost{background:none;border-color:rgba(0,0,0,0);color:#2e63b8;text-decoration:none}.button.is-ghost:hover,.button.is-ghost.is-hovered{color:#2e63b8;text-decoration:underline}.button.is-white{background-color:#fff;border-color:transparent;color:#0a0a0a}.button.is-white:hover,.button.is-white.is-hovered{background-color:#f9f9f9;border-color:transparent;color:#0a0a0a}.button.is-white:focus,.button.is-white.is-focused{border-color:transparent;color:#0a0a0a}.button.is-white:focus:not(:active),.button.is-white.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}.button.is-white:active,.button.is-white.is-active{background-color:#f2f2f2;border-color:transparent;color:#0a0a0a}.button.is-white[disabled],fieldset[disabled] .button.is-white{background-color:#fff;border-color:#fff;box-shadow:none}.button.is-white.is-inverted{background-color:#0a0a0a;color:#fff}.button.is-white.is-inverted:hover,.button.is-white.is-inverted.is-hovered{background-color:#000}.button.is-white.is-inverted[disabled],fieldset[disabled] .button.is-white.is-inverted{background-color:#0a0a0a;border-color:transparent;box-shadow:none;color:#fff}.button.is-white.is-loading::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}.button.is-white.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-white.is-outlined:hover,.button.is-white.is-outlined.is-hovered,.button.is-white.is-outlined:focus,.button.is-white.is-outlined.is-focused{background-color:#fff;border-color:#fff;color:#0a0a0a}.button.is-white.is-outlined.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-white.is-outlined.is-loading:hover::after,.button.is-white.is-outlined.is-loading.is-hovered::after,.button.is-white.is-outlined.is-loading:focus::after,.button.is-white.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}.button.is-white.is-outlined[disabled],fieldset[disabled] .button.is-white.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-white.is-inverted.is-outlined{background-color:transparent;border-color:#0a0a0a;color:#0a0a0a}.button.is-white.is-inverted.is-outlined:hover,.button.is-white.is-inverted.is-outlined.is-hovered,.button.is-white.is-inverted.is-outlined:focus,.button.is-white.is-inverted.is-outlined.is-focused{background-color:#0a0a0a;color:#fff}.button.is-white.is-inverted.is-outlined.is-loading:hover::after,.button.is-white.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-white.is-inverted.is-outlined.is-loading:focus::after,.button.is-white.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-white.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-white.is-inverted.is-outlined{background-color:transparent;border-color:#0a0a0a;box-shadow:none;color:#0a0a0a}.button.is-black{background-color:#0a0a0a;border-color:transparent;color:#fff}.button.is-black:hover,.button.is-black.is-hovered{background-color:#040404;border-color:transparent;color:#fff}.button.is-black:focus,.button.is-black.is-focused{border-color:transparent;color:#fff}.button.is-black:focus:not(:active),.button.is-black.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}.button.is-black:active,.button.is-black.is-active{background-color:#000;border-color:transparent;color:#fff}.button.is-black[disabled],fieldset[disabled] .button.is-black{background-color:#0a0a0a;border-color:#0a0a0a;box-shadow:none}.button.is-black.is-inverted{background-color:#fff;color:#0a0a0a}.button.is-black.is-inverted:hover,.button.is-black.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-black.is-inverted[disabled],fieldset[disabled] .button.is-black.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#0a0a0a}.button.is-black.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-black.is-outlined{background-color:transparent;border-color:#0a0a0a;color:#0a0a0a}.button.is-black.is-outlined:hover,.button.is-black.is-outlined.is-hovered,.button.is-black.is-outlined:focus,.button.is-black.is-outlined.is-focused{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}.button.is-black.is-outlined.is-loading::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}.button.is-black.is-outlined.is-loading:hover::after,.button.is-black.is-outlined.is-loading.is-hovered::after,.button.is-black.is-outlined.is-loading:focus::after,.button.is-black.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-black.is-outlined[disabled],fieldset[disabled] .button.is-black.is-outlined{background-color:transparent;border-color:#0a0a0a;box-shadow:none;color:#0a0a0a}.button.is-black.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-black.is-inverted.is-outlined:hover,.button.is-black.is-inverted.is-outlined.is-hovered,.button.is-black.is-inverted.is-outlined:focus,.button.is-black.is-inverted.is-outlined.is-focused{background-color:#fff;color:#0a0a0a}.button.is-black.is-inverted.is-outlined.is-loading:hover::after,.button.is-black.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-black.is-inverted.is-outlined.is-loading:focus::after,.button.is-black.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #0a0a0a #0a0a0a !important}.button.is-black.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-black.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-light{background-color:#f5f5f5;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-light:hover,.button.is-light.is-hovered{background-color:#eee;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-light:focus,.button.is-light.is-focused{border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-light:focus:not(:active),.button.is-light.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(245,245,245,0.25)}.button.is-light:active,.button.is-light.is-active{background-color:#e8e8e8;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-light[disabled],fieldset[disabled] .button.is-light{background-color:#f5f5f5;border-color:#f5f5f5;box-shadow:none}.button.is-light.is-inverted{background-color:rgba(0,0,0,0.7);color:#f5f5f5}.button.is-light.is-inverted:hover,.button.is-light.is-inverted.is-hovered{background-color:rgba(0,0,0,0.7)}.button.is-light.is-inverted[disabled],fieldset[disabled] .button.is-light.is-inverted{background-color:rgba(0,0,0,0.7);border-color:transparent;box-shadow:none;color:#f5f5f5}.button.is-light.is-loading::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}.button.is-light.is-outlined{background-color:transparent;border-color:#f5f5f5;color:#f5f5f5}.button.is-light.is-outlined:hover,.button.is-light.is-outlined.is-hovered,.button.is-light.is-outlined:focus,.button.is-light.is-outlined.is-focused{background-color:#f5f5f5;border-color:#f5f5f5;color:rgba(0,0,0,0.7)}.button.is-light.is-outlined.is-loading::after{border-color:transparent transparent #f5f5f5 #f5f5f5 !important}.button.is-light.is-outlined.is-loading:hover::after,.button.is-light.is-outlined.is-loading.is-hovered::after,.button.is-light.is-outlined.is-loading:focus::after,.button.is-light.is-outlined.is-loading.is-focused::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}.button.is-light.is-outlined[disabled],fieldset[disabled] .button.is-light.is-outlined{background-color:transparent;border-color:#f5f5f5;box-shadow:none;color:#f5f5f5}.button.is-light.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);color:rgba(0,0,0,0.7)}.button.is-light.is-inverted.is-outlined:hover,.button.is-light.is-inverted.is-outlined.is-hovered,.button.is-light.is-inverted.is-outlined:focus,.button.is-light.is-inverted.is-outlined.is-focused{background-color:rgba(0,0,0,0.7);color:#f5f5f5}.button.is-light.is-inverted.is-outlined.is-loading:hover::after,.button.is-light.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-light.is-inverted.is-outlined.is-loading:focus::after,.button.is-light.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #f5f5f5 #f5f5f5 !important}.button.is-light.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-light.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);box-shadow:none;color:rgba(0,0,0,0.7)}.button.is-dark,.content kbd.button{background-color:#363636;border-color:transparent;color:#fff}.button.is-dark:hover,.content kbd.button:hover,.button.is-dark.is-hovered,.content kbd.button.is-hovered{background-color:#2f2f2f;border-color:transparent;color:#fff}.button.is-dark:focus,.content kbd.button:focus,.button.is-dark.is-focused,.content kbd.button.is-focused{border-color:transparent;color:#fff}.button.is-dark:focus:not(:active),.content kbd.button:focus:not(:active),.button.is-dark.is-focused:not(:active),.content kbd.button.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(54,54,54,0.25)}.button.is-dark:active,.content kbd.button:active,.button.is-dark.is-active,.content kbd.button.is-active{background-color:#292929;border-color:transparent;color:#fff}.button.is-dark[disabled],.content kbd.button[disabled],fieldset[disabled] .button.is-dark,fieldset[disabled] .content kbd.button,.content fieldset[disabled] kbd.button{background-color:#363636;border-color:#363636;box-shadow:none}.button.is-dark.is-inverted,.content kbd.button.is-inverted{background-color:#fff;color:#363636}.button.is-dark.is-inverted:hover,.content kbd.button.is-inverted:hover,.button.is-dark.is-inverted.is-hovered,.content kbd.button.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-dark.is-inverted[disabled],.content kbd.button.is-inverted[disabled],fieldset[disabled] .button.is-dark.is-inverted,fieldset[disabled] .content kbd.button.is-inverted,.content fieldset[disabled] kbd.button.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#363636}.button.is-dark.is-loading::after,.content kbd.button.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-dark.is-outlined,.content kbd.button.is-outlined{background-color:transparent;border-color:#363636;color:#363636}.button.is-dark.is-outlined:hover,.content kbd.button.is-outlined:hover,.button.is-dark.is-outlined.is-hovered,.content kbd.button.is-outlined.is-hovered,.button.is-dark.is-outlined:focus,.content kbd.button.is-outlined:focus,.button.is-dark.is-outlined.is-focused,.content kbd.button.is-outlined.is-focused{background-color:#363636;border-color:#363636;color:#fff}.button.is-dark.is-outlined.is-loading::after,.content kbd.button.is-outlined.is-loading::after{border-color:transparent transparent #363636 #363636 !important}.button.is-dark.is-outlined.is-loading:hover::after,.content kbd.button.is-outlined.is-loading:hover::after,.button.is-dark.is-outlined.is-loading.is-hovered::after,.content kbd.button.is-outlined.is-loading.is-hovered::after,.button.is-dark.is-outlined.is-loading:focus::after,.content kbd.button.is-outlined.is-loading:focus::after,.button.is-dark.is-outlined.is-loading.is-focused::after,.content kbd.button.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-dark.is-outlined[disabled],.content kbd.button.is-outlined[disabled],fieldset[disabled] .button.is-dark.is-outlined,fieldset[disabled] .content kbd.button.is-outlined,.content fieldset[disabled] kbd.button.is-outlined{background-color:transparent;border-color:#363636;box-shadow:none;color:#363636}.button.is-dark.is-inverted.is-outlined,.content kbd.button.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-dark.is-inverted.is-outlined:hover,.content kbd.button.is-inverted.is-outlined:hover,.button.is-dark.is-inverted.is-outlined.is-hovered,.content kbd.button.is-inverted.is-outlined.is-hovered,.button.is-dark.is-inverted.is-outlined:focus,.content kbd.button.is-inverted.is-outlined:focus,.button.is-dark.is-inverted.is-outlined.is-focused,.content kbd.button.is-inverted.is-outlined.is-focused{background-color:#fff;color:#363636}.button.is-dark.is-inverted.is-outlined.is-loading:hover::after,.content kbd.button.is-inverted.is-outlined.is-loading:hover::after,.button.is-dark.is-inverted.is-outlined.is-loading.is-hovered::after,.content kbd.button.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-dark.is-inverted.is-outlined.is-loading:focus::after,.content kbd.button.is-inverted.is-outlined.is-loading:focus::after,.button.is-dark.is-inverted.is-outlined.is-loading.is-focused::after,.content kbd.button.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #363636 #363636 !important}.button.is-dark.is-inverted.is-outlined[disabled],.content kbd.button.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-dark.is-inverted.is-outlined,fieldset[disabled] .content kbd.button.is-inverted.is-outlined,.content fieldset[disabled] kbd.button.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-primary,.docstring>section>a.button.docs-sourcelink{background-color:#4eb5de;border-color:transparent;color:#fff}.button.is-primary:hover,.docstring>section>a.button.docs-sourcelink:hover,.button.is-primary.is-hovered,.docstring>section>a.button.is-hovered.docs-sourcelink{background-color:#43b1dc;border-color:transparent;color:#fff}.button.is-primary:focus,.docstring>section>a.button.docs-sourcelink:focus,.button.is-primary.is-focused,.docstring>section>a.button.is-focused.docs-sourcelink{border-color:transparent;color:#fff}.button.is-primary:focus:not(:active),.docstring>section>a.button.docs-sourcelink:focus:not(:active),.button.is-primary.is-focused:not(:active),.docstring>section>a.button.is-focused.docs-sourcelink:not(:active){box-shadow:0 0 0 0.125em rgba(78,181,222,0.25)}.button.is-primary:active,.docstring>section>a.button.docs-sourcelink:active,.button.is-primary.is-active,.docstring>section>a.button.is-active.docs-sourcelink{background-color:#39acda;border-color:transparent;color:#fff}.button.is-primary[disabled],.docstring>section>a.button.docs-sourcelink[disabled],fieldset[disabled] .button.is-primary,fieldset[disabled] .docstring>section>a.button.docs-sourcelink{background-color:#4eb5de;border-color:#4eb5de;box-shadow:none}.button.is-primary.is-inverted,.docstring>section>a.button.is-inverted.docs-sourcelink{background-color:#fff;color:#4eb5de}.button.is-primary.is-inverted:hover,.docstring>section>a.button.is-inverted.docs-sourcelink:hover,.button.is-primary.is-inverted.is-hovered,.docstring>section>a.button.is-inverted.is-hovered.docs-sourcelink{background-color:#f2f2f2}.button.is-primary.is-inverted[disabled],.docstring>section>a.button.is-inverted.docs-sourcelink[disabled],fieldset[disabled] .button.is-primary.is-inverted,fieldset[disabled] .docstring>section>a.button.is-inverted.docs-sourcelink{background-color:#fff;border-color:transparent;box-shadow:none;color:#4eb5de}.button.is-primary.is-loading::after,.docstring>section>a.button.is-loading.docs-sourcelink::after{border-color:transparent transparent #fff #fff !important}.button.is-primary.is-outlined,.docstring>section>a.button.is-outlined.docs-sourcelink{background-color:transparent;border-color:#4eb5de;color:#4eb5de}.button.is-primary.is-outlined:hover,.docstring>section>a.button.is-outlined.docs-sourcelink:hover,.button.is-primary.is-outlined.is-hovered,.docstring>section>a.button.is-outlined.is-hovered.docs-sourcelink,.button.is-primary.is-outlined:focus,.docstring>section>a.button.is-outlined.docs-sourcelink:focus,.button.is-primary.is-outlined.is-focused,.docstring>section>a.button.is-outlined.is-focused.docs-sourcelink{background-color:#4eb5de;border-color:#4eb5de;color:#fff}.button.is-primary.is-outlined.is-loading::after,.docstring>section>a.button.is-outlined.is-loading.docs-sourcelink::after{border-color:transparent transparent #4eb5de #4eb5de !important}.button.is-primary.is-outlined.is-loading:hover::after,.docstring>section>a.button.is-outlined.is-loading.docs-sourcelink:hover::after,.button.is-primary.is-outlined.is-loading.is-hovered::after,.docstring>section>a.button.is-outlined.is-loading.is-hovered.docs-sourcelink::after,.button.is-primary.is-outlined.is-loading:focus::after,.docstring>section>a.button.is-outlined.is-loading.docs-sourcelink:focus::after,.button.is-primary.is-outlined.is-loading.is-focused::after,.docstring>section>a.button.is-outlined.is-loading.is-focused.docs-sourcelink::after{border-color:transparent transparent #fff #fff !important}.button.is-primary.is-outlined[disabled],.docstring>section>a.button.is-outlined.docs-sourcelink[disabled],fieldset[disabled] .button.is-primary.is-outlined,fieldset[disabled] .docstring>section>a.button.is-outlined.docs-sourcelink{background-color:transparent;border-color:#4eb5de;box-shadow:none;color:#4eb5de}.button.is-primary.is-inverted.is-outlined,.docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink{background-color:transparent;border-color:#fff;color:#fff}.button.is-primary.is-inverted.is-outlined:hover,.docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink:hover,.button.is-primary.is-inverted.is-outlined.is-hovered,.docstring>section>a.button.is-inverted.is-outlined.is-hovered.docs-sourcelink,.button.is-primary.is-inverted.is-outlined:focus,.docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink:focus,.button.is-primary.is-inverted.is-outlined.is-focused,.docstring>section>a.button.is-inverted.is-outlined.is-focused.docs-sourcelink{background-color:#fff;color:#4eb5de}.button.is-primary.is-inverted.is-outlined.is-loading:hover::after,.docstring>section>a.button.is-inverted.is-outlined.is-loading.docs-sourcelink:hover::after,.button.is-primary.is-inverted.is-outlined.is-loading.is-hovered::after,.docstring>section>a.button.is-inverted.is-outlined.is-loading.is-hovered.docs-sourcelink::after,.button.is-primary.is-inverted.is-outlined.is-loading:focus::after,.docstring>section>a.button.is-inverted.is-outlined.is-loading.docs-sourcelink:focus::after,.button.is-primary.is-inverted.is-outlined.is-loading.is-focused::after,.docstring>section>a.button.is-inverted.is-outlined.is-loading.is-focused.docs-sourcelink::after{border-color:transparent transparent #4eb5de #4eb5de !important}.button.is-primary.is-inverted.is-outlined[disabled],.docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink[disabled],fieldset[disabled] .button.is-primary.is-inverted.is-outlined,fieldset[disabled] .docstring>section>a.button.is-inverted.is-outlined.docs-sourcelink{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-primary.is-light,.docstring>section>a.button.is-light.docs-sourcelink{background-color:#eef8fc;color:#1a6d8e}.button.is-primary.is-light:hover,.docstring>section>a.button.is-light.docs-sourcelink:hover,.button.is-primary.is-light.is-hovered,.docstring>section>a.button.is-light.is-hovered.docs-sourcelink{background-color:#e3f3fa;border-color:transparent;color:#1a6d8e}.button.is-primary.is-light:active,.docstring>section>a.button.is-light.docs-sourcelink:active,.button.is-primary.is-light.is-active,.docstring>section>a.button.is-light.is-active.docs-sourcelink{background-color:#d8eff8;border-color:transparent;color:#1a6d8e}.button.is-link{background-color:#2e63b8;border-color:transparent;color:#fff}.button.is-link:hover,.button.is-link.is-hovered{background-color:#2b5eae;border-color:transparent;color:#fff}.button.is-link:focus,.button.is-link.is-focused{border-color:transparent;color:#fff}.button.is-link:focus:not(:active),.button.is-link.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(46,99,184,0.25)}.button.is-link:active,.button.is-link.is-active{background-color:#2958a4;border-color:transparent;color:#fff}.button.is-link[disabled],fieldset[disabled] .button.is-link{background-color:#2e63b8;border-color:#2e63b8;box-shadow:none}.button.is-link.is-inverted{background-color:#fff;color:#2e63b8}.button.is-link.is-inverted:hover,.button.is-link.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-link.is-inverted[disabled],fieldset[disabled] .button.is-link.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#2e63b8}.button.is-link.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-link.is-outlined{background-color:transparent;border-color:#2e63b8;color:#2e63b8}.button.is-link.is-outlined:hover,.button.is-link.is-outlined.is-hovered,.button.is-link.is-outlined:focus,.button.is-link.is-outlined.is-focused{background-color:#2e63b8;border-color:#2e63b8;color:#fff}.button.is-link.is-outlined.is-loading::after{border-color:transparent transparent #2e63b8 #2e63b8 !important}.button.is-link.is-outlined.is-loading:hover::after,.button.is-link.is-outlined.is-loading.is-hovered::after,.button.is-link.is-outlined.is-loading:focus::after,.button.is-link.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-link.is-outlined[disabled],fieldset[disabled] .button.is-link.is-outlined{background-color:transparent;border-color:#2e63b8;box-shadow:none;color:#2e63b8}.button.is-link.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-link.is-inverted.is-outlined:hover,.button.is-link.is-inverted.is-outlined.is-hovered,.button.is-link.is-inverted.is-outlined:focus,.button.is-link.is-inverted.is-outlined.is-focused{background-color:#fff;color:#2e63b8}.button.is-link.is-inverted.is-outlined.is-loading:hover::after,.button.is-link.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-link.is-inverted.is-outlined.is-loading:focus::after,.button.is-link.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #2e63b8 #2e63b8 !important}.button.is-link.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-link.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-link.is-light{background-color:#eff3fb;color:#3169c4}.button.is-link.is-light:hover,.button.is-link.is-light.is-hovered{background-color:#e4ecf8;border-color:transparent;color:#3169c4}.button.is-link.is-light:active,.button.is-link.is-light.is-active{background-color:#dae5f6;border-color:transparent;color:#3169c4}.button.is-info{background-color:#209cee;border-color:transparent;color:#fff}.button.is-info:hover,.button.is-info.is-hovered{background-color:#1497ed;border-color:transparent;color:#fff}.button.is-info:focus,.button.is-info.is-focused{border-color:transparent;color:#fff}.button.is-info:focus:not(:active),.button.is-info.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(32,156,238,0.25)}.button.is-info:active,.button.is-info.is-active{background-color:#1190e3;border-color:transparent;color:#fff}.button.is-info[disabled],fieldset[disabled] .button.is-info{background-color:#209cee;border-color:#209cee;box-shadow:none}.button.is-info.is-inverted{background-color:#fff;color:#209cee}.button.is-info.is-inverted:hover,.button.is-info.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-info.is-inverted[disabled],fieldset[disabled] .button.is-info.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#209cee}.button.is-info.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-info.is-outlined{background-color:transparent;border-color:#209cee;color:#209cee}.button.is-info.is-outlined:hover,.button.is-info.is-outlined.is-hovered,.button.is-info.is-outlined:focus,.button.is-info.is-outlined.is-focused{background-color:#209cee;border-color:#209cee;color:#fff}.button.is-info.is-outlined.is-loading::after{border-color:transparent transparent #209cee #209cee !important}.button.is-info.is-outlined.is-loading:hover::after,.button.is-info.is-outlined.is-loading.is-hovered::after,.button.is-info.is-outlined.is-loading:focus::after,.button.is-info.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-info.is-outlined[disabled],fieldset[disabled] .button.is-info.is-outlined{background-color:transparent;border-color:#209cee;box-shadow:none;color:#209cee}.button.is-info.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-info.is-inverted.is-outlined:hover,.button.is-info.is-inverted.is-outlined.is-hovered,.button.is-info.is-inverted.is-outlined:focus,.button.is-info.is-inverted.is-outlined.is-focused{background-color:#fff;color:#209cee}.button.is-info.is-inverted.is-outlined.is-loading:hover::after,.button.is-info.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-info.is-inverted.is-outlined.is-loading:focus::after,.button.is-info.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #209cee #209cee !important}.button.is-info.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-info.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-info.is-light{background-color:#ecf7fe;color:#0e72b4}.button.is-info.is-light:hover,.button.is-info.is-light.is-hovered{background-color:#e0f1fd;border-color:transparent;color:#0e72b4}.button.is-info.is-light:active,.button.is-info.is-light.is-active{background-color:#d4ecfc;border-color:transparent;color:#0e72b4}.button.is-success{background-color:#22c35b;border-color:transparent;color:#fff}.button.is-success:hover,.button.is-success.is-hovered{background-color:#20b856;border-color:transparent;color:#fff}.button.is-success:focus,.button.is-success.is-focused{border-color:transparent;color:#fff}.button.is-success:focus:not(:active),.button.is-success.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(34,195,91,0.25)}.button.is-success:active,.button.is-success.is-active{background-color:#1ead51;border-color:transparent;color:#fff}.button.is-success[disabled],fieldset[disabled] .button.is-success{background-color:#22c35b;border-color:#22c35b;box-shadow:none}.button.is-success.is-inverted{background-color:#fff;color:#22c35b}.button.is-success.is-inverted:hover,.button.is-success.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-success.is-inverted[disabled],fieldset[disabled] .button.is-success.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#22c35b}.button.is-success.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-success.is-outlined{background-color:transparent;border-color:#22c35b;color:#22c35b}.button.is-success.is-outlined:hover,.button.is-success.is-outlined.is-hovered,.button.is-success.is-outlined:focus,.button.is-success.is-outlined.is-focused{background-color:#22c35b;border-color:#22c35b;color:#fff}.button.is-success.is-outlined.is-loading::after{border-color:transparent transparent #22c35b #22c35b !important}.button.is-success.is-outlined.is-loading:hover::after,.button.is-success.is-outlined.is-loading.is-hovered::after,.button.is-success.is-outlined.is-loading:focus::after,.button.is-success.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-success.is-outlined[disabled],fieldset[disabled] .button.is-success.is-outlined{background-color:transparent;border-color:#22c35b;box-shadow:none;color:#22c35b}.button.is-success.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-success.is-inverted.is-outlined:hover,.button.is-success.is-inverted.is-outlined.is-hovered,.button.is-success.is-inverted.is-outlined:focus,.button.is-success.is-inverted.is-outlined.is-focused{background-color:#fff;color:#22c35b}.button.is-success.is-inverted.is-outlined.is-loading:hover::after,.button.is-success.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-success.is-inverted.is-outlined.is-loading:focus::after,.button.is-success.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #22c35b #22c35b !important}.button.is-success.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-success.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-success.is-light{background-color:#eefcf3;color:#198f43}.button.is-success.is-light:hover,.button.is-success.is-light.is-hovered{background-color:#e3faeb;border-color:transparent;color:#198f43}.button.is-success.is-light:active,.button.is-success.is-light.is-active{background-color:#d8f8e3;border-color:transparent;color:#198f43}.button.is-warning{background-color:#ffdd57;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-warning:hover,.button.is-warning.is-hovered{background-color:#ffda4a;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-warning:focus,.button.is-warning.is-focused{border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-warning:focus:not(:active),.button.is-warning.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(255,221,87,0.25)}.button.is-warning:active,.button.is-warning.is-active{background-color:#ffd83e;border-color:transparent;color:rgba(0,0,0,0.7)}.button.is-warning[disabled],fieldset[disabled] .button.is-warning{background-color:#ffdd57;border-color:#ffdd57;box-shadow:none}.button.is-warning.is-inverted{background-color:rgba(0,0,0,0.7);color:#ffdd57}.button.is-warning.is-inverted:hover,.button.is-warning.is-inverted.is-hovered{background-color:rgba(0,0,0,0.7)}.button.is-warning.is-inverted[disabled],fieldset[disabled] .button.is-warning.is-inverted{background-color:rgba(0,0,0,0.7);border-color:transparent;box-shadow:none;color:#ffdd57}.button.is-warning.is-loading::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}.button.is-warning.is-outlined{background-color:transparent;border-color:#ffdd57;color:#ffdd57}.button.is-warning.is-outlined:hover,.button.is-warning.is-outlined.is-hovered,.button.is-warning.is-outlined:focus,.button.is-warning.is-outlined.is-focused{background-color:#ffdd57;border-color:#ffdd57;color:rgba(0,0,0,0.7)}.button.is-warning.is-outlined.is-loading::after{border-color:transparent transparent #ffdd57 #ffdd57 !important}.button.is-warning.is-outlined.is-loading:hover::after,.button.is-warning.is-outlined.is-loading.is-hovered::after,.button.is-warning.is-outlined.is-loading:focus::after,.button.is-warning.is-outlined.is-loading.is-focused::after{border-color:transparent transparent rgba(0,0,0,0.7) rgba(0,0,0,0.7) !important}.button.is-warning.is-outlined[disabled],fieldset[disabled] .button.is-warning.is-outlined{background-color:transparent;border-color:#ffdd57;box-shadow:none;color:#ffdd57}.button.is-warning.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);color:rgba(0,0,0,0.7)}.button.is-warning.is-inverted.is-outlined:hover,.button.is-warning.is-inverted.is-outlined.is-hovered,.button.is-warning.is-inverted.is-outlined:focus,.button.is-warning.is-inverted.is-outlined.is-focused{background-color:rgba(0,0,0,0.7);color:#ffdd57}.button.is-warning.is-inverted.is-outlined.is-loading:hover::after,.button.is-warning.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-warning.is-inverted.is-outlined.is-loading:focus::after,.button.is-warning.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #ffdd57 #ffdd57 !important}.button.is-warning.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-warning.is-inverted.is-outlined{background-color:transparent;border-color:rgba(0,0,0,0.7);box-shadow:none;color:rgba(0,0,0,0.7)}.button.is-warning.is-light{background-color:#fffbeb;color:#947600}.button.is-warning.is-light:hover,.button.is-warning.is-light.is-hovered{background-color:#fff8de;border-color:transparent;color:#947600}.button.is-warning.is-light:active,.button.is-warning.is-light.is-active{background-color:#fff6d1;border-color:transparent;color:#947600}.button.is-danger{background-color:#da0b00;border-color:transparent;color:#fff}.button.is-danger:hover,.button.is-danger.is-hovered{background-color:#cd0a00;border-color:transparent;color:#fff}.button.is-danger:focus,.button.is-danger.is-focused{border-color:transparent;color:#fff}.button.is-danger:focus:not(:active),.button.is-danger.is-focused:not(:active){box-shadow:0 0 0 0.125em rgba(218,11,0,0.25)}.button.is-danger:active,.button.is-danger.is-active{background-color:#c10a00;border-color:transparent;color:#fff}.button.is-danger[disabled],fieldset[disabled] .button.is-danger{background-color:#da0b00;border-color:#da0b00;box-shadow:none}.button.is-danger.is-inverted{background-color:#fff;color:#da0b00}.button.is-danger.is-inverted:hover,.button.is-danger.is-inverted.is-hovered{background-color:#f2f2f2}.button.is-danger.is-inverted[disabled],fieldset[disabled] .button.is-danger.is-inverted{background-color:#fff;border-color:transparent;box-shadow:none;color:#da0b00}.button.is-danger.is-loading::after{border-color:transparent transparent #fff #fff !important}.button.is-danger.is-outlined{background-color:transparent;border-color:#da0b00;color:#da0b00}.button.is-danger.is-outlined:hover,.button.is-danger.is-outlined.is-hovered,.button.is-danger.is-outlined:focus,.button.is-danger.is-outlined.is-focused{background-color:#da0b00;border-color:#da0b00;color:#fff}.button.is-danger.is-outlined.is-loading::after{border-color:transparent transparent #da0b00 #da0b00 !important}.button.is-danger.is-outlined.is-loading:hover::after,.button.is-danger.is-outlined.is-loading.is-hovered::after,.button.is-danger.is-outlined.is-loading:focus::after,.button.is-danger.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #fff #fff !important}.button.is-danger.is-outlined[disabled],fieldset[disabled] .button.is-danger.is-outlined{background-color:transparent;border-color:#da0b00;box-shadow:none;color:#da0b00}.button.is-danger.is-inverted.is-outlined{background-color:transparent;border-color:#fff;color:#fff}.button.is-danger.is-inverted.is-outlined:hover,.button.is-danger.is-inverted.is-outlined.is-hovered,.button.is-danger.is-inverted.is-outlined:focus,.button.is-danger.is-inverted.is-outlined.is-focused{background-color:#fff;color:#da0b00}.button.is-danger.is-inverted.is-outlined.is-loading:hover::after,.button.is-danger.is-inverted.is-outlined.is-loading.is-hovered::after,.button.is-danger.is-inverted.is-outlined.is-loading:focus::after,.button.is-danger.is-inverted.is-outlined.is-loading.is-focused::after{border-color:transparent transparent #da0b00 #da0b00 !important}.button.is-danger.is-inverted.is-outlined[disabled],fieldset[disabled] .button.is-danger.is-inverted.is-outlined{background-color:transparent;border-color:#fff;box-shadow:none;color:#fff}.button.is-danger.is-light{background-color:#ffeceb;color:#f50c00}.button.is-danger.is-light:hover,.button.is-danger.is-light.is-hovered{background-color:#ffe0de;border-color:transparent;color:#f50c00}.button.is-danger.is-light:active,.button.is-danger.is-light.is-active{background-color:#ffd3d1;border-color:transparent;color:#f50c00}.button.is-small,#documenter .docs-sidebar form.docs-search>input.button{font-size:.75rem}.button.is-small:not(.is-rounded),#documenter .docs-sidebar form.docs-search>input.button:not(.is-rounded){border-radius:2px}.button.is-normal{font-size:1rem}.button.is-medium{font-size:1.25rem}.button.is-large{font-size:1.5rem}.button[disabled],fieldset[disabled] .button{background-color:#fff;border-color:#dbdbdb;box-shadow:none;opacity:.5}.button.is-fullwidth{display:flex;width:100%}.button.is-loading{color:transparent !important;pointer-events:none}.button.is-loading::after{position:absolute;left:calc(50% - (1em * 0.5));top:calc(50% - (1em * 0.5));position:absolute !important}.button.is-static{background-color:#f5f5f5;border-color:#dbdbdb;color:#6b6b6b;box-shadow:none;pointer-events:none}.button.is-rounded,#documenter .docs-sidebar form.docs-search>input.button{border-radius:9999px;padding-left:calc(1em + 0.25em);padding-right:calc(1em + 0.25em)}.buttons{align-items:center;display:flex;flex-wrap:wrap;justify-content:flex-start}.buttons .button{margin-bottom:0.5rem}.buttons .button:not(:last-child):not(.is-fullwidth){margin-right:.5rem}.buttons:last-child{margin-bottom:-0.5rem}.buttons:not(:last-child){margin-bottom:1rem}.buttons.are-small .button:not(.is-normal):not(.is-medium):not(.is-large){font-size:.75rem}.buttons.are-small .button:not(.is-normal):not(.is-medium):not(.is-large):not(.is-rounded){border-radius:2px}.buttons.are-medium .button:not(.is-small):not(.is-normal):not(.is-large){font-size:1.25rem}.buttons.are-large .button:not(.is-small):not(.is-normal):not(.is-medium){font-size:1.5rem}.buttons.has-addons .button:not(:first-child){border-bottom-left-radius:0;border-top-left-radius:0}.buttons.has-addons .button:not(:last-child){border-bottom-right-radius:0;border-top-right-radius:0;margin-right:-1px}.buttons.has-addons .button:last-child{margin-right:0}.buttons.has-addons .button:hover,.buttons.has-addons .button.is-hovered{z-index:2}.buttons.has-addons .button:focus,.buttons.has-addons .button.is-focused,.buttons.has-addons .button:active,.buttons.has-addons .button.is-active,.buttons.has-addons .button.is-selected{z-index:3}.buttons.has-addons .button:focus:hover,.buttons.has-addons .button.is-focused:hover,.buttons.has-addons .button:active:hover,.buttons.has-addons .button.is-active:hover,.buttons.has-addons .button.is-selected:hover{z-index:4}.buttons.has-addons .button.is-expanded{flex-grow:1;flex-shrink:1}.buttons.is-centered{justify-content:center}.buttons.is-centered:not(.has-addons) .button:not(.is-fullwidth){margin-left:0.25rem;margin-right:0.25rem}.buttons.is-right{justify-content:flex-end}.buttons.is-right:not(.has-addons) .button:not(.is-fullwidth){margin-left:0.25rem;margin-right:0.25rem}@media screen and (max-width: 768px){.button.is-responsive.is-small,#documenter .docs-sidebar form.docs-search>input.is-responsive{font-size:.5625rem}.button.is-responsive,.button.is-responsive.is-normal{font-size:.65625rem}.button.is-responsive.is-medium{font-size:.75rem}.button.is-responsive.is-large{font-size:1rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.button.is-responsive.is-small,#documenter .docs-sidebar form.docs-search>input.is-responsive{font-size:.65625rem}.button.is-responsive,.button.is-responsive.is-normal{font-size:.75rem}.button.is-responsive.is-medium{font-size:1rem}.button.is-responsive.is-large{font-size:1.25rem}}.container{flex-grow:1;margin:0 auto;position:relative;width:auto}.container.is-fluid{max-width:none !important;padding-left:32px;padding-right:32px;width:100%}@media screen and (min-width: 1056px){.container{max-width:992px}}@media screen and (max-width: 1215px){.container.is-widescreen:not(.is-max-desktop){max-width:1152px}}@media screen and (max-width: 1407px){.container.is-fullhd:not(.is-max-desktop):not(.is-max-widescreen){max-width:1344px}}@media screen and (min-width: 1216px){.container:not(.is-max-desktop){max-width:1152px}}@media screen and (min-width: 1408px){.container:not(.is-max-desktop):not(.is-max-widescreen){max-width:1344px}}.content li+li{margin-top:0.25em}.content p:not(:last-child),.content dl:not(:last-child),.content ol:not(:last-child),.content ul:not(:last-child),.content blockquote:not(:last-child),.content pre:not(:last-child),.content table:not(:last-child){margin-bottom:1em}.content h1,.content h2,.content h3,.content h4,.content h5,.content h6{color:#222;font-weight:600;line-height:1.125}.content h1{font-size:2em;margin-bottom:0.5em}.content h1:not(:first-child){margin-top:1em}.content h2{font-size:1.75em;margin-bottom:0.5714em}.content h2:not(:first-child){margin-top:1.1428em}.content h3{font-size:1.5em;margin-bottom:0.6666em}.content h3:not(:first-child){margin-top:1.3333em}.content h4{font-size:1.25em;margin-bottom:0.8em}.content h5{font-size:1.125em;margin-bottom:0.8888em}.content h6{font-size:1em;margin-bottom:1em}.content blockquote{background-color:#f5f5f5;border-left:5px solid #dbdbdb;padding:1.25em 1.5em}.content ol{list-style-position:outside;margin-left:2em;margin-top:1em}.content ol:not([type]){list-style-type:decimal}.content ol.is-lower-alpha:not([type]){list-style-type:lower-alpha}.content ol.is-lower-roman:not([type]){list-style-type:lower-roman}.content ol.is-upper-alpha:not([type]){list-style-type:upper-alpha}.content ol.is-upper-roman:not([type]){list-style-type:upper-roman}.content ul{list-style:disc outside;margin-left:2em;margin-top:1em}.content ul ul{list-style-type:circle;margin-top:0.5em}.content ul ul ul{list-style-type:square}.content dd{margin-left:2em}.content figure{margin-left:2em;margin-right:2em;text-align:center}.content figure:not(:first-child){margin-top:2em}.content figure:not(:last-child){margin-bottom:2em}.content figure img{display:inline-block}.content figure figcaption{font-style:italic}.content pre{-webkit-overflow-scrolling:touch;overflow-x:auto;padding:0;white-space:pre;word-wrap:normal}.content sup,.content sub{font-size:75%}.content table{width:100%}.content table td,.content table th{border:1px solid #dbdbdb;border-width:0 0 1px;padding:0.5em 0.75em;vertical-align:top}.content table th{color:#222}.content table th:not([align]){text-align:inherit}.content table thead td,.content table thead th{border-width:0 0 2px;color:#222}.content table tfoot td,.content table tfoot th{border-width:2px 0 0;color:#222}.content table tbody tr:last-child td,.content table tbody tr:last-child th{border-bottom-width:0}.content .tabs li+li{margin-top:0}.content.is-small,#documenter .docs-sidebar form.docs-search>input.content{font-size:.75rem}.content.is-normal{font-size:1rem}.content.is-medium{font-size:1.25rem}.content.is-large{font-size:1.5rem}.icon{align-items:center;display:inline-flex;justify-content:center;height:1.5rem;width:1.5rem}.icon.is-small,#documenter .docs-sidebar form.docs-search>input.icon{height:1rem;width:1rem}.icon.is-medium{height:2rem;width:2rem}.icon.is-large{height:3rem;width:3rem}.icon-text{align-items:flex-start;color:inherit;display:inline-flex;flex-wrap:wrap;line-height:1.5rem;vertical-align:top}.icon-text .icon{flex-grow:0;flex-shrink:0}.icon-text .icon:not(:last-child){margin-right:.25em}.icon-text .icon:not(:first-child){margin-left:.25em}div.icon-text{display:flex}.image,#documenter .docs-sidebar .docs-logo>img{display:block;position:relative}.image img,#documenter .docs-sidebar .docs-logo>img img{display:block;height:auto;width:100%}.image img.is-rounded,#documenter .docs-sidebar .docs-logo>img img.is-rounded{border-radius:9999px}.image.is-fullwidth,#documenter .docs-sidebar .docs-logo>img.is-fullwidth{width:100%}.image.is-square img,#documenter .docs-sidebar .docs-logo>img.is-square img,.image.is-square .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-square .has-ratio,.image.is-1by1 img,#documenter .docs-sidebar .docs-logo>img.is-1by1 img,.image.is-1by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by1 .has-ratio,.image.is-5by4 img,#documenter .docs-sidebar .docs-logo>img.is-5by4 img,.image.is-5by4 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-5by4 .has-ratio,.image.is-4by3 img,#documenter .docs-sidebar .docs-logo>img.is-4by3 img,.image.is-4by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-4by3 .has-ratio,.image.is-3by2 img,#documenter .docs-sidebar .docs-logo>img.is-3by2 img,.image.is-3by2 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by2 .has-ratio,.image.is-5by3 img,#documenter .docs-sidebar .docs-logo>img.is-5by3 img,.image.is-5by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-5by3 .has-ratio,.image.is-16by9 img,#documenter .docs-sidebar .docs-logo>img.is-16by9 img,.image.is-16by9 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-16by9 .has-ratio,.image.is-2by1 img,#documenter .docs-sidebar .docs-logo>img.is-2by1 img,.image.is-2by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-2by1 .has-ratio,.image.is-3by1 img,#documenter .docs-sidebar .docs-logo>img.is-3by1 img,.image.is-3by1 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by1 .has-ratio,.image.is-4by5 img,#documenter .docs-sidebar .docs-logo>img.is-4by5 img,.image.is-4by5 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-4by5 .has-ratio,.image.is-3by4 img,#documenter .docs-sidebar .docs-logo>img.is-3by4 img,.image.is-3by4 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by4 .has-ratio,.image.is-2by3 img,#documenter .docs-sidebar .docs-logo>img.is-2by3 img,.image.is-2by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-2by3 .has-ratio,.image.is-3by5 img,#documenter .docs-sidebar .docs-logo>img.is-3by5 img,.image.is-3by5 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-3by5 .has-ratio,.image.is-9by16 img,#documenter .docs-sidebar .docs-logo>img.is-9by16 img,.image.is-9by16 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-9by16 .has-ratio,.image.is-1by2 img,#documenter .docs-sidebar .docs-logo>img.is-1by2 img,.image.is-1by2 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by2 .has-ratio,.image.is-1by3 img,#documenter .docs-sidebar .docs-logo>img.is-1by3 img,.image.is-1by3 .has-ratio,#documenter .docs-sidebar .docs-logo>img.is-1by3 .has-ratio{height:100%;width:100%}.image.is-square,#documenter .docs-sidebar .docs-logo>img.is-square,.image.is-1by1,#documenter .docs-sidebar .docs-logo>img.is-1by1{padding-top:100%}.image.is-5by4,#documenter .docs-sidebar .docs-logo>img.is-5by4{padding-top:80%}.image.is-4by3,#documenter .docs-sidebar .docs-logo>img.is-4by3{padding-top:75%}.image.is-3by2,#documenter .docs-sidebar .docs-logo>img.is-3by2{padding-top:66.6666%}.image.is-5by3,#documenter .docs-sidebar .docs-logo>img.is-5by3{padding-top:60%}.image.is-16by9,#documenter .docs-sidebar .docs-logo>img.is-16by9{padding-top:56.25%}.image.is-2by1,#documenter .docs-sidebar .docs-logo>img.is-2by1{padding-top:50%}.image.is-3by1,#documenter .docs-sidebar .docs-logo>img.is-3by1{padding-top:33.3333%}.image.is-4by5,#documenter .docs-sidebar .docs-logo>img.is-4by5{padding-top:125%}.image.is-3by4,#documenter .docs-sidebar .docs-logo>img.is-3by4{padding-top:133.3333%}.image.is-2by3,#documenter .docs-sidebar .docs-logo>img.is-2by3{padding-top:150%}.image.is-3by5,#documenter .docs-sidebar .docs-logo>img.is-3by5{padding-top:166.6666%}.image.is-9by16,#documenter .docs-sidebar .docs-logo>img.is-9by16{padding-top:177.7777%}.image.is-1by2,#documenter .docs-sidebar .docs-logo>img.is-1by2{padding-top:200%}.image.is-1by3,#documenter .docs-sidebar .docs-logo>img.is-1by3{padding-top:300%}.image.is-16x16,#documenter .docs-sidebar .docs-logo>img.is-16x16{height:16px;width:16px}.image.is-24x24,#documenter .docs-sidebar .docs-logo>img.is-24x24{height:24px;width:24px}.image.is-32x32,#documenter .docs-sidebar .docs-logo>img.is-32x32{height:32px;width:32px}.image.is-48x48,#documenter .docs-sidebar .docs-logo>img.is-48x48{height:48px;width:48px}.image.is-64x64,#documenter .docs-sidebar .docs-logo>img.is-64x64{height:64px;width:64px}.image.is-96x96,#documenter .docs-sidebar .docs-logo>img.is-96x96{height:96px;width:96px}.image.is-128x128,#documenter .docs-sidebar .docs-logo>img.is-128x128{height:128px;width:128px}.notification{background-color:#f5f5f5;border-radius:4px;position:relative;padding:1.25rem 2.5rem 1.25rem 1.5rem}.notification a:not(.button):not(.dropdown-item){color:currentColor;text-decoration:underline}.notification strong{color:currentColor}.notification code,.notification pre{background:#fff}.notification pre code{background:transparent}.notification>.delete{right:.5rem;position:absolute;top:0.5rem}.notification .title,.notification .subtitle,.notification .content{color:currentColor}.notification.is-white{background-color:#fff;color:#0a0a0a}.notification.is-black{background-color:#0a0a0a;color:#fff}.notification.is-light{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.notification.is-dark,.content kbd.notification{background-color:#363636;color:#fff}.notification.is-primary,.docstring>section>a.notification.docs-sourcelink{background-color:#4eb5de;color:#fff}.notification.is-primary.is-light,.docstring>section>a.notification.is-light.docs-sourcelink{background-color:#eef8fc;color:#1a6d8e}.notification.is-link{background-color:#2e63b8;color:#fff}.notification.is-link.is-light{background-color:#eff3fb;color:#3169c4}.notification.is-info{background-color:#209cee;color:#fff}.notification.is-info.is-light{background-color:#ecf7fe;color:#0e72b4}.notification.is-success{background-color:#22c35b;color:#fff}.notification.is-success.is-light{background-color:#eefcf3;color:#198f43}.notification.is-warning{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.notification.is-warning.is-light{background-color:#fffbeb;color:#947600}.notification.is-danger{background-color:#da0b00;color:#fff}.notification.is-danger.is-light{background-color:#ffeceb;color:#f50c00}.progress{-moz-appearance:none;-webkit-appearance:none;border:none;border-radius:9999px;display:block;height:1rem;overflow:hidden;padding:0;width:100%}.progress::-webkit-progress-bar{background-color:#ededed}.progress::-webkit-progress-value{background-color:#222}.progress::-moz-progress-bar{background-color:#222}.progress::-ms-fill{background-color:#222;border:none}.progress.is-white::-webkit-progress-value{background-color:#fff}.progress.is-white::-moz-progress-bar{background-color:#fff}.progress.is-white::-ms-fill{background-color:#fff}.progress.is-white:indeterminate{background-image:linear-gradient(to right, #fff 30%, #ededed 30%)}.progress.is-black::-webkit-progress-value{background-color:#0a0a0a}.progress.is-black::-moz-progress-bar{background-color:#0a0a0a}.progress.is-black::-ms-fill{background-color:#0a0a0a}.progress.is-black:indeterminate{background-image:linear-gradient(to right, #0a0a0a 30%, #ededed 30%)}.progress.is-light::-webkit-progress-value{background-color:#f5f5f5}.progress.is-light::-moz-progress-bar{background-color:#f5f5f5}.progress.is-light::-ms-fill{background-color:#f5f5f5}.progress.is-light:indeterminate{background-image:linear-gradient(to right, #f5f5f5 30%, #ededed 30%)}.progress.is-dark::-webkit-progress-value,.content kbd.progress::-webkit-progress-value{background-color:#363636}.progress.is-dark::-moz-progress-bar,.content kbd.progress::-moz-progress-bar{background-color:#363636}.progress.is-dark::-ms-fill,.content kbd.progress::-ms-fill{background-color:#363636}.progress.is-dark:indeterminate,.content kbd.progress:indeterminate{background-image:linear-gradient(to right, #363636 30%, #ededed 30%)}.progress.is-primary::-webkit-progress-value,.docstring>section>a.progress.docs-sourcelink::-webkit-progress-value{background-color:#4eb5de}.progress.is-primary::-moz-progress-bar,.docstring>section>a.progress.docs-sourcelink::-moz-progress-bar{background-color:#4eb5de}.progress.is-primary::-ms-fill,.docstring>section>a.progress.docs-sourcelink::-ms-fill{background-color:#4eb5de}.progress.is-primary:indeterminate,.docstring>section>a.progress.docs-sourcelink:indeterminate{background-image:linear-gradient(to right, #4eb5de 30%, #ededed 30%)}.progress.is-link::-webkit-progress-value{background-color:#2e63b8}.progress.is-link::-moz-progress-bar{background-color:#2e63b8}.progress.is-link::-ms-fill{background-color:#2e63b8}.progress.is-link:indeterminate{background-image:linear-gradient(to right, #2e63b8 30%, #ededed 30%)}.progress.is-info::-webkit-progress-value{background-color:#209cee}.progress.is-info::-moz-progress-bar{background-color:#209cee}.progress.is-info::-ms-fill{background-color:#209cee}.progress.is-info:indeterminate{background-image:linear-gradient(to right, #209cee 30%, #ededed 30%)}.progress.is-success::-webkit-progress-value{background-color:#22c35b}.progress.is-success::-moz-progress-bar{background-color:#22c35b}.progress.is-success::-ms-fill{background-color:#22c35b}.progress.is-success:indeterminate{background-image:linear-gradient(to right, #22c35b 30%, #ededed 30%)}.progress.is-warning::-webkit-progress-value{background-color:#ffdd57}.progress.is-warning::-moz-progress-bar{background-color:#ffdd57}.progress.is-warning::-ms-fill{background-color:#ffdd57}.progress.is-warning:indeterminate{background-image:linear-gradient(to right, #ffdd57 30%, #ededed 30%)}.progress.is-danger::-webkit-progress-value{background-color:#da0b00}.progress.is-danger::-moz-progress-bar{background-color:#da0b00}.progress.is-danger::-ms-fill{background-color:#da0b00}.progress.is-danger:indeterminate{background-image:linear-gradient(to right, #da0b00 30%, #ededed 30%)}.progress:indeterminate{animation-duration:1.5s;animation-iteration-count:infinite;animation-name:moveIndeterminate;animation-timing-function:linear;background-color:#ededed;background-image:linear-gradient(to right, #222 30%, #ededed 30%);background-position:top left;background-repeat:no-repeat;background-size:150% 150%}.progress:indeterminate::-webkit-progress-bar{background-color:transparent}.progress:indeterminate::-moz-progress-bar{background-color:transparent}.progress:indeterminate::-ms-fill{animation-name:none}.progress.is-small,#documenter .docs-sidebar form.docs-search>input.progress{height:.75rem}.progress.is-medium{height:1.25rem}.progress.is-large{height:1.5rem}@keyframes moveIndeterminate{from{background-position:200% 0}to{background-position:-200% 0}}.table{background-color:#fff;color:#222}.table td,.table th{border:1px solid #dbdbdb;border-width:0 0 1px;padding:0.5em 0.75em;vertical-align:top}.table td.is-white,.table th.is-white{background-color:#fff;border-color:#fff;color:#0a0a0a}.table td.is-black,.table th.is-black{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}.table td.is-light,.table th.is-light{background-color:#f5f5f5;border-color:#f5f5f5;color:rgba(0,0,0,0.7)}.table td.is-dark,.table th.is-dark{background-color:#363636;border-color:#363636;color:#fff}.table td.is-primary,.table th.is-primary{background-color:#4eb5de;border-color:#4eb5de;color:#fff}.table td.is-link,.table th.is-link{background-color:#2e63b8;border-color:#2e63b8;color:#fff}.table td.is-info,.table th.is-info{background-color:#209cee;border-color:#209cee;color:#fff}.table td.is-success,.table th.is-success{background-color:#22c35b;border-color:#22c35b;color:#fff}.table td.is-warning,.table th.is-warning{background-color:#ffdd57;border-color:#ffdd57;color:rgba(0,0,0,0.7)}.table td.is-danger,.table th.is-danger{background-color:#da0b00;border-color:#da0b00;color:#fff}.table td.is-narrow,.table th.is-narrow{white-space:nowrap;width:1%}.table td.is-selected,.table th.is-selected{background-color:#4eb5de;color:#fff}.table td.is-selected a,.table td.is-selected strong,.table th.is-selected a,.table th.is-selected strong{color:currentColor}.table td.is-vcentered,.table th.is-vcentered{vertical-align:middle}.table th{color:#222}.table th:not([align]){text-align:left}.table tr.is-selected{background-color:#4eb5de;color:#fff}.table tr.is-selected a,.table tr.is-selected strong{color:currentColor}.table tr.is-selected td,.table tr.is-selected th{border-color:#fff;color:currentColor}.table thead{background-color:rgba(0,0,0,0)}.table thead td,.table thead th{border-width:0 0 2px;color:#222}.table tfoot{background-color:rgba(0,0,0,0)}.table tfoot td,.table tfoot th{border-width:2px 0 0;color:#222}.table tbody{background-color:rgba(0,0,0,0)}.table tbody tr:last-child td,.table tbody tr:last-child th{border-bottom-width:0}.table.is-bordered td,.table.is-bordered th{border-width:1px}.table.is-bordered tr:last-child td,.table.is-bordered tr:last-child th{border-bottom-width:1px}.table.is-fullwidth{width:100%}.table.is-hoverable tbody tr:not(.is-selected):hover{background-color:#fafafa}.table.is-hoverable.is-striped tbody tr:not(.is-selected):hover{background-color:#fafafa}.table.is-hoverable.is-striped tbody tr:not(.is-selected):hover:nth-child(even){background-color:#f5f5f5}.table.is-narrow td,.table.is-narrow th{padding:0.25em 0.5em}.table.is-striped tbody tr:not(.is-selected):nth-child(even){background-color:#fafafa}.table-container{-webkit-overflow-scrolling:touch;overflow:auto;overflow-y:hidden;max-width:100%}.tags{align-items:center;display:flex;flex-wrap:wrap;justify-content:flex-start}.tags .tag,.tags .content kbd,.content .tags kbd,.tags .docstring>section>a.docs-sourcelink{margin-bottom:0.5rem}.tags .tag:not(:last-child),.tags .content kbd:not(:last-child),.content .tags kbd:not(:last-child),.tags .docstring>section>a.docs-sourcelink:not(:last-child){margin-right:.5rem}.tags:last-child{margin-bottom:-0.5rem}.tags:not(:last-child){margin-bottom:1rem}.tags.are-medium .tag:not(.is-normal):not(.is-large),.tags.are-medium .content kbd:not(.is-normal):not(.is-large),.content .tags.are-medium kbd:not(.is-normal):not(.is-large),.tags.are-medium .docstring>section>a.docs-sourcelink:not(.is-normal):not(.is-large){font-size:1rem}.tags.are-large .tag:not(.is-normal):not(.is-medium),.tags.are-large .content kbd:not(.is-normal):not(.is-medium),.content .tags.are-large kbd:not(.is-normal):not(.is-medium),.tags.are-large .docstring>section>a.docs-sourcelink:not(.is-normal):not(.is-medium){font-size:1.25rem}.tags.is-centered{justify-content:center}.tags.is-centered .tag,.tags.is-centered .content kbd,.content .tags.is-centered kbd,.tags.is-centered .docstring>section>a.docs-sourcelink{margin-right:0.25rem;margin-left:0.25rem}.tags.is-right{justify-content:flex-end}.tags.is-right .tag:not(:first-child),.tags.is-right .content kbd:not(:first-child),.content .tags.is-right kbd:not(:first-child),.tags.is-right .docstring>section>a.docs-sourcelink:not(:first-child){margin-left:0.5rem}.tags.is-right .tag:not(:last-child),.tags.is-right .content kbd:not(:last-child),.content .tags.is-right kbd:not(:last-child),.tags.is-right .docstring>section>a.docs-sourcelink:not(:last-child){margin-right:0}.tags.has-addons .tag,.tags.has-addons .content kbd,.content .tags.has-addons kbd,.tags.has-addons .docstring>section>a.docs-sourcelink{margin-right:0}.tags.has-addons .tag:not(:first-child),.tags.has-addons .content kbd:not(:first-child),.content .tags.has-addons kbd:not(:first-child),.tags.has-addons .docstring>section>a.docs-sourcelink:not(:first-child){margin-left:0;border-top-left-radius:0;border-bottom-left-radius:0}.tags.has-addons .tag:not(:last-child),.tags.has-addons .content kbd:not(:last-child),.content .tags.has-addons kbd:not(:last-child),.tags.has-addons .docstring>section>a.docs-sourcelink:not(:last-child){border-top-right-radius:0;border-bottom-right-radius:0}.tag:not(body),.content kbd:not(body),.docstring>section>a.docs-sourcelink:not(body){align-items:center;background-color:#f5f5f5;border-radius:4px;color:#222;display:inline-flex;font-size:.75rem;height:2em;justify-content:center;line-height:1.5;padding-left:0.75em;padding-right:0.75em;white-space:nowrap}.tag:not(body) .delete,.content kbd:not(body) .delete,.docstring>section>a.docs-sourcelink:not(body) .delete{margin-left:.25rem;margin-right:-.375rem}.tag.is-white:not(body),.content kbd.is-white:not(body),.docstring>section>a.docs-sourcelink.is-white:not(body){background-color:#fff;color:#0a0a0a}.tag.is-black:not(body),.content kbd.is-black:not(body),.docstring>section>a.docs-sourcelink.is-black:not(body){background-color:#0a0a0a;color:#fff}.tag.is-light:not(body),.content kbd.is-light:not(body),.docstring>section>a.docs-sourcelink.is-light:not(body){background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.tag.is-dark:not(body),.content kbd:not(body),.docstring>section>a.docs-sourcelink.is-dark:not(body),.content .docstring>section>kbd:not(body){background-color:#363636;color:#fff}.tag.is-primary:not(body),.content kbd.is-primary:not(body),.docstring>section>a.docs-sourcelink:not(body){background-color:#4eb5de;color:#fff}.tag.is-primary.is-light:not(body),.content kbd.is-primary.is-light:not(body),.docstring>section>a.docs-sourcelink.is-light:not(body){background-color:#eef8fc;color:#1a6d8e}.tag.is-link:not(body),.content kbd.is-link:not(body),.docstring>section>a.docs-sourcelink.is-link:not(body){background-color:#2e63b8;color:#fff}.tag.is-link.is-light:not(body),.content kbd.is-link.is-light:not(body),.docstring>section>a.docs-sourcelink.is-link.is-light:not(body){background-color:#eff3fb;color:#3169c4}.tag.is-info:not(body),.content kbd.is-info:not(body),.docstring>section>a.docs-sourcelink.is-info:not(body){background-color:#209cee;color:#fff}.tag.is-info.is-light:not(body),.content kbd.is-info.is-light:not(body),.docstring>section>a.docs-sourcelink.is-info.is-light:not(body){background-color:#ecf7fe;color:#0e72b4}.tag.is-success:not(body),.content kbd.is-success:not(body),.docstring>section>a.docs-sourcelink.is-success:not(body){background-color:#22c35b;color:#fff}.tag.is-success.is-light:not(body),.content kbd.is-success.is-light:not(body),.docstring>section>a.docs-sourcelink.is-success.is-light:not(body){background-color:#eefcf3;color:#198f43}.tag.is-warning:not(body),.content kbd.is-warning:not(body),.docstring>section>a.docs-sourcelink.is-warning:not(body){background-color:#ffdd57;color:rgba(0,0,0,0.7)}.tag.is-warning.is-light:not(body),.content kbd.is-warning.is-light:not(body),.docstring>section>a.docs-sourcelink.is-warning.is-light:not(body){background-color:#fffbeb;color:#947600}.tag.is-danger:not(body),.content kbd.is-danger:not(body),.docstring>section>a.docs-sourcelink.is-danger:not(body){background-color:#da0b00;color:#fff}.tag.is-danger.is-light:not(body),.content kbd.is-danger.is-light:not(body),.docstring>section>a.docs-sourcelink.is-danger.is-light:not(body){background-color:#ffeceb;color:#f50c00}.tag.is-normal:not(body),.content kbd.is-normal:not(body),.docstring>section>a.docs-sourcelink.is-normal:not(body){font-size:.75rem}.tag.is-medium:not(body),.content kbd.is-medium:not(body),.docstring>section>a.docs-sourcelink.is-medium:not(body){font-size:1rem}.tag.is-large:not(body),.content kbd.is-large:not(body),.docstring>section>a.docs-sourcelink.is-large:not(body){font-size:1.25rem}.tag:not(body) .icon:first-child:not(:last-child),.content kbd:not(body) .icon:first-child:not(:last-child),.docstring>section>a.docs-sourcelink:not(body) .icon:first-child:not(:last-child){margin-left:-.375em;margin-right:.1875em}.tag:not(body) .icon:last-child:not(:first-child),.content kbd:not(body) .icon:last-child:not(:first-child),.docstring>section>a.docs-sourcelink:not(body) .icon:last-child:not(:first-child){margin-left:.1875em;margin-right:-.375em}.tag:not(body) .icon:first-child:last-child,.content kbd:not(body) .icon:first-child:last-child,.docstring>section>a.docs-sourcelink:not(body) .icon:first-child:last-child{margin-left:-.375em;margin-right:-.375em}.tag.is-delete:not(body),.content kbd.is-delete:not(body),.docstring>section>a.docs-sourcelink.is-delete:not(body){margin-left:1px;padding:0;position:relative;width:2em}.tag.is-delete:not(body)::before,.content kbd.is-delete:not(body)::before,.docstring>section>a.docs-sourcelink.is-delete:not(body)::before,.tag.is-delete:not(body)::after,.content kbd.is-delete:not(body)::after,.docstring>section>a.docs-sourcelink.is-delete:not(body)::after{background-color:currentColor;content:"";display:block;left:50%;position:absolute;top:50%;transform:translateX(-50%) translateY(-50%) rotate(45deg);transform-origin:center center}.tag.is-delete:not(body)::before,.content kbd.is-delete:not(body)::before,.docstring>section>a.docs-sourcelink.is-delete:not(body)::before{height:1px;width:50%}.tag.is-delete:not(body)::after,.content kbd.is-delete:not(body)::after,.docstring>section>a.docs-sourcelink.is-delete:not(body)::after{height:50%;width:1px}.tag.is-delete:not(body):hover,.content kbd.is-delete:not(body):hover,.docstring>section>a.docs-sourcelink.is-delete:not(body):hover,.tag.is-delete:not(body):focus,.content kbd.is-delete:not(body):focus,.docstring>section>a.docs-sourcelink.is-delete:not(body):focus{background-color:#e8e8e8}.tag.is-delete:not(body):active,.content kbd.is-delete:not(body):active,.docstring>section>a.docs-sourcelink.is-delete:not(body):active{background-color:#dbdbdb}.tag.is-rounded:not(body),#documenter .docs-sidebar form.docs-search>input:not(body),.content kbd.is-rounded:not(body),#documenter .docs-sidebar .content form.docs-search>input:not(body),.docstring>section>a.docs-sourcelink.is-rounded:not(body){border-radius:9999px}a.tag:hover,.docstring>section>a.docs-sourcelink:hover{text-decoration:underline}.title,.subtitle{word-break:break-word}.title em,.title span,.subtitle em,.subtitle span{font-weight:inherit}.title sub,.subtitle sub{font-size:.75em}.title sup,.subtitle sup{font-size:.75em}.title .tag,.title .content kbd,.content .title kbd,.title .docstring>section>a.docs-sourcelink,.subtitle .tag,.subtitle .content kbd,.content .subtitle kbd,.subtitle .docstring>section>a.docs-sourcelink{vertical-align:middle}.title{color:#222;font-size:2rem;font-weight:600;line-height:1.125}.title strong{color:inherit;font-weight:inherit}.title:not(.is-spaced)+.subtitle{margin-top:-1.25rem}.title.is-1{font-size:3rem}.title.is-2{font-size:2.5rem}.title.is-3{font-size:2rem}.title.is-4{font-size:1.5rem}.title.is-5{font-size:1.25rem}.title.is-6{font-size:1rem}.title.is-7{font-size:.75rem}.subtitle{color:#222;font-size:1.25rem;font-weight:400;line-height:1.25}.subtitle strong{color:#222;font-weight:600}.subtitle:not(.is-spaced)+.title{margin-top:-1.25rem}.subtitle.is-1{font-size:3rem}.subtitle.is-2{font-size:2.5rem}.subtitle.is-3{font-size:2rem}.subtitle.is-4{font-size:1.5rem}.subtitle.is-5{font-size:1.25rem}.subtitle.is-6{font-size:1rem}.subtitle.is-7{font-size:.75rem}.heading{display:block;font-size:11px;letter-spacing:1px;margin-bottom:5px;text-transform:uppercase}.number{align-items:center;background-color:#f5f5f5;border-radius:9999px;display:inline-flex;font-size:1.25rem;height:2em;justify-content:center;margin-right:1.5rem;min-width:2.5em;padding:0.25rem 0.5rem;text-align:center;vertical-align:top}.select select,.textarea,.input,#documenter .docs-sidebar form.docs-search>input{background-color:#fff;border-color:#dbdbdb;border-radius:4px;color:#222}.select select::-moz-placeholder,.textarea::-moz-placeholder,.input::-moz-placeholder,#documenter .docs-sidebar form.docs-search>input::-moz-placeholder{color:#707070}.select select::-webkit-input-placeholder,.textarea::-webkit-input-placeholder,.input::-webkit-input-placeholder,#documenter .docs-sidebar form.docs-search>input::-webkit-input-placeholder{color:#707070}.select select:-moz-placeholder,.textarea:-moz-placeholder,.input:-moz-placeholder,#documenter .docs-sidebar form.docs-search>input:-moz-placeholder{color:#707070}.select select:-ms-input-placeholder,.textarea:-ms-input-placeholder,.input:-ms-input-placeholder,#documenter .docs-sidebar form.docs-search>input:-ms-input-placeholder{color:#707070}.select select:hover,.textarea:hover,.input:hover,#documenter .docs-sidebar form.docs-search>input:hover,.select select.is-hovered,.is-hovered.textarea,.is-hovered.input,#documenter .docs-sidebar form.docs-search>input.is-hovered{border-color:#b5b5b5}.select select:focus,.textarea:focus,.input:focus,#documenter .docs-sidebar form.docs-search>input:focus,.select select.is-focused,.is-focused.textarea,.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.select select:active,.textarea:active,.input:active,#documenter .docs-sidebar form.docs-search>input:active,.select select.is-active,.is-active.textarea,.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{border-color:#2e63b8;box-shadow:0 0 0 0.125em rgba(46,99,184,0.25)}.select select[disabled],.textarea[disabled],.input[disabled],#documenter .docs-sidebar form.docs-search>input[disabled],fieldset[disabled] .select select,.select fieldset[disabled] select,fieldset[disabled] .textarea,fieldset[disabled] .input,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input{background-color:#f5f5f5;border-color:#f5f5f5;box-shadow:none;color:#6b6b6b}.select select[disabled]::-moz-placeholder,.textarea[disabled]::-moz-placeholder,.input[disabled]::-moz-placeholder,#documenter .docs-sidebar form.docs-search>input[disabled]::-moz-placeholder,fieldset[disabled] .select select::-moz-placeholder,.select fieldset[disabled] select::-moz-placeholder,fieldset[disabled] .textarea::-moz-placeholder,fieldset[disabled] .input::-moz-placeholder,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input::-moz-placeholder,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input::-moz-placeholder{color:rgba(107,107,107,0.3)}.select select[disabled]::-webkit-input-placeholder,.textarea[disabled]::-webkit-input-placeholder,.input[disabled]::-webkit-input-placeholder,#documenter .docs-sidebar form.docs-search>input[disabled]::-webkit-input-placeholder,fieldset[disabled] .select select::-webkit-input-placeholder,.select fieldset[disabled] select::-webkit-input-placeholder,fieldset[disabled] .textarea::-webkit-input-placeholder,fieldset[disabled] .input::-webkit-input-placeholder,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input::-webkit-input-placeholder,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input::-webkit-input-placeholder{color:rgba(107,107,107,0.3)}.select select[disabled]:-moz-placeholder,.textarea[disabled]:-moz-placeholder,.input[disabled]:-moz-placeholder,#documenter .docs-sidebar form.docs-search>input[disabled]:-moz-placeholder,fieldset[disabled] .select select:-moz-placeholder,.select fieldset[disabled] select:-moz-placeholder,fieldset[disabled] .textarea:-moz-placeholder,fieldset[disabled] .input:-moz-placeholder,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input:-moz-placeholder,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input:-moz-placeholder{color:rgba(107,107,107,0.3)}.select select[disabled]:-ms-input-placeholder,.textarea[disabled]:-ms-input-placeholder,.input[disabled]:-ms-input-placeholder,#documenter .docs-sidebar form.docs-search>input[disabled]:-ms-input-placeholder,fieldset[disabled] .select select:-ms-input-placeholder,.select fieldset[disabled] select:-ms-input-placeholder,fieldset[disabled] .textarea:-ms-input-placeholder,fieldset[disabled] .input:-ms-input-placeholder,fieldset[disabled] #documenter .docs-sidebar form.docs-search>input:-ms-input-placeholder,#documenter .docs-sidebar fieldset[disabled] form.docs-search>input:-ms-input-placeholder{color:rgba(107,107,107,0.3)}.textarea,.input,#documenter .docs-sidebar form.docs-search>input{box-shadow:inset 0 0.0625em 0.125em rgba(10,10,10,0.05);max-width:100%;width:100%}.textarea[readonly],.input[readonly],#documenter .docs-sidebar form.docs-search>input[readonly]{box-shadow:none}.is-white.textarea,.is-white.input,#documenter .docs-sidebar form.docs-search>input.is-white{border-color:#fff}.is-white.textarea:focus,.is-white.input:focus,#documenter .docs-sidebar form.docs-search>input.is-white:focus,.is-white.is-focused.textarea,.is-white.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-white.textarea:active,.is-white.input:active,#documenter .docs-sidebar form.docs-search>input.is-white:active,.is-white.is-active.textarea,.is-white.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}.is-black.textarea,.is-black.input,#documenter .docs-sidebar form.docs-search>input.is-black{border-color:#0a0a0a}.is-black.textarea:focus,.is-black.input:focus,#documenter .docs-sidebar form.docs-search>input.is-black:focus,.is-black.is-focused.textarea,.is-black.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-black.textarea:active,.is-black.input:active,#documenter .docs-sidebar form.docs-search>input.is-black:active,.is-black.is-active.textarea,.is-black.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}.is-light.textarea,.is-light.input,#documenter .docs-sidebar form.docs-search>input.is-light{border-color:#f5f5f5}.is-light.textarea:focus,.is-light.input:focus,#documenter .docs-sidebar form.docs-search>input.is-light:focus,.is-light.is-focused.textarea,.is-light.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-light.textarea:active,.is-light.input:active,#documenter .docs-sidebar form.docs-search>input.is-light:active,.is-light.is-active.textarea,.is-light.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(245,245,245,0.25)}.is-dark.textarea,.content kbd.textarea,.is-dark.input,#documenter .docs-sidebar form.docs-search>input.is-dark,.content kbd.input{border-color:#363636}.is-dark.textarea:focus,.content kbd.textarea:focus,.is-dark.input:focus,#documenter .docs-sidebar form.docs-search>input.is-dark:focus,.content kbd.input:focus,.is-dark.is-focused.textarea,.content kbd.is-focused.textarea,.is-dark.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.content kbd.is-focused.input,#documenter .docs-sidebar .content form.docs-search>input.is-focused,.is-dark.textarea:active,.content kbd.textarea:active,.is-dark.input:active,#documenter .docs-sidebar form.docs-search>input.is-dark:active,.content kbd.input:active,.is-dark.is-active.textarea,.content kbd.is-active.textarea,.is-dark.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active,.content kbd.is-active.input,#documenter .docs-sidebar .content form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(54,54,54,0.25)}.is-primary.textarea,.docstring>section>a.textarea.docs-sourcelink,.is-primary.input,#documenter .docs-sidebar form.docs-search>input.is-primary,.docstring>section>a.input.docs-sourcelink{border-color:#4eb5de}.is-primary.textarea:focus,.docstring>section>a.textarea.docs-sourcelink:focus,.is-primary.input:focus,#documenter .docs-sidebar form.docs-search>input.is-primary:focus,.docstring>section>a.input.docs-sourcelink:focus,.is-primary.is-focused.textarea,.docstring>section>a.is-focused.textarea.docs-sourcelink,.is-primary.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.docstring>section>a.is-focused.input.docs-sourcelink,.is-primary.textarea:active,.docstring>section>a.textarea.docs-sourcelink:active,.is-primary.input:active,#documenter .docs-sidebar form.docs-search>input.is-primary:active,.docstring>section>a.input.docs-sourcelink:active,.is-primary.is-active.textarea,.docstring>section>a.is-active.textarea.docs-sourcelink,.is-primary.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active,.docstring>section>a.is-active.input.docs-sourcelink{box-shadow:0 0 0 0.125em rgba(78,181,222,0.25)}.is-link.textarea,.is-link.input,#documenter .docs-sidebar form.docs-search>input.is-link{border-color:#2e63b8}.is-link.textarea:focus,.is-link.input:focus,#documenter .docs-sidebar form.docs-search>input.is-link:focus,.is-link.is-focused.textarea,.is-link.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-link.textarea:active,.is-link.input:active,#documenter .docs-sidebar form.docs-search>input.is-link:active,.is-link.is-active.textarea,.is-link.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(46,99,184,0.25)}.is-info.textarea,.is-info.input,#documenter .docs-sidebar form.docs-search>input.is-info{border-color:#209cee}.is-info.textarea:focus,.is-info.input:focus,#documenter .docs-sidebar form.docs-search>input.is-info:focus,.is-info.is-focused.textarea,.is-info.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-info.textarea:active,.is-info.input:active,#documenter .docs-sidebar form.docs-search>input.is-info:active,.is-info.is-active.textarea,.is-info.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(32,156,238,0.25)}.is-success.textarea,.is-success.input,#documenter .docs-sidebar form.docs-search>input.is-success{border-color:#22c35b}.is-success.textarea:focus,.is-success.input:focus,#documenter .docs-sidebar form.docs-search>input.is-success:focus,.is-success.is-focused.textarea,.is-success.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-success.textarea:active,.is-success.input:active,#documenter .docs-sidebar form.docs-search>input.is-success:active,.is-success.is-active.textarea,.is-success.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(34,195,91,0.25)}.is-warning.textarea,.is-warning.input,#documenter .docs-sidebar form.docs-search>input.is-warning{border-color:#ffdd57}.is-warning.textarea:focus,.is-warning.input:focus,#documenter .docs-sidebar form.docs-search>input.is-warning:focus,.is-warning.is-focused.textarea,.is-warning.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-warning.textarea:active,.is-warning.input:active,#documenter .docs-sidebar form.docs-search>input.is-warning:active,.is-warning.is-active.textarea,.is-warning.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(255,221,87,0.25)}.is-danger.textarea,.is-danger.input,#documenter .docs-sidebar form.docs-search>input.is-danger{border-color:#da0b00}.is-danger.textarea:focus,.is-danger.input:focus,#documenter .docs-sidebar form.docs-search>input.is-danger:focus,.is-danger.is-focused.textarea,.is-danger.is-focused.input,#documenter .docs-sidebar form.docs-search>input.is-focused,.is-danger.textarea:active,.is-danger.input:active,#documenter .docs-sidebar form.docs-search>input.is-danger:active,.is-danger.is-active.textarea,.is-danger.is-active.input,#documenter .docs-sidebar form.docs-search>input.is-active{box-shadow:0 0 0 0.125em rgba(218,11,0,0.25)}.is-small.textarea,.is-small.input,#documenter .docs-sidebar form.docs-search>input{border-radius:2px;font-size:.75rem}.is-medium.textarea,.is-medium.input,#documenter .docs-sidebar form.docs-search>input.is-medium{font-size:1.25rem}.is-large.textarea,.is-large.input,#documenter .docs-sidebar form.docs-search>input.is-large{font-size:1.5rem}.is-fullwidth.textarea,.is-fullwidth.input,#documenter .docs-sidebar form.docs-search>input.is-fullwidth{display:block;width:100%}.is-inline.textarea,.is-inline.input,#documenter .docs-sidebar form.docs-search>input.is-inline{display:inline;width:auto}.input.is-rounded,#documenter .docs-sidebar form.docs-search>input{border-radius:9999px;padding-left:calc(calc(0.75em - 1px) + 0.375em);padding-right:calc(calc(0.75em - 1px) + 0.375em)}.input.is-static,#documenter .docs-sidebar form.docs-search>input.is-static{background-color:transparent;border-color:transparent;box-shadow:none;padding-left:0;padding-right:0}.textarea{display:block;max-width:100%;min-width:100%;padding:calc(0.75em - 1px);resize:vertical}.textarea:not([rows]){max-height:40em;min-height:8em}.textarea[rows]{height:initial}.textarea.has-fixed-size{resize:none}.radio,.checkbox{cursor:pointer;display:inline-block;line-height:1.25;position:relative}.radio input,.checkbox input{cursor:pointer}.radio:hover,.checkbox:hover{color:#222}.radio[disabled],.checkbox[disabled],fieldset[disabled] .radio,fieldset[disabled] .checkbox,.radio input[disabled],.checkbox input[disabled]{color:#6b6b6b;cursor:not-allowed}.radio+.radio{margin-left:.5em}.select{display:inline-block;max-width:100%;position:relative;vertical-align:top}.select:not(.is-multiple){height:2.5em}.select:not(.is-multiple):not(.is-loading)::after{border-color:#2e63b8;right:1.125em;z-index:4}.select.is-rounded select,#documenter .docs-sidebar form.docs-search>input.select select{border-radius:9999px;padding-left:1em}.select select{cursor:pointer;display:block;font-size:1em;max-width:100%;outline:none}.select select::-ms-expand{display:none}.select select[disabled]:hover,fieldset[disabled] .select select:hover{border-color:#f5f5f5}.select select:not([multiple]){padding-right:2.5em}.select select[multiple]{height:auto;padding:0}.select select[multiple] option{padding:0.5em 1em}.select:not(.is-multiple):not(.is-loading):hover::after{border-color:#222}.select.is-white:not(:hover)::after{border-color:#fff}.select.is-white select{border-color:#fff}.select.is-white select:hover,.select.is-white select.is-hovered{border-color:#f2f2f2}.select.is-white select:focus,.select.is-white select.is-focused,.select.is-white select:active,.select.is-white select.is-active{box-shadow:0 0 0 0.125em rgba(255,255,255,0.25)}.select.is-black:not(:hover)::after{border-color:#0a0a0a}.select.is-black select{border-color:#0a0a0a}.select.is-black select:hover,.select.is-black select.is-hovered{border-color:#000}.select.is-black select:focus,.select.is-black select.is-focused,.select.is-black select:active,.select.is-black select.is-active{box-shadow:0 0 0 0.125em rgba(10,10,10,0.25)}.select.is-light:not(:hover)::after{border-color:#f5f5f5}.select.is-light select{border-color:#f5f5f5}.select.is-light select:hover,.select.is-light select.is-hovered{border-color:#e8e8e8}.select.is-light select:focus,.select.is-light select.is-focused,.select.is-light select:active,.select.is-light select.is-active{box-shadow:0 0 0 0.125em rgba(245,245,245,0.25)}.select.is-dark:not(:hover)::after,.content kbd.select:not(:hover)::after{border-color:#363636}.select.is-dark select,.content kbd.select select{border-color:#363636}.select.is-dark select:hover,.content kbd.select select:hover,.select.is-dark select.is-hovered,.content kbd.select select.is-hovered{border-color:#292929}.select.is-dark select:focus,.content kbd.select select:focus,.select.is-dark select.is-focused,.content kbd.select select.is-focused,.select.is-dark select:active,.content kbd.select select:active,.select.is-dark select.is-active,.content kbd.select select.is-active{box-shadow:0 0 0 0.125em rgba(54,54,54,0.25)}.select.is-primary:not(:hover)::after,.docstring>section>a.select.docs-sourcelink:not(:hover)::after{border-color:#4eb5de}.select.is-primary select,.docstring>section>a.select.docs-sourcelink select{border-color:#4eb5de}.select.is-primary select:hover,.docstring>section>a.select.docs-sourcelink select:hover,.select.is-primary select.is-hovered,.docstring>section>a.select.docs-sourcelink select.is-hovered{border-color:#39acda}.select.is-primary select:focus,.docstring>section>a.select.docs-sourcelink select:focus,.select.is-primary select.is-focused,.docstring>section>a.select.docs-sourcelink select.is-focused,.select.is-primary select:active,.docstring>section>a.select.docs-sourcelink select:active,.select.is-primary select.is-active,.docstring>section>a.select.docs-sourcelink select.is-active{box-shadow:0 0 0 0.125em rgba(78,181,222,0.25)}.select.is-link:not(:hover)::after{border-color:#2e63b8}.select.is-link select{border-color:#2e63b8}.select.is-link select:hover,.select.is-link select.is-hovered{border-color:#2958a4}.select.is-link select:focus,.select.is-link select.is-focused,.select.is-link select:active,.select.is-link select.is-active{box-shadow:0 0 0 0.125em rgba(46,99,184,0.25)}.select.is-info:not(:hover)::after{border-color:#209cee}.select.is-info select{border-color:#209cee}.select.is-info select:hover,.select.is-info select.is-hovered{border-color:#1190e3}.select.is-info select:focus,.select.is-info select.is-focused,.select.is-info select:active,.select.is-info select.is-active{box-shadow:0 0 0 0.125em rgba(32,156,238,0.25)}.select.is-success:not(:hover)::after{border-color:#22c35b}.select.is-success select{border-color:#22c35b}.select.is-success select:hover,.select.is-success select.is-hovered{border-color:#1ead51}.select.is-success select:focus,.select.is-success select.is-focused,.select.is-success select:active,.select.is-success select.is-active{box-shadow:0 0 0 0.125em rgba(34,195,91,0.25)}.select.is-warning:not(:hover)::after{border-color:#ffdd57}.select.is-warning select{border-color:#ffdd57}.select.is-warning select:hover,.select.is-warning select.is-hovered{border-color:#ffd83e}.select.is-warning select:focus,.select.is-warning select.is-focused,.select.is-warning select:active,.select.is-warning select.is-active{box-shadow:0 0 0 0.125em rgba(255,221,87,0.25)}.select.is-danger:not(:hover)::after{border-color:#da0b00}.select.is-danger select{border-color:#da0b00}.select.is-danger select:hover,.select.is-danger select.is-hovered{border-color:#c10a00}.select.is-danger select:focus,.select.is-danger select.is-focused,.select.is-danger select:active,.select.is-danger select.is-active{box-shadow:0 0 0 0.125em rgba(218,11,0,0.25)}.select.is-small,#documenter .docs-sidebar form.docs-search>input.select{border-radius:2px;font-size:.75rem}.select.is-medium{font-size:1.25rem}.select.is-large{font-size:1.5rem}.select.is-disabled::after{border-color:#6b6b6b !important;opacity:0.5}.select.is-fullwidth{width:100%}.select.is-fullwidth select{width:100%}.select.is-loading::after{margin-top:0;position:absolute;right:.625em;top:0.625em;transform:none}.select.is-loading.is-small:after,#documenter .docs-sidebar form.docs-search>input.is-loading:after{font-size:.75rem}.select.is-loading.is-medium:after{font-size:1.25rem}.select.is-loading.is-large:after{font-size:1.5rem}.file{align-items:stretch;display:flex;justify-content:flex-start;position:relative}.file.is-white .file-cta{background-color:#fff;border-color:transparent;color:#0a0a0a}.file.is-white:hover .file-cta,.file.is-white.is-hovered .file-cta{background-color:#f9f9f9;border-color:transparent;color:#0a0a0a}.file.is-white:focus .file-cta,.file.is-white.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(255,255,255,0.25);color:#0a0a0a}.file.is-white:active .file-cta,.file.is-white.is-active .file-cta{background-color:#f2f2f2;border-color:transparent;color:#0a0a0a}.file.is-black .file-cta{background-color:#0a0a0a;border-color:transparent;color:#fff}.file.is-black:hover .file-cta,.file.is-black.is-hovered .file-cta{background-color:#040404;border-color:transparent;color:#fff}.file.is-black:focus .file-cta,.file.is-black.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(10,10,10,0.25);color:#fff}.file.is-black:active .file-cta,.file.is-black.is-active .file-cta{background-color:#000;border-color:transparent;color:#fff}.file.is-light .file-cta{background-color:#f5f5f5;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-light:hover .file-cta,.file.is-light.is-hovered .file-cta{background-color:#eee;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-light:focus .file-cta,.file.is-light.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(245,245,245,0.25);color:rgba(0,0,0,0.7)}.file.is-light:active .file-cta,.file.is-light.is-active .file-cta{background-color:#e8e8e8;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-dark .file-cta,.content kbd.file .file-cta{background-color:#363636;border-color:transparent;color:#fff}.file.is-dark:hover .file-cta,.content kbd.file:hover .file-cta,.file.is-dark.is-hovered .file-cta,.content kbd.file.is-hovered .file-cta{background-color:#2f2f2f;border-color:transparent;color:#fff}.file.is-dark:focus .file-cta,.content kbd.file:focus .file-cta,.file.is-dark.is-focused .file-cta,.content kbd.file.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(54,54,54,0.25);color:#fff}.file.is-dark:active .file-cta,.content kbd.file:active .file-cta,.file.is-dark.is-active .file-cta,.content kbd.file.is-active .file-cta{background-color:#292929;border-color:transparent;color:#fff}.file.is-primary .file-cta,.docstring>section>a.file.docs-sourcelink .file-cta{background-color:#4eb5de;border-color:transparent;color:#fff}.file.is-primary:hover .file-cta,.docstring>section>a.file.docs-sourcelink:hover .file-cta,.file.is-primary.is-hovered .file-cta,.docstring>section>a.file.is-hovered.docs-sourcelink .file-cta{background-color:#43b1dc;border-color:transparent;color:#fff}.file.is-primary:focus .file-cta,.docstring>section>a.file.docs-sourcelink:focus .file-cta,.file.is-primary.is-focused .file-cta,.docstring>section>a.file.is-focused.docs-sourcelink .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(78,181,222,0.25);color:#fff}.file.is-primary:active .file-cta,.docstring>section>a.file.docs-sourcelink:active .file-cta,.file.is-primary.is-active .file-cta,.docstring>section>a.file.is-active.docs-sourcelink .file-cta{background-color:#39acda;border-color:transparent;color:#fff}.file.is-link .file-cta{background-color:#2e63b8;border-color:transparent;color:#fff}.file.is-link:hover .file-cta,.file.is-link.is-hovered .file-cta{background-color:#2b5eae;border-color:transparent;color:#fff}.file.is-link:focus .file-cta,.file.is-link.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(46,99,184,0.25);color:#fff}.file.is-link:active .file-cta,.file.is-link.is-active .file-cta{background-color:#2958a4;border-color:transparent;color:#fff}.file.is-info .file-cta{background-color:#209cee;border-color:transparent;color:#fff}.file.is-info:hover .file-cta,.file.is-info.is-hovered .file-cta{background-color:#1497ed;border-color:transparent;color:#fff}.file.is-info:focus .file-cta,.file.is-info.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(32,156,238,0.25);color:#fff}.file.is-info:active .file-cta,.file.is-info.is-active .file-cta{background-color:#1190e3;border-color:transparent;color:#fff}.file.is-success .file-cta{background-color:#22c35b;border-color:transparent;color:#fff}.file.is-success:hover .file-cta,.file.is-success.is-hovered .file-cta{background-color:#20b856;border-color:transparent;color:#fff}.file.is-success:focus .file-cta,.file.is-success.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(34,195,91,0.25);color:#fff}.file.is-success:active .file-cta,.file.is-success.is-active .file-cta{background-color:#1ead51;border-color:transparent;color:#fff}.file.is-warning .file-cta{background-color:#ffdd57;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-warning:hover .file-cta,.file.is-warning.is-hovered .file-cta{background-color:#ffda4a;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-warning:focus .file-cta,.file.is-warning.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(255,221,87,0.25);color:rgba(0,0,0,0.7)}.file.is-warning:active .file-cta,.file.is-warning.is-active .file-cta{background-color:#ffd83e;border-color:transparent;color:rgba(0,0,0,0.7)}.file.is-danger .file-cta{background-color:#da0b00;border-color:transparent;color:#fff}.file.is-danger:hover .file-cta,.file.is-danger.is-hovered .file-cta{background-color:#cd0a00;border-color:transparent;color:#fff}.file.is-danger:focus .file-cta,.file.is-danger.is-focused .file-cta{border-color:transparent;box-shadow:0 0 0.5em rgba(218,11,0,0.25);color:#fff}.file.is-danger:active .file-cta,.file.is-danger.is-active .file-cta{background-color:#c10a00;border-color:transparent;color:#fff}.file.is-small,#documenter .docs-sidebar form.docs-search>input.file{font-size:.75rem}.file.is-normal{font-size:1rem}.file.is-medium{font-size:1.25rem}.file.is-medium .file-icon .fa{font-size:21px}.file.is-large{font-size:1.5rem}.file.is-large .file-icon .fa{font-size:28px}.file.has-name .file-cta{border-bottom-right-radius:0;border-top-right-radius:0}.file.has-name .file-name{border-bottom-left-radius:0;border-top-left-radius:0}.file.has-name.is-empty .file-cta{border-radius:4px}.file.has-name.is-empty .file-name{display:none}.file.is-boxed .file-label{flex-direction:column}.file.is-boxed .file-cta{flex-direction:column;height:auto;padding:1em 3em}.file.is-boxed .file-name{border-width:0 1px 1px}.file.is-boxed .file-icon{height:1.5em;width:1.5em}.file.is-boxed .file-icon .fa{font-size:21px}.file.is-boxed.is-small .file-icon .fa,#documenter .docs-sidebar form.docs-search>input.is-boxed .file-icon .fa{font-size:14px}.file.is-boxed.is-medium .file-icon .fa{font-size:28px}.file.is-boxed.is-large .file-icon .fa{font-size:35px}.file.is-boxed.has-name .file-cta{border-radius:4px 4px 0 0}.file.is-boxed.has-name .file-name{border-radius:0 0 4px 4px;border-width:0 1px 1px}.file.is-centered{justify-content:center}.file.is-fullwidth .file-label{width:100%}.file.is-fullwidth .file-name{flex-grow:1;max-width:none}.file.is-right{justify-content:flex-end}.file.is-right .file-cta{border-radius:0 4px 4px 0}.file.is-right .file-name{border-radius:4px 0 0 4px;border-width:1px 0 1px 1px;order:-1}.file-label{align-items:stretch;display:flex;cursor:pointer;justify-content:flex-start;overflow:hidden;position:relative}.file-label:hover .file-cta{background-color:#eee;color:#222}.file-label:hover .file-name{border-color:#d5d5d5}.file-label:active .file-cta{background-color:#e8e8e8;color:#222}.file-label:active .file-name{border-color:#cfcfcf}.file-input{height:100%;left:0;opacity:0;outline:none;position:absolute;top:0;width:100%}.file-cta,.file-name{border-color:#dbdbdb;border-radius:4px;font-size:1em;padding-left:1em;padding-right:1em;white-space:nowrap}.file-cta{background-color:#f5f5f5;color:#222}.file-name{border-color:#dbdbdb;border-style:solid;border-width:1px 1px 1px 0;display:block;max-width:16em;overflow:hidden;text-align:inherit;text-overflow:ellipsis}.file-icon{align-items:center;display:flex;height:1em;justify-content:center;margin-right:.5em;width:1em}.file-icon .fa{font-size:14px}.label{color:#222;display:block;font-size:1rem;font-weight:700}.label:not(:last-child){margin-bottom:0.5em}.label.is-small,#documenter .docs-sidebar form.docs-search>input.label{font-size:.75rem}.label.is-medium{font-size:1.25rem}.label.is-large{font-size:1.5rem}.help{display:block;font-size:.75rem;margin-top:0.25rem}.help.is-white{color:#fff}.help.is-black{color:#0a0a0a}.help.is-light{color:#f5f5f5}.help.is-dark,.content kbd.help{color:#363636}.help.is-primary,.docstring>section>a.help.docs-sourcelink{color:#4eb5de}.help.is-link{color:#2e63b8}.help.is-info{color:#209cee}.help.is-success{color:#22c35b}.help.is-warning{color:#ffdd57}.help.is-danger{color:#da0b00}.field:not(:last-child){margin-bottom:0.75rem}.field.has-addons{display:flex;justify-content:flex-start}.field.has-addons .control:not(:last-child){margin-right:-1px}.field.has-addons .control:not(:first-child):not(:last-child) .button,.field.has-addons .control:not(:first-child):not(:last-child) .input,.field.has-addons .control:not(:first-child):not(:last-child) #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar .field.has-addons .control:not(:first-child):not(:last-child) form.docs-search>input,.field.has-addons .control:not(:first-child):not(:last-child) .select select{border-radius:0}.field.has-addons .control:first-child:not(:only-child) .button,.field.has-addons .control:first-child:not(:only-child) .input,.field.has-addons .control:first-child:not(:only-child) #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar .field.has-addons .control:first-child:not(:only-child) form.docs-search>input,.field.has-addons .control:first-child:not(:only-child) .select select{border-bottom-right-radius:0;border-top-right-radius:0}.field.has-addons .control:last-child:not(:only-child) .button,.field.has-addons .control:last-child:not(:only-child) .input,.field.has-addons .control:last-child:not(:only-child) #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar .field.has-addons .control:last-child:not(:only-child) form.docs-search>input,.field.has-addons .control:last-child:not(:only-child) .select select{border-bottom-left-radius:0;border-top-left-radius:0}.field.has-addons .control .button:not([disabled]):hover,.field.has-addons .control .button.is-hovered:not([disabled]),.field.has-addons .control .input:not([disabled]):hover,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):hover,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):hover,.field.has-addons .control .input.is-hovered:not([disabled]),.field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-hovered:not([disabled]),#documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-hovered:not([disabled]),.field.has-addons .control .select select:not([disabled]):hover,.field.has-addons .control .select select.is-hovered:not([disabled]){z-index:2}.field.has-addons .control .button:not([disabled]):focus,.field.has-addons .control .button.is-focused:not([disabled]),.field.has-addons .control .button:not([disabled]):active,.field.has-addons .control .button.is-active:not([disabled]),.field.has-addons .control .input:not([disabled]):focus,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):focus,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):focus,.field.has-addons .control .input.is-focused:not([disabled]),.field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-focused:not([disabled]),#documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-focused:not([disabled]),.field.has-addons .control .input:not([disabled]):active,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):active,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):active,.field.has-addons .control .input.is-active:not([disabled]),.field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-active:not([disabled]),#documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-active:not([disabled]),.field.has-addons .control .select select:not([disabled]):focus,.field.has-addons .control .select select.is-focused:not([disabled]),.field.has-addons .control .select select:not([disabled]):active,.field.has-addons .control .select select.is-active:not([disabled]){z-index:3}.field.has-addons .control .button:not([disabled]):focus:hover,.field.has-addons .control .button.is-focused:not([disabled]):hover,.field.has-addons .control .button:not([disabled]):active:hover,.field.has-addons .control .button.is-active:not([disabled]):hover,.field.has-addons .control .input:not([disabled]):focus:hover,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):focus:hover,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):focus:hover,.field.has-addons .control .input.is-focused:not([disabled]):hover,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-focused:not([disabled]):hover,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-focused:not([disabled]):hover,.field.has-addons .control .input:not([disabled]):active:hover,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input:not([disabled]):active:hover,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input:not([disabled]):active:hover,.field.has-addons .control .input.is-active:not([disabled]):hover,.field.has-addons .control #documenter .docs-sidebar form.docs-search>input.is-active:not([disabled]):hover,#documenter .docs-sidebar .field.has-addons .control form.docs-search>input.is-active:not([disabled]):hover,.field.has-addons .control .select select:not([disabled]):focus:hover,.field.has-addons .control .select select.is-focused:not([disabled]):hover,.field.has-addons .control .select select:not([disabled]):active:hover,.field.has-addons .control .select select.is-active:not([disabled]):hover{z-index:4}.field.has-addons .control.is-expanded{flex-grow:1;flex-shrink:1}.field.has-addons.has-addons-centered{justify-content:center}.field.has-addons.has-addons-right{justify-content:flex-end}.field.has-addons.has-addons-fullwidth .control{flex-grow:1;flex-shrink:0}.field.is-grouped{display:flex;justify-content:flex-start}.field.is-grouped>.control{flex-shrink:0}.field.is-grouped>.control:not(:last-child){margin-bottom:0;margin-right:.75rem}.field.is-grouped>.control.is-expanded{flex-grow:1;flex-shrink:1}.field.is-grouped.is-grouped-centered{justify-content:center}.field.is-grouped.is-grouped-right{justify-content:flex-end}.field.is-grouped.is-grouped-multiline{flex-wrap:wrap}.field.is-grouped.is-grouped-multiline>.control:last-child,.field.is-grouped.is-grouped-multiline>.control:not(:last-child){margin-bottom:0.75rem}.field.is-grouped.is-grouped-multiline:last-child{margin-bottom:-0.75rem}.field.is-grouped.is-grouped-multiline:not(:last-child){margin-bottom:0}@media screen and (min-width: 769px),print{.field.is-horizontal{display:flex}}.field-label .label{font-size:inherit}@media screen and (max-width: 768px){.field-label{margin-bottom:0.5rem}}@media screen and (min-width: 769px),print{.field-label{flex-basis:0;flex-grow:1;flex-shrink:0;margin-right:1.5rem;text-align:right}.field-label.is-small,#documenter .docs-sidebar form.docs-search>input.field-label{font-size:.75rem;padding-top:0.375em}.field-label.is-normal{padding-top:0.375em}.field-label.is-medium{font-size:1.25rem;padding-top:0.375em}.field-label.is-large{font-size:1.5rem;padding-top:0.375em}}.field-body .field .field{margin-bottom:0}@media screen and (min-width: 769px),print{.field-body{display:flex;flex-basis:0;flex-grow:5;flex-shrink:1}.field-body .field{margin-bottom:0}.field-body>.field{flex-shrink:1}.field-body>.field:not(.is-narrow){flex-grow:1}.field-body>.field:not(:last-child){margin-right:.75rem}}.control{box-sizing:border-box;clear:both;font-size:1rem;position:relative;text-align:inherit}.control.has-icons-left .input:focus~.icon,.control.has-icons-left #documenter .docs-sidebar form.docs-search>input:focus~.icon,#documenter .docs-sidebar .control.has-icons-left form.docs-search>input:focus~.icon,.control.has-icons-left .select:focus~.icon,.control.has-icons-right .input:focus~.icon,.control.has-icons-right #documenter .docs-sidebar form.docs-search>input:focus~.icon,#documenter .docs-sidebar .control.has-icons-right form.docs-search>input:focus~.icon,.control.has-icons-right .select:focus~.icon{color:#222}.control.has-icons-left .input.is-small~.icon,.control.has-icons-left #documenter .docs-sidebar form.docs-search>input~.icon,#documenter .docs-sidebar .control.has-icons-left form.docs-search>input~.icon,.control.has-icons-left .select.is-small~.icon,.control.has-icons-right .input.is-small~.icon,.control.has-icons-right #documenter .docs-sidebar form.docs-search>input~.icon,#documenter .docs-sidebar .control.has-icons-right form.docs-search>input~.icon,.control.has-icons-right .select.is-small~.icon{font-size:.75rem}.control.has-icons-left .input.is-medium~.icon,.control.has-icons-left #documenter .docs-sidebar form.docs-search>input.is-medium~.icon,#documenter .docs-sidebar .control.has-icons-left form.docs-search>input.is-medium~.icon,.control.has-icons-left .select.is-medium~.icon,.control.has-icons-right .input.is-medium~.icon,.control.has-icons-right #documenter .docs-sidebar form.docs-search>input.is-medium~.icon,#documenter .docs-sidebar .control.has-icons-right form.docs-search>input.is-medium~.icon,.control.has-icons-right .select.is-medium~.icon{font-size:1.25rem}.control.has-icons-left .input.is-large~.icon,.control.has-icons-left #documenter .docs-sidebar form.docs-search>input.is-large~.icon,#documenter .docs-sidebar .control.has-icons-left form.docs-search>input.is-large~.icon,.control.has-icons-left .select.is-large~.icon,.control.has-icons-right .input.is-large~.icon,.control.has-icons-right #documenter .docs-sidebar form.docs-search>input.is-large~.icon,#documenter .docs-sidebar .control.has-icons-right form.docs-search>input.is-large~.icon,.control.has-icons-right .select.is-large~.icon{font-size:1.5rem}.control.has-icons-left .icon,.control.has-icons-right .icon{color:#dbdbdb;height:2.5em;pointer-events:none;position:absolute;top:0;width:2.5em;z-index:4}.control.has-icons-left .input,.control.has-icons-left #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar .control.has-icons-left form.docs-search>input,.control.has-icons-left .select select{padding-left:2.5em}.control.has-icons-left .icon.is-left{left:0}.control.has-icons-right .input,.control.has-icons-right #documenter .docs-sidebar form.docs-search>input,#documenter .docs-sidebar .control.has-icons-right form.docs-search>input,.control.has-icons-right .select select{padding-right:2.5em}.control.has-icons-right .icon.is-right{right:0}.control.is-loading::after{position:absolute !important;right:.625em;top:0.625em;z-index:4}.control.is-loading.is-small:after,#documenter .docs-sidebar form.docs-search>input.is-loading:after{font-size:.75rem}.control.is-loading.is-medium:after{font-size:1.25rem}.control.is-loading.is-large:after{font-size:1.5rem}.breadcrumb{font-size:1rem;white-space:nowrap}.breadcrumb a{align-items:center;color:#2e63b8;display:flex;justify-content:center;padding:0 .75em}.breadcrumb a:hover{color:#363636}.breadcrumb li{align-items:center;display:flex}.breadcrumb li:first-child a{padding-left:0}.breadcrumb li.is-active a{color:#222;cursor:default;pointer-events:none}.breadcrumb li+li::before{color:#b5b5b5;content:"\0002f"}.breadcrumb ul,.breadcrumb ol{align-items:flex-start;display:flex;flex-wrap:wrap;justify-content:flex-start}.breadcrumb .icon:first-child{margin-right:.5em}.breadcrumb .icon:last-child{margin-left:.5em}.breadcrumb.is-centered ol,.breadcrumb.is-centered ul{justify-content:center}.breadcrumb.is-right ol,.breadcrumb.is-right ul{justify-content:flex-end}.breadcrumb.is-small,#documenter .docs-sidebar form.docs-search>input.breadcrumb{font-size:.75rem}.breadcrumb.is-medium{font-size:1.25rem}.breadcrumb.is-large{font-size:1.5rem}.breadcrumb.has-arrow-separator li+li::before{content:"\02192"}.breadcrumb.has-bullet-separator li+li::before{content:"\02022"}.breadcrumb.has-dot-separator li+li::before{content:"\000b7"}.breadcrumb.has-succeeds-separator li+li::before{content:"\0227B"}.card{background-color:#fff;border-radius:.25rem;box-shadow:#bbb;color:#222;max-width:100%;position:relative}.card-footer:first-child,.card-content:first-child,.card-header:first-child{border-top-left-radius:.25rem;border-top-right-radius:.25rem}.card-footer:last-child,.card-content:last-child,.card-header:last-child{border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem}.card-header{background-color:rgba(0,0,0,0);align-items:stretch;box-shadow:0 0.125em 0.25em rgba(10,10,10,0.1);display:flex}.card-header-title{align-items:center;color:#222;display:flex;flex-grow:1;font-weight:700;padding:0.75rem 1rem}.card-header-title.is-centered{justify-content:center}.card-header-icon{-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;color:currentColor;font-family:inherit;font-size:1em;margin:0;padding:0;align-items:center;cursor:pointer;display:flex;justify-content:center;padding:0.75rem 1rem}.card-image{display:block;position:relative}.card-image:first-child img{border-top-left-radius:.25rem;border-top-right-radius:.25rem}.card-image:last-child img{border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem}.card-content{background-color:rgba(0,0,0,0);padding:1.5rem}.card-footer{background-color:rgba(0,0,0,0);border-top:1px solid #ededed;align-items:stretch;display:flex}.card-footer-item{align-items:center;display:flex;flex-basis:0;flex-grow:1;flex-shrink:0;justify-content:center;padding:.75rem}.card-footer-item:not(:last-child){border-right:1px solid #ededed}.card .media:not(:last-child){margin-bottom:1.5rem}.dropdown{display:inline-flex;position:relative;vertical-align:top}.dropdown.is-active .dropdown-menu,.dropdown.is-hoverable:hover .dropdown-menu{display:block}.dropdown.is-right .dropdown-menu{left:auto;right:0}.dropdown.is-up .dropdown-menu{bottom:100%;padding-bottom:4px;padding-top:initial;top:auto}.dropdown-menu{display:none;left:0;min-width:12rem;padding-top:4px;position:absolute;top:100%;z-index:20}.dropdown-content{background-color:#fff;border-radius:4px;box-shadow:#bbb;padding-bottom:.5rem;padding-top:.5rem}.dropdown-item{color:#222;display:block;font-size:0.875rem;line-height:1.5;padding:0.375rem 1rem;position:relative}a.dropdown-item,button.dropdown-item{padding-right:3rem;text-align:inherit;white-space:nowrap;width:100%}a.dropdown-item:hover,button.dropdown-item:hover{background-color:#f5f5f5;color:#0a0a0a}a.dropdown-item.is-active,button.dropdown-item.is-active{background-color:#2e63b8;color:#fff}.dropdown-divider{background-color:#ededed;border:none;display:block;height:1px;margin:0.5rem 0}.level{align-items:center;justify-content:space-between}.level code{border-radius:4px}.level img{display:inline-block;vertical-align:top}.level.is-mobile{display:flex}.level.is-mobile .level-left,.level.is-mobile .level-right{display:flex}.level.is-mobile .level-left+.level-right{margin-top:0}.level.is-mobile .level-item:not(:last-child){margin-bottom:0;margin-right:.75rem}.level.is-mobile .level-item:not(.is-narrow){flex-grow:1}@media screen and (min-width: 769px),print{.level{display:flex}.level>.level-item:not(.is-narrow){flex-grow:1}}.level-item{align-items:center;display:flex;flex-basis:auto;flex-grow:0;flex-shrink:0;justify-content:center}.level-item .title,.level-item .subtitle{margin-bottom:0}@media screen and (max-width: 768px){.level-item:not(:last-child){margin-bottom:.75rem}}.level-left,.level-right{flex-basis:auto;flex-grow:0;flex-shrink:0}.level-left .level-item.is-flexible,.level-right .level-item.is-flexible{flex-grow:1}@media screen and (min-width: 769px),print{.level-left .level-item:not(:last-child),.level-right .level-item:not(:last-child){margin-right:.75rem}}.level-left{align-items:center;justify-content:flex-start}@media screen and (max-width: 768px){.level-left+.level-right{margin-top:1.5rem}}@media screen and (min-width: 769px),print{.level-left{display:flex}}.level-right{align-items:center;justify-content:flex-end}@media screen and (min-width: 769px),print{.level-right{display:flex}}.media{align-items:flex-start;display:flex;text-align:inherit}.media .content:not(:last-child){margin-bottom:.75rem}.media .media{border-top:1px solid rgba(219,219,219,0.5);display:flex;padding-top:.75rem}.media .media .content:not(:last-child),.media .media .control:not(:last-child){margin-bottom:.5rem}.media .media .media{padding-top:.5rem}.media .media .media+.media{margin-top:.5rem}.media+.media{border-top:1px solid rgba(219,219,219,0.5);margin-top:1rem;padding-top:1rem}.media.is-large+.media{margin-top:1.5rem;padding-top:1.5rem}.media-left,.media-right{flex-basis:auto;flex-grow:0;flex-shrink:0}.media-left{margin-right:1rem}.media-right{margin-left:1rem}.media-content{flex-basis:auto;flex-grow:1;flex-shrink:1;text-align:inherit}@media screen and (max-width: 768px){.media-content{overflow-x:auto}}.menu{font-size:1rem}.menu.is-small,#documenter .docs-sidebar form.docs-search>input.menu{font-size:.75rem}.menu.is-medium{font-size:1.25rem}.menu.is-large{font-size:1.5rem}.menu-list{line-height:1.25}.menu-list a{border-radius:2px;color:#222;display:block;padding:0.5em 0.75em}.menu-list a:hover{background-color:#f5f5f5;color:#222}.menu-list a.is-active{background-color:#2e63b8;color:#fff}.menu-list li ul{border-left:1px solid #dbdbdb;margin:.75em;padding-left:.75em}.menu-label{color:#6b6b6b;font-size:.75em;letter-spacing:.1em;text-transform:uppercase}.menu-label:not(:first-child){margin-top:1em}.menu-label:not(:last-child){margin-bottom:1em}.message{background-color:#f5f5f5;border-radius:4px;font-size:1rem}.message strong{color:currentColor}.message a:not(.button):not(.tag):not(.dropdown-item){color:currentColor;text-decoration:underline}.message.is-small,#documenter .docs-sidebar form.docs-search>input.message{font-size:.75rem}.message.is-medium{font-size:1.25rem}.message.is-large{font-size:1.5rem}.message.is-white{background-color:#fff}.message.is-white .message-header{background-color:#fff;color:#0a0a0a}.message.is-white .message-body{border-color:#fff}.message.is-black{background-color:#fafafa}.message.is-black .message-header{background-color:#0a0a0a;color:#fff}.message.is-black .message-body{border-color:#0a0a0a}.message.is-light{background-color:#fafafa}.message.is-light .message-header{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.message.is-light .message-body{border-color:#f5f5f5}.message.is-dark,.content kbd.message{background-color:#fafafa}.message.is-dark .message-header,.content kbd.message .message-header{background-color:#363636;color:#fff}.message.is-dark .message-body,.content kbd.message .message-body{border-color:#363636}.message.is-primary,.docstring>section>a.message.docs-sourcelink{background-color:#eef8fc}.message.is-primary .message-header,.docstring>section>a.message.docs-sourcelink .message-header{background-color:#4eb5de;color:#fff}.message.is-primary .message-body,.docstring>section>a.message.docs-sourcelink .message-body{border-color:#4eb5de;color:#1a6d8e}.message.is-link{background-color:#eff3fb}.message.is-link .message-header{background-color:#2e63b8;color:#fff}.message.is-link .message-body{border-color:#2e63b8;color:#3169c4}.message.is-info{background-color:#ecf7fe}.message.is-info .message-header{background-color:#209cee;color:#fff}.message.is-info .message-body{border-color:#209cee;color:#0e72b4}.message.is-success{background-color:#eefcf3}.message.is-success .message-header{background-color:#22c35b;color:#fff}.message.is-success .message-body{border-color:#22c35b;color:#198f43}.message.is-warning{background-color:#fffbeb}.message.is-warning .message-header{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.message.is-warning .message-body{border-color:#ffdd57;color:#947600}.message.is-danger{background-color:#ffeceb}.message.is-danger .message-header{background-color:#da0b00;color:#fff}.message.is-danger .message-body{border-color:#da0b00;color:#f50c00}.message-header{align-items:center;background-color:#222;border-radius:4px 4px 0 0;color:#fff;display:flex;font-weight:700;justify-content:space-between;line-height:1.25;padding:0.75em 1em;position:relative}.message-header .delete{flex-grow:0;flex-shrink:0;margin-left:.75em}.message-header+.message-body{border-width:0;border-top-left-radius:0;border-top-right-radius:0}.message-body{border-color:#dbdbdb;border-radius:4px;border-style:solid;border-width:0 0 0 4px;color:#222;padding:1.25em 1.5em}.message-body code,.message-body pre{background-color:#fff}.message-body pre code{background-color:rgba(0,0,0,0)}.modal{align-items:center;display:none;flex-direction:column;justify-content:center;overflow:hidden;position:fixed;z-index:40}.modal.is-active{display:flex}.modal-background{background-color:rgba(10,10,10,0.86)}.modal-content,.modal-card{margin:0 20px;max-height:calc(100vh - 160px);overflow:auto;position:relative;width:100%}@media screen and (min-width: 769px){.modal-content,.modal-card{margin:0 auto;max-height:calc(100vh - 40px);width:640px}}.modal-close{background:none;height:40px;position:fixed;right:20px;top:20px;width:40px}.modal-card{display:flex;flex-direction:column;max-height:calc(100vh - 40px);overflow:hidden;-ms-overflow-y:visible}.modal-card-head,.modal-card-foot{align-items:center;background-color:#f5f5f5;display:flex;flex-shrink:0;justify-content:flex-start;padding:20px;position:relative}.modal-card-head{border-bottom:1px solid #dbdbdb;border-top-left-radius:6px;border-top-right-radius:6px}.modal-card-title{color:#222;flex-grow:1;flex-shrink:0;font-size:1.5rem;line-height:1}.modal-card-foot{border-bottom-left-radius:6px;border-bottom-right-radius:6px;border-top:1px solid #dbdbdb}.modal-card-foot .button:not(:last-child){margin-right:.5em}.modal-card-body{-webkit-overflow-scrolling:touch;background-color:#fff;flex-grow:1;flex-shrink:1;overflow:auto;padding:20px}.navbar{background-color:#fff;min-height:3.25rem;position:relative;z-index:30}.navbar.is-white{background-color:#fff;color:#0a0a0a}.navbar.is-white .navbar-brand>.navbar-item,.navbar.is-white .navbar-brand .navbar-link{color:#0a0a0a}.navbar.is-white .navbar-brand>a.navbar-item:focus,.navbar.is-white .navbar-brand>a.navbar-item:hover,.navbar.is-white .navbar-brand>a.navbar-item.is-active,.navbar.is-white .navbar-brand .navbar-link:focus,.navbar.is-white .navbar-brand .navbar-link:hover,.navbar.is-white .navbar-brand .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}.navbar.is-white .navbar-brand .navbar-link::after{border-color:#0a0a0a}.navbar.is-white .navbar-burger{color:#0a0a0a}@media screen and (min-width: 1056px){.navbar.is-white .navbar-start>.navbar-item,.navbar.is-white .navbar-start .navbar-link,.navbar.is-white .navbar-end>.navbar-item,.navbar.is-white .navbar-end .navbar-link{color:#0a0a0a}.navbar.is-white .navbar-start>a.navbar-item:focus,.navbar.is-white .navbar-start>a.navbar-item:hover,.navbar.is-white .navbar-start>a.navbar-item.is-active,.navbar.is-white .navbar-start .navbar-link:focus,.navbar.is-white .navbar-start .navbar-link:hover,.navbar.is-white .navbar-start .navbar-link.is-active,.navbar.is-white .navbar-end>a.navbar-item:focus,.navbar.is-white .navbar-end>a.navbar-item:hover,.navbar.is-white .navbar-end>a.navbar-item.is-active,.navbar.is-white .navbar-end .navbar-link:focus,.navbar.is-white .navbar-end .navbar-link:hover,.navbar.is-white .navbar-end .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}.navbar.is-white .navbar-start .navbar-link::after,.navbar.is-white .navbar-end .navbar-link::after{border-color:#0a0a0a}.navbar.is-white .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-white .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-white .navbar-item.has-dropdown.is-active .navbar-link{background-color:#f2f2f2;color:#0a0a0a}.navbar.is-white .navbar-dropdown a.navbar-item.is-active{background-color:#fff;color:#0a0a0a}}.navbar.is-black{background-color:#0a0a0a;color:#fff}.navbar.is-black .navbar-brand>.navbar-item,.navbar.is-black .navbar-brand .navbar-link{color:#fff}.navbar.is-black .navbar-brand>a.navbar-item:focus,.navbar.is-black .navbar-brand>a.navbar-item:hover,.navbar.is-black .navbar-brand>a.navbar-item.is-active,.navbar.is-black .navbar-brand .navbar-link:focus,.navbar.is-black .navbar-brand .navbar-link:hover,.navbar.is-black .navbar-brand .navbar-link.is-active{background-color:#000;color:#fff}.navbar.is-black .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-black .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-black .navbar-start>.navbar-item,.navbar.is-black .navbar-start .navbar-link,.navbar.is-black .navbar-end>.navbar-item,.navbar.is-black .navbar-end .navbar-link{color:#fff}.navbar.is-black .navbar-start>a.navbar-item:focus,.navbar.is-black .navbar-start>a.navbar-item:hover,.navbar.is-black .navbar-start>a.navbar-item.is-active,.navbar.is-black .navbar-start .navbar-link:focus,.navbar.is-black .navbar-start .navbar-link:hover,.navbar.is-black .navbar-start .navbar-link.is-active,.navbar.is-black .navbar-end>a.navbar-item:focus,.navbar.is-black .navbar-end>a.navbar-item:hover,.navbar.is-black .navbar-end>a.navbar-item.is-active,.navbar.is-black .navbar-end .navbar-link:focus,.navbar.is-black .navbar-end .navbar-link:hover,.navbar.is-black .navbar-end .navbar-link.is-active{background-color:#000;color:#fff}.navbar.is-black .navbar-start .navbar-link::after,.navbar.is-black .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-black .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-black .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-black .navbar-item.has-dropdown.is-active .navbar-link{background-color:#000;color:#fff}.navbar.is-black .navbar-dropdown a.navbar-item.is-active{background-color:#0a0a0a;color:#fff}}.navbar.is-light{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-brand>.navbar-item,.navbar.is-light .navbar-brand .navbar-link{color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-brand>a.navbar-item:focus,.navbar.is-light .navbar-brand>a.navbar-item:hover,.navbar.is-light .navbar-brand>a.navbar-item.is-active,.navbar.is-light .navbar-brand .navbar-link:focus,.navbar.is-light .navbar-brand .navbar-link:hover,.navbar.is-light .navbar-brand .navbar-link.is-active{background-color:#e8e8e8;color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-brand .navbar-link::after{border-color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-burger{color:rgba(0,0,0,0.7)}@media screen and (min-width: 1056px){.navbar.is-light .navbar-start>.navbar-item,.navbar.is-light .navbar-start .navbar-link,.navbar.is-light .navbar-end>.navbar-item,.navbar.is-light .navbar-end .navbar-link{color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-start>a.navbar-item:focus,.navbar.is-light .navbar-start>a.navbar-item:hover,.navbar.is-light .navbar-start>a.navbar-item.is-active,.navbar.is-light .navbar-start .navbar-link:focus,.navbar.is-light .navbar-start .navbar-link:hover,.navbar.is-light .navbar-start .navbar-link.is-active,.navbar.is-light .navbar-end>a.navbar-item:focus,.navbar.is-light .navbar-end>a.navbar-item:hover,.navbar.is-light .navbar-end>a.navbar-item.is-active,.navbar.is-light .navbar-end .navbar-link:focus,.navbar.is-light .navbar-end .navbar-link:hover,.navbar.is-light .navbar-end .navbar-link.is-active{background-color:#e8e8e8;color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-start .navbar-link::after,.navbar.is-light .navbar-end .navbar-link::after{border-color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-light .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-light .navbar-item.has-dropdown.is-active .navbar-link{background-color:#e8e8e8;color:rgba(0,0,0,0.7)}.navbar.is-light .navbar-dropdown a.navbar-item.is-active{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}}.navbar.is-dark,.content kbd.navbar{background-color:#363636;color:#fff}.navbar.is-dark .navbar-brand>.navbar-item,.content kbd.navbar .navbar-brand>.navbar-item,.navbar.is-dark .navbar-brand .navbar-link,.content kbd.navbar .navbar-brand .navbar-link{color:#fff}.navbar.is-dark .navbar-brand>a.navbar-item:focus,.content kbd.navbar .navbar-brand>a.navbar-item:focus,.navbar.is-dark .navbar-brand>a.navbar-item:hover,.content kbd.navbar .navbar-brand>a.navbar-item:hover,.navbar.is-dark .navbar-brand>a.navbar-item.is-active,.content kbd.navbar .navbar-brand>a.navbar-item.is-active,.navbar.is-dark .navbar-brand .navbar-link:focus,.content kbd.navbar .navbar-brand .navbar-link:focus,.navbar.is-dark .navbar-brand .navbar-link:hover,.content kbd.navbar .navbar-brand .navbar-link:hover,.navbar.is-dark .navbar-brand .navbar-link.is-active,.content kbd.navbar .navbar-brand .navbar-link.is-active{background-color:#292929;color:#fff}.navbar.is-dark .navbar-brand .navbar-link::after,.content kbd.navbar .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-dark .navbar-burger,.content kbd.navbar .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-dark .navbar-start>.navbar-item,.content kbd.navbar .navbar-start>.navbar-item,.navbar.is-dark .navbar-start .navbar-link,.content kbd.navbar .navbar-start .navbar-link,.navbar.is-dark .navbar-end>.navbar-item,.content kbd.navbar .navbar-end>.navbar-item,.navbar.is-dark .navbar-end .navbar-link,.content kbd.navbar .navbar-end .navbar-link{color:#fff}.navbar.is-dark .navbar-start>a.navbar-item:focus,.content kbd.navbar .navbar-start>a.navbar-item:focus,.navbar.is-dark .navbar-start>a.navbar-item:hover,.content kbd.navbar .navbar-start>a.navbar-item:hover,.navbar.is-dark .navbar-start>a.navbar-item.is-active,.content kbd.navbar .navbar-start>a.navbar-item.is-active,.navbar.is-dark .navbar-start .navbar-link:focus,.content kbd.navbar .navbar-start .navbar-link:focus,.navbar.is-dark .navbar-start .navbar-link:hover,.content kbd.navbar .navbar-start .navbar-link:hover,.navbar.is-dark .navbar-start .navbar-link.is-active,.content kbd.navbar .navbar-start .navbar-link.is-active,.navbar.is-dark .navbar-end>a.navbar-item:focus,.content kbd.navbar .navbar-end>a.navbar-item:focus,.navbar.is-dark .navbar-end>a.navbar-item:hover,.content kbd.navbar .navbar-end>a.navbar-item:hover,.navbar.is-dark .navbar-end>a.navbar-item.is-active,.content kbd.navbar .navbar-end>a.navbar-item.is-active,.navbar.is-dark .navbar-end .navbar-link:focus,.content kbd.navbar .navbar-end .navbar-link:focus,.navbar.is-dark .navbar-end .navbar-link:hover,.content kbd.navbar .navbar-end .navbar-link:hover,.navbar.is-dark .navbar-end .navbar-link.is-active,.content kbd.navbar .navbar-end .navbar-link.is-active{background-color:#292929;color:#fff}.navbar.is-dark .navbar-start .navbar-link::after,.content kbd.navbar .navbar-start .navbar-link::after,.navbar.is-dark .navbar-end .navbar-link::after,.content kbd.navbar .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-dark .navbar-item.has-dropdown:focus .navbar-link,.content kbd.navbar .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-dark .navbar-item.has-dropdown:hover .navbar-link,.content kbd.navbar .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-dark .navbar-item.has-dropdown.is-active .navbar-link,.content kbd.navbar .navbar-item.has-dropdown.is-active .navbar-link{background-color:#292929;color:#fff}.navbar.is-dark .navbar-dropdown a.navbar-item.is-active,.content kbd.navbar .navbar-dropdown a.navbar-item.is-active{background-color:#363636;color:#fff}}.navbar.is-primary,.docstring>section>a.navbar.docs-sourcelink{background-color:#4eb5de;color:#fff}.navbar.is-primary .navbar-brand>.navbar-item,.docstring>section>a.navbar.docs-sourcelink .navbar-brand>.navbar-item,.navbar.is-primary .navbar-brand .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link{color:#fff}.navbar.is-primary .navbar-brand>a.navbar-item:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item:focus,.navbar.is-primary .navbar-brand>a.navbar-item:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item:hover,.navbar.is-primary .navbar-brand>a.navbar-item.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-brand>a.navbar-item.is-active,.navbar.is-primary .navbar-brand .navbar-link:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link:focus,.navbar.is-primary .navbar-brand .navbar-link:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link:hover,.navbar.is-primary .navbar-brand .navbar-link.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link.is-active{background-color:#39acda;color:#fff}.navbar.is-primary .navbar-brand .navbar-link::after,.docstring>section>a.navbar.docs-sourcelink .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-primary .navbar-burger,.docstring>section>a.navbar.docs-sourcelink .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-primary .navbar-start>.navbar-item,.docstring>section>a.navbar.docs-sourcelink .navbar-start>.navbar-item,.navbar.is-primary .navbar-start .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link,.navbar.is-primary .navbar-end>.navbar-item,.docstring>section>a.navbar.docs-sourcelink .navbar-end>.navbar-item,.navbar.is-primary .navbar-end .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link{color:#fff}.navbar.is-primary .navbar-start>a.navbar-item:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item:focus,.navbar.is-primary .navbar-start>a.navbar-item:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item:hover,.navbar.is-primary .navbar-start>a.navbar-item.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-start>a.navbar-item.is-active,.navbar.is-primary .navbar-start .navbar-link:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link:focus,.navbar.is-primary .navbar-start .navbar-link:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link:hover,.navbar.is-primary .navbar-start .navbar-link.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link.is-active,.navbar.is-primary .navbar-end>a.navbar-item:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item:focus,.navbar.is-primary .navbar-end>a.navbar-item:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item:hover,.navbar.is-primary .navbar-end>a.navbar-item.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-end>a.navbar-item.is-active,.navbar.is-primary .navbar-end .navbar-link:focus,.docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link:focus,.navbar.is-primary .navbar-end .navbar-link:hover,.docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link:hover,.navbar.is-primary .navbar-end .navbar-link.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link.is-active{background-color:#39acda;color:#fff}.navbar.is-primary .navbar-start .navbar-link::after,.docstring>section>a.navbar.docs-sourcelink .navbar-start .navbar-link::after,.navbar.is-primary .navbar-end .navbar-link::after,.docstring>section>a.navbar.docs-sourcelink .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-primary .navbar-item.has-dropdown:focus .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-primary .navbar-item.has-dropdown:hover .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-primary .navbar-item.has-dropdown.is-active .navbar-link,.docstring>section>a.navbar.docs-sourcelink .navbar-item.has-dropdown.is-active .navbar-link{background-color:#39acda;color:#fff}.navbar.is-primary .navbar-dropdown a.navbar-item.is-active,.docstring>section>a.navbar.docs-sourcelink .navbar-dropdown a.navbar-item.is-active{background-color:#4eb5de;color:#fff}}.navbar.is-link{background-color:#2e63b8;color:#fff}.navbar.is-link .navbar-brand>.navbar-item,.navbar.is-link .navbar-brand .navbar-link{color:#fff}.navbar.is-link .navbar-brand>a.navbar-item:focus,.navbar.is-link .navbar-brand>a.navbar-item:hover,.navbar.is-link .navbar-brand>a.navbar-item.is-active,.navbar.is-link .navbar-brand .navbar-link:focus,.navbar.is-link .navbar-brand .navbar-link:hover,.navbar.is-link .navbar-brand .navbar-link.is-active{background-color:#2958a4;color:#fff}.navbar.is-link .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-link .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-link .navbar-start>.navbar-item,.navbar.is-link .navbar-start .navbar-link,.navbar.is-link .navbar-end>.navbar-item,.navbar.is-link .navbar-end .navbar-link{color:#fff}.navbar.is-link .navbar-start>a.navbar-item:focus,.navbar.is-link .navbar-start>a.navbar-item:hover,.navbar.is-link .navbar-start>a.navbar-item.is-active,.navbar.is-link .navbar-start .navbar-link:focus,.navbar.is-link .navbar-start .navbar-link:hover,.navbar.is-link .navbar-start .navbar-link.is-active,.navbar.is-link .navbar-end>a.navbar-item:focus,.navbar.is-link .navbar-end>a.navbar-item:hover,.navbar.is-link .navbar-end>a.navbar-item.is-active,.navbar.is-link .navbar-end .navbar-link:focus,.navbar.is-link .navbar-end .navbar-link:hover,.navbar.is-link .navbar-end .navbar-link.is-active{background-color:#2958a4;color:#fff}.navbar.is-link .navbar-start .navbar-link::after,.navbar.is-link .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-link .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-link .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-link .navbar-item.has-dropdown.is-active .navbar-link{background-color:#2958a4;color:#fff}.navbar.is-link .navbar-dropdown a.navbar-item.is-active{background-color:#2e63b8;color:#fff}}.navbar.is-info{background-color:#209cee;color:#fff}.navbar.is-info .navbar-brand>.navbar-item,.navbar.is-info .navbar-brand .navbar-link{color:#fff}.navbar.is-info .navbar-brand>a.navbar-item:focus,.navbar.is-info .navbar-brand>a.navbar-item:hover,.navbar.is-info .navbar-brand>a.navbar-item.is-active,.navbar.is-info .navbar-brand .navbar-link:focus,.navbar.is-info .navbar-brand .navbar-link:hover,.navbar.is-info .navbar-brand .navbar-link.is-active{background-color:#1190e3;color:#fff}.navbar.is-info .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-info .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-info .navbar-start>.navbar-item,.navbar.is-info .navbar-start .navbar-link,.navbar.is-info .navbar-end>.navbar-item,.navbar.is-info .navbar-end .navbar-link{color:#fff}.navbar.is-info .navbar-start>a.navbar-item:focus,.navbar.is-info .navbar-start>a.navbar-item:hover,.navbar.is-info .navbar-start>a.navbar-item.is-active,.navbar.is-info .navbar-start .navbar-link:focus,.navbar.is-info .navbar-start .navbar-link:hover,.navbar.is-info .navbar-start .navbar-link.is-active,.navbar.is-info .navbar-end>a.navbar-item:focus,.navbar.is-info .navbar-end>a.navbar-item:hover,.navbar.is-info .navbar-end>a.navbar-item.is-active,.navbar.is-info .navbar-end .navbar-link:focus,.navbar.is-info .navbar-end .navbar-link:hover,.navbar.is-info .navbar-end .navbar-link.is-active{background-color:#1190e3;color:#fff}.navbar.is-info .navbar-start .navbar-link::after,.navbar.is-info .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-info .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-info .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-info .navbar-item.has-dropdown.is-active .navbar-link{background-color:#1190e3;color:#fff}.navbar.is-info .navbar-dropdown a.navbar-item.is-active{background-color:#209cee;color:#fff}}.navbar.is-success{background-color:#22c35b;color:#fff}.navbar.is-success .navbar-brand>.navbar-item,.navbar.is-success .navbar-brand .navbar-link{color:#fff}.navbar.is-success .navbar-brand>a.navbar-item:focus,.navbar.is-success .navbar-brand>a.navbar-item:hover,.navbar.is-success .navbar-brand>a.navbar-item.is-active,.navbar.is-success .navbar-brand .navbar-link:focus,.navbar.is-success .navbar-brand .navbar-link:hover,.navbar.is-success .navbar-brand .navbar-link.is-active{background-color:#1ead51;color:#fff}.navbar.is-success .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-success .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-success .navbar-start>.navbar-item,.navbar.is-success .navbar-start .navbar-link,.navbar.is-success .navbar-end>.navbar-item,.navbar.is-success .navbar-end .navbar-link{color:#fff}.navbar.is-success .navbar-start>a.navbar-item:focus,.navbar.is-success .navbar-start>a.navbar-item:hover,.navbar.is-success .navbar-start>a.navbar-item.is-active,.navbar.is-success .navbar-start .navbar-link:focus,.navbar.is-success .navbar-start .navbar-link:hover,.navbar.is-success .navbar-start .navbar-link.is-active,.navbar.is-success .navbar-end>a.navbar-item:focus,.navbar.is-success .navbar-end>a.navbar-item:hover,.navbar.is-success .navbar-end>a.navbar-item.is-active,.navbar.is-success .navbar-end .navbar-link:focus,.navbar.is-success .navbar-end .navbar-link:hover,.navbar.is-success .navbar-end .navbar-link.is-active{background-color:#1ead51;color:#fff}.navbar.is-success .navbar-start .navbar-link::after,.navbar.is-success .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-success .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-success .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-success .navbar-item.has-dropdown.is-active .navbar-link{background-color:#1ead51;color:#fff}.navbar.is-success .navbar-dropdown a.navbar-item.is-active{background-color:#22c35b;color:#fff}}.navbar.is-warning{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-brand>.navbar-item,.navbar.is-warning .navbar-brand .navbar-link{color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-brand>a.navbar-item:focus,.navbar.is-warning .navbar-brand>a.navbar-item:hover,.navbar.is-warning .navbar-brand>a.navbar-item.is-active,.navbar.is-warning .navbar-brand .navbar-link:focus,.navbar.is-warning .navbar-brand .navbar-link:hover,.navbar.is-warning .navbar-brand .navbar-link.is-active{background-color:#ffd83e;color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-brand .navbar-link::after{border-color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-burger{color:rgba(0,0,0,0.7)}@media screen and (min-width: 1056px){.navbar.is-warning .navbar-start>.navbar-item,.navbar.is-warning .navbar-start .navbar-link,.navbar.is-warning .navbar-end>.navbar-item,.navbar.is-warning .navbar-end .navbar-link{color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-start>a.navbar-item:focus,.navbar.is-warning .navbar-start>a.navbar-item:hover,.navbar.is-warning .navbar-start>a.navbar-item.is-active,.navbar.is-warning .navbar-start .navbar-link:focus,.navbar.is-warning .navbar-start .navbar-link:hover,.navbar.is-warning .navbar-start .navbar-link.is-active,.navbar.is-warning .navbar-end>a.navbar-item:focus,.navbar.is-warning .navbar-end>a.navbar-item:hover,.navbar.is-warning .navbar-end>a.navbar-item.is-active,.navbar.is-warning .navbar-end .navbar-link:focus,.navbar.is-warning .navbar-end .navbar-link:hover,.navbar.is-warning .navbar-end .navbar-link.is-active{background-color:#ffd83e;color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-start .navbar-link::after,.navbar.is-warning .navbar-end .navbar-link::after{border-color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-warning .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-warning .navbar-item.has-dropdown.is-active .navbar-link{background-color:#ffd83e;color:rgba(0,0,0,0.7)}.navbar.is-warning .navbar-dropdown a.navbar-item.is-active{background-color:#ffdd57;color:rgba(0,0,0,0.7)}}.navbar.is-danger{background-color:#da0b00;color:#fff}.navbar.is-danger .navbar-brand>.navbar-item,.navbar.is-danger .navbar-brand .navbar-link{color:#fff}.navbar.is-danger .navbar-brand>a.navbar-item:focus,.navbar.is-danger .navbar-brand>a.navbar-item:hover,.navbar.is-danger .navbar-brand>a.navbar-item.is-active,.navbar.is-danger .navbar-brand .navbar-link:focus,.navbar.is-danger .navbar-brand .navbar-link:hover,.navbar.is-danger .navbar-brand .navbar-link.is-active{background-color:#c10a00;color:#fff}.navbar.is-danger .navbar-brand .navbar-link::after{border-color:#fff}.navbar.is-danger .navbar-burger{color:#fff}@media screen and (min-width: 1056px){.navbar.is-danger .navbar-start>.navbar-item,.navbar.is-danger .navbar-start .navbar-link,.navbar.is-danger .navbar-end>.navbar-item,.navbar.is-danger .navbar-end .navbar-link{color:#fff}.navbar.is-danger .navbar-start>a.navbar-item:focus,.navbar.is-danger .navbar-start>a.navbar-item:hover,.navbar.is-danger .navbar-start>a.navbar-item.is-active,.navbar.is-danger .navbar-start .navbar-link:focus,.navbar.is-danger .navbar-start .navbar-link:hover,.navbar.is-danger .navbar-start .navbar-link.is-active,.navbar.is-danger .navbar-end>a.navbar-item:focus,.navbar.is-danger .navbar-end>a.navbar-item:hover,.navbar.is-danger .navbar-end>a.navbar-item.is-active,.navbar.is-danger .navbar-end .navbar-link:focus,.navbar.is-danger .navbar-end .navbar-link:hover,.navbar.is-danger .navbar-end .navbar-link.is-active{background-color:#c10a00;color:#fff}.navbar.is-danger .navbar-start .navbar-link::after,.navbar.is-danger .navbar-end .navbar-link::after{border-color:#fff}.navbar.is-danger .navbar-item.has-dropdown:focus .navbar-link,.navbar.is-danger .navbar-item.has-dropdown:hover .navbar-link,.navbar.is-danger .navbar-item.has-dropdown.is-active .navbar-link{background-color:#c10a00;color:#fff}.navbar.is-danger .navbar-dropdown a.navbar-item.is-active{background-color:#da0b00;color:#fff}}.navbar>.container{align-items:stretch;display:flex;min-height:3.25rem;width:100%}.navbar.has-shadow{box-shadow:0 2px 0 0 #f5f5f5}.navbar.is-fixed-bottom,.navbar.is-fixed-top{left:0;position:fixed;right:0;z-index:30}.navbar.is-fixed-bottom{bottom:0}.navbar.is-fixed-bottom.has-shadow{box-shadow:0 -2px 0 0 #f5f5f5}.navbar.is-fixed-top{top:0}html.has-navbar-fixed-top,body.has-navbar-fixed-top{padding-top:3.25rem}html.has-navbar-fixed-bottom,body.has-navbar-fixed-bottom{padding-bottom:3.25rem}.navbar-brand,.navbar-tabs{align-items:stretch;display:flex;flex-shrink:0;min-height:3.25rem}.navbar-brand a.navbar-item:focus,.navbar-brand a.navbar-item:hover{background-color:transparent}.navbar-tabs{-webkit-overflow-scrolling:touch;max-width:100vw;overflow-x:auto;overflow-y:hidden}.navbar-burger{color:#222;-moz-appearance:none;-webkit-appearance:none;appearance:none;background:none;border:none;cursor:pointer;display:block;height:3.25rem;position:relative;width:3.25rem;margin-left:auto}.navbar-burger span{background-color:currentColor;display:block;height:1px;left:calc(50% - 8px);position:absolute;transform-origin:center;transition-duration:86ms;transition-property:background-color, opacity, transform;transition-timing-function:ease-out;width:16px}.navbar-burger span:nth-child(1){top:calc(50% - 6px)}.navbar-burger span:nth-child(2){top:calc(50% - 1px)}.navbar-burger span:nth-child(3){top:calc(50% + 4px)}.navbar-burger:hover{background-color:rgba(0,0,0,0.05)}.navbar-burger.is-active span:nth-child(1){transform:translateY(5px) rotate(45deg)}.navbar-burger.is-active span:nth-child(2){opacity:0}.navbar-burger.is-active span:nth-child(3){transform:translateY(-5px) rotate(-45deg)}.navbar-menu{display:none}.navbar-item,.navbar-link{color:#222;display:block;line-height:1.5;padding:0.5rem 0.75rem;position:relative}.navbar-item .icon:only-child,.navbar-link .icon:only-child{margin-left:-0.25rem;margin-right:-0.25rem}a.navbar-item,.navbar-link{cursor:pointer}a.navbar-item:focus,a.navbar-item:focus-within,a.navbar-item:hover,a.navbar-item.is-active,.navbar-link:focus,.navbar-link:focus-within,.navbar-link:hover,.navbar-link.is-active{background-color:#fafafa;color:#2e63b8}.navbar-item{flex-grow:0;flex-shrink:0}.navbar-item img{max-height:1.75rem}.navbar-item.has-dropdown{padding:0}.navbar-item.is-expanded{flex-grow:1;flex-shrink:1}.navbar-item.is-tab{border-bottom:1px solid transparent;min-height:3.25rem;padding-bottom:calc(0.5rem - 1px)}.navbar-item.is-tab:focus,.navbar-item.is-tab:hover{background-color:rgba(0,0,0,0);border-bottom-color:#2e63b8}.navbar-item.is-tab.is-active{background-color:rgba(0,0,0,0);border-bottom-color:#2e63b8;border-bottom-style:solid;border-bottom-width:3px;color:#2e63b8;padding-bottom:calc(0.5rem - 3px)}.navbar-content{flex-grow:1;flex-shrink:1}.navbar-link:not(.is-arrowless){padding-right:2.5em}.navbar-link:not(.is-arrowless)::after{border-color:#2e63b8;margin-top:-0.375em;right:1.125em}.navbar-dropdown{font-size:0.875rem;padding-bottom:0.5rem;padding-top:0.5rem}.navbar-dropdown .navbar-item{padding-left:1.5rem;padding-right:1.5rem}.navbar-divider{background-color:#f5f5f5;border:none;display:none;height:2px;margin:0.5rem 0}@media screen and (max-width: 1055px){.navbar>.container{display:block}.navbar-brand .navbar-item,.navbar-tabs .navbar-item{align-items:center;display:flex}.navbar-link::after{display:none}.navbar-menu{background-color:#fff;box-shadow:0 8px 16px rgba(10,10,10,0.1);padding:0.5rem 0}.navbar-menu.is-active{display:block}.navbar.is-fixed-bottom-touch,.navbar.is-fixed-top-touch{left:0;position:fixed;right:0;z-index:30}.navbar.is-fixed-bottom-touch{bottom:0}.navbar.is-fixed-bottom-touch.has-shadow{box-shadow:0 -2px 3px rgba(10,10,10,0.1)}.navbar.is-fixed-top-touch{top:0}.navbar.is-fixed-top .navbar-menu,.navbar.is-fixed-top-touch .navbar-menu{-webkit-overflow-scrolling:touch;max-height:calc(100vh - 3.25rem);overflow:auto}html.has-navbar-fixed-top-touch,body.has-navbar-fixed-top-touch{padding-top:3.25rem}html.has-navbar-fixed-bottom-touch,body.has-navbar-fixed-bottom-touch{padding-bottom:3.25rem}}@media screen and (min-width: 1056px){.navbar,.navbar-menu,.navbar-start,.navbar-end{align-items:stretch;display:flex}.navbar{min-height:3.25rem}.navbar.is-spaced{padding:1rem 2rem}.navbar.is-spaced .navbar-start,.navbar.is-spaced .navbar-end{align-items:center}.navbar.is-spaced a.navbar-item,.navbar.is-spaced .navbar-link{border-radius:4px}.navbar.is-transparent a.navbar-item:focus,.navbar.is-transparent a.navbar-item:hover,.navbar.is-transparent a.navbar-item.is-active,.navbar.is-transparent .navbar-link:focus,.navbar.is-transparent .navbar-link:hover,.navbar.is-transparent .navbar-link.is-active{background-color:transparent !important}.navbar.is-transparent .navbar-item.has-dropdown.is-active .navbar-link,.navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:focus .navbar-link,.navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:focus-within .navbar-link,.navbar.is-transparent .navbar-item.has-dropdown.is-hoverable:hover .navbar-link{background-color:transparent !important}.navbar.is-transparent .navbar-dropdown a.navbar-item:focus,.navbar.is-transparent .navbar-dropdown a.navbar-item:hover{background-color:#f5f5f5;color:#0a0a0a}.navbar.is-transparent .navbar-dropdown a.navbar-item.is-active{background-color:#f5f5f5;color:#2e63b8}.navbar-burger{display:none}.navbar-item,.navbar-link{align-items:center;display:flex}.navbar-item.has-dropdown{align-items:stretch}.navbar-item.has-dropdown-up .navbar-link::after{transform:rotate(135deg) translate(0.25em, -0.25em)}.navbar-item.has-dropdown-up .navbar-dropdown{border-bottom:2px solid #dbdbdb;border-radius:6px 6px 0 0;border-top:none;bottom:100%;box-shadow:0 -8px 8px rgba(10,10,10,0.1);top:auto}.navbar-item.is-active .navbar-dropdown,.navbar-item.is-hoverable:focus .navbar-dropdown,.navbar-item.is-hoverable:focus-within .navbar-dropdown,.navbar-item.is-hoverable:hover .navbar-dropdown{display:block}.navbar.is-spaced .navbar-item.is-active .navbar-dropdown,.navbar-item.is-active .navbar-dropdown.is-boxed,.navbar.is-spaced .navbar-item.is-hoverable:focus .navbar-dropdown,.navbar-item.is-hoverable:focus .navbar-dropdown.is-boxed,.navbar.is-spaced .navbar-item.is-hoverable:focus-within .navbar-dropdown,.navbar-item.is-hoverable:focus-within .navbar-dropdown.is-boxed,.navbar.is-spaced .navbar-item.is-hoverable:hover .navbar-dropdown,.navbar-item.is-hoverable:hover .navbar-dropdown.is-boxed{opacity:1;pointer-events:auto;transform:translateY(0)}.navbar-menu{flex-grow:1;flex-shrink:0}.navbar-start{justify-content:flex-start;margin-right:auto}.navbar-end{justify-content:flex-end;margin-left:auto}.navbar-dropdown{background-color:#fff;border-bottom-left-radius:6px;border-bottom-right-radius:6px;border-top:2px solid #dbdbdb;box-shadow:0 8px 8px rgba(10,10,10,0.1);display:none;font-size:0.875rem;left:0;min-width:100%;position:absolute;top:100%;z-index:20}.navbar-dropdown .navbar-item{padding:0.375rem 1rem;white-space:nowrap}.navbar-dropdown a.navbar-item{padding-right:3rem}.navbar-dropdown a.navbar-item:focus,.navbar-dropdown a.navbar-item:hover{background-color:#f5f5f5;color:#0a0a0a}.navbar-dropdown a.navbar-item.is-active{background-color:#f5f5f5;color:#2e63b8}.navbar.is-spaced .navbar-dropdown,.navbar-dropdown.is-boxed{border-radius:6px;border-top:none;box-shadow:0 8px 8px rgba(10,10,10,0.1), 0 0 0 1px rgba(10,10,10,0.1);display:block;opacity:0;pointer-events:none;top:calc(100% + (-4px));transform:translateY(-5px);transition-duration:86ms;transition-property:opacity, transform}.navbar-dropdown.is-right{left:auto;right:0}.navbar-divider{display:block}.navbar>.container .navbar-brand,.container>.navbar .navbar-brand{margin-left:-.75rem}.navbar>.container .navbar-menu,.container>.navbar .navbar-menu{margin-right:-.75rem}.navbar.is-fixed-bottom-desktop,.navbar.is-fixed-top-desktop{left:0;position:fixed;right:0;z-index:30}.navbar.is-fixed-bottom-desktop{bottom:0}.navbar.is-fixed-bottom-desktop.has-shadow{box-shadow:0 -2px 3px rgba(10,10,10,0.1)}.navbar.is-fixed-top-desktop{top:0}html.has-navbar-fixed-top-desktop,body.has-navbar-fixed-top-desktop{padding-top:3.25rem}html.has-navbar-fixed-bottom-desktop,body.has-navbar-fixed-bottom-desktop{padding-bottom:3.25rem}html.has-spaced-navbar-fixed-top,body.has-spaced-navbar-fixed-top{padding-top:5.25rem}html.has-spaced-navbar-fixed-bottom,body.has-spaced-navbar-fixed-bottom{padding-bottom:5.25rem}a.navbar-item.is-active,.navbar-link.is-active{color:#0a0a0a}a.navbar-item.is-active:not(:focus):not(:hover),.navbar-link.is-active:not(:focus):not(:hover){background-color:rgba(0,0,0,0)}.navbar-item.has-dropdown:focus .navbar-link,.navbar-item.has-dropdown:hover .navbar-link,.navbar-item.has-dropdown.is-active .navbar-link{background-color:#fafafa}}.hero.is-fullheight-with-navbar{min-height:calc(100vh - 3.25rem)}.pagination{font-size:1rem;margin:-.25rem}.pagination.is-small,#documenter .docs-sidebar form.docs-search>input.pagination{font-size:.75rem}.pagination.is-medium{font-size:1.25rem}.pagination.is-large{font-size:1.5rem}.pagination.is-rounded .pagination-previous,#documenter .docs-sidebar form.docs-search>input.pagination .pagination-previous,.pagination.is-rounded .pagination-next,#documenter .docs-sidebar form.docs-search>input.pagination .pagination-next{padding-left:1em;padding-right:1em;border-radius:9999px}.pagination.is-rounded .pagination-link,#documenter .docs-sidebar form.docs-search>input.pagination .pagination-link{border-radius:9999px}.pagination,.pagination-list{align-items:center;display:flex;justify-content:center;text-align:center}.pagination-previous,.pagination-next,.pagination-link,.pagination-ellipsis{font-size:1em;justify-content:center;margin:.25rem;padding-left:.5em;padding-right:.5em;text-align:center}.pagination-previous,.pagination-next,.pagination-link{border-color:#dbdbdb;color:#222;min-width:2.5em}.pagination-previous:hover,.pagination-next:hover,.pagination-link:hover{border-color:#b5b5b5;color:#363636}.pagination-previous:focus,.pagination-next:focus,.pagination-link:focus{border-color:#3c5dcd}.pagination-previous:active,.pagination-next:active,.pagination-link:active{box-shadow:inset 0 1px 2px rgba(10,10,10,0.2)}.pagination-previous[disabled],.pagination-previous.is-disabled,.pagination-next[disabled],.pagination-next.is-disabled,.pagination-link[disabled],.pagination-link.is-disabled{background-color:#dbdbdb;border-color:#dbdbdb;box-shadow:none;color:#6b6b6b;opacity:0.5}.pagination-previous,.pagination-next{padding-left:.75em;padding-right:.75em;white-space:nowrap}.pagination-link.is-current{background-color:#2e63b8;border-color:#2e63b8;color:#fff}.pagination-ellipsis{color:#b5b5b5;pointer-events:none}.pagination-list{flex-wrap:wrap}.pagination-list li{list-style:none}@media screen and (max-width: 768px){.pagination{flex-wrap:wrap}.pagination-previous,.pagination-next{flex-grow:1;flex-shrink:1}.pagination-list li{flex-grow:1;flex-shrink:1}}@media screen and (min-width: 769px),print{.pagination-list{flex-grow:1;flex-shrink:1;justify-content:flex-start;order:1}.pagination-previous,.pagination-next,.pagination-link,.pagination-ellipsis{margin-bottom:0;margin-top:0}.pagination-previous{order:2}.pagination-next{order:3}.pagination{justify-content:space-between;margin-bottom:0;margin-top:0}.pagination.is-centered .pagination-previous{order:1}.pagination.is-centered .pagination-list{justify-content:center;order:2}.pagination.is-centered .pagination-next{order:3}.pagination.is-right .pagination-previous{order:1}.pagination.is-right .pagination-next{order:2}.pagination.is-right .pagination-list{justify-content:flex-end;order:3}}.panel{border-radius:6px;box-shadow:#bbb;font-size:1rem}.panel:not(:last-child){margin-bottom:1.5rem}.panel.is-white .panel-heading{background-color:#fff;color:#0a0a0a}.panel.is-white .panel-tabs a.is-active{border-bottom-color:#fff}.panel.is-white .panel-block.is-active .panel-icon{color:#fff}.panel.is-black .panel-heading{background-color:#0a0a0a;color:#fff}.panel.is-black .panel-tabs a.is-active{border-bottom-color:#0a0a0a}.panel.is-black .panel-block.is-active .panel-icon{color:#0a0a0a}.panel.is-light .panel-heading{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.panel.is-light .panel-tabs a.is-active{border-bottom-color:#f5f5f5}.panel.is-light .panel-block.is-active .panel-icon{color:#f5f5f5}.panel.is-dark .panel-heading,.content kbd.panel .panel-heading{background-color:#363636;color:#fff}.panel.is-dark .panel-tabs a.is-active,.content kbd.panel .panel-tabs a.is-active{border-bottom-color:#363636}.panel.is-dark .panel-block.is-active .panel-icon,.content kbd.panel .panel-block.is-active .panel-icon{color:#363636}.panel.is-primary .panel-heading,.docstring>section>a.panel.docs-sourcelink .panel-heading{background-color:#4eb5de;color:#fff}.panel.is-primary .panel-tabs a.is-active,.docstring>section>a.panel.docs-sourcelink .panel-tabs a.is-active{border-bottom-color:#4eb5de}.panel.is-primary .panel-block.is-active .panel-icon,.docstring>section>a.panel.docs-sourcelink .panel-block.is-active .panel-icon{color:#4eb5de}.panel.is-link .panel-heading{background-color:#2e63b8;color:#fff}.panel.is-link .panel-tabs a.is-active{border-bottom-color:#2e63b8}.panel.is-link .panel-block.is-active .panel-icon{color:#2e63b8}.panel.is-info .panel-heading{background-color:#209cee;color:#fff}.panel.is-info .panel-tabs a.is-active{border-bottom-color:#209cee}.panel.is-info .panel-block.is-active .panel-icon{color:#209cee}.panel.is-success .panel-heading{background-color:#22c35b;color:#fff}.panel.is-success .panel-tabs a.is-active{border-bottom-color:#22c35b}.panel.is-success .panel-block.is-active .panel-icon{color:#22c35b}.panel.is-warning .panel-heading{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.panel.is-warning .panel-tabs a.is-active{border-bottom-color:#ffdd57}.panel.is-warning .panel-block.is-active .panel-icon{color:#ffdd57}.panel.is-danger .panel-heading{background-color:#da0b00;color:#fff}.panel.is-danger .panel-tabs a.is-active{border-bottom-color:#da0b00}.panel.is-danger .panel-block.is-active .panel-icon{color:#da0b00}.panel-tabs:not(:last-child),.panel-block:not(:last-child){border-bottom:1px solid #ededed}.panel-heading{background-color:#ededed;border-radius:6px 6px 0 0;color:#222;font-size:1.25em;font-weight:700;line-height:1.25;padding:0.75em 1em}.panel-tabs{align-items:flex-end;display:flex;font-size:.875em;justify-content:center}.panel-tabs a{border-bottom:1px solid #dbdbdb;margin-bottom:-1px;padding:0.5em}.panel-tabs a.is-active{border-bottom-color:#4a4a4a;color:#363636}.panel-list a{color:#222}.panel-list a:hover{color:#2e63b8}.panel-block{align-items:center;color:#222;display:flex;justify-content:flex-start;padding:0.5em 0.75em}.panel-block input[type="checkbox"]{margin-right:.75em}.panel-block>.control{flex-grow:1;flex-shrink:1;width:100%}.panel-block.is-wrapped{flex-wrap:wrap}.panel-block.is-active{border-left-color:#2e63b8;color:#363636}.panel-block.is-active .panel-icon{color:#2e63b8}.panel-block:last-child{border-bottom-left-radius:6px;border-bottom-right-radius:6px}a.panel-block,label.panel-block{cursor:pointer}a.panel-block:hover,label.panel-block:hover{background-color:#f5f5f5}.panel-icon{display:inline-block;font-size:14px;height:1em;line-height:1em;text-align:center;vertical-align:top;width:1em;color:#6b6b6b;margin-right:.75em}.panel-icon .fa{font-size:inherit;line-height:inherit}.tabs{-webkit-overflow-scrolling:touch;align-items:stretch;display:flex;font-size:1rem;justify-content:space-between;overflow:hidden;overflow-x:auto;white-space:nowrap}.tabs a{align-items:center;border-bottom-color:#dbdbdb;border-bottom-style:solid;border-bottom-width:1px;color:#222;display:flex;justify-content:center;margin-bottom:-1px;padding:0.5em 1em;vertical-align:top}.tabs a:hover{border-bottom-color:#222;color:#222}.tabs li{display:block}.tabs li.is-active a{border-bottom-color:#2e63b8;color:#2e63b8}.tabs ul{align-items:center;border-bottom-color:#dbdbdb;border-bottom-style:solid;border-bottom-width:1px;display:flex;flex-grow:1;flex-shrink:0;justify-content:flex-start}.tabs ul.is-left{padding-right:0.75em}.tabs ul.is-center{flex:none;justify-content:center;padding-left:0.75em;padding-right:0.75em}.tabs ul.is-right{justify-content:flex-end;padding-left:0.75em}.tabs .icon:first-child{margin-right:.5em}.tabs .icon:last-child{margin-left:.5em}.tabs.is-centered ul{justify-content:center}.tabs.is-right ul{justify-content:flex-end}.tabs.is-boxed a{border:1px solid transparent;border-radius:4px 4px 0 0}.tabs.is-boxed a:hover{background-color:#f5f5f5;border-bottom-color:#dbdbdb}.tabs.is-boxed li.is-active a{background-color:#fff;border-color:#dbdbdb;border-bottom-color:rgba(0,0,0,0) !important}.tabs.is-fullwidth li{flex-grow:1;flex-shrink:0}.tabs.is-toggle a{border-color:#dbdbdb;border-style:solid;border-width:1px;margin-bottom:0;position:relative}.tabs.is-toggle a:hover{background-color:#f5f5f5;border-color:#b5b5b5;z-index:2}.tabs.is-toggle li+li{margin-left:-1px}.tabs.is-toggle li:first-child a{border-top-left-radius:4px;border-bottom-left-radius:4px}.tabs.is-toggle li:last-child a{border-top-right-radius:4px;border-bottom-right-radius:4px}.tabs.is-toggle li.is-active a{background-color:#2e63b8;border-color:#2e63b8;color:#fff;z-index:1}.tabs.is-toggle ul{border-bottom:none}.tabs.is-toggle.is-toggle-rounded li:first-child a{border-bottom-left-radius:9999px;border-top-left-radius:9999px;padding-left:1.25em}.tabs.is-toggle.is-toggle-rounded li:last-child a{border-bottom-right-radius:9999px;border-top-right-radius:9999px;padding-right:1.25em}.tabs.is-small,#documenter .docs-sidebar form.docs-search>input.tabs{font-size:.75rem}.tabs.is-medium{font-size:1.25rem}.tabs.is-large{font-size:1.5rem}.column{display:block;flex-basis:0;flex-grow:1;flex-shrink:1;padding:.75rem}.columns.is-mobile>.column.is-narrow{flex:none;width:unset}.columns.is-mobile>.column.is-full{flex:none;width:100%}.columns.is-mobile>.column.is-three-quarters{flex:none;width:75%}.columns.is-mobile>.column.is-two-thirds{flex:none;width:66.6666%}.columns.is-mobile>.column.is-half{flex:none;width:50%}.columns.is-mobile>.column.is-one-third{flex:none;width:33.3333%}.columns.is-mobile>.column.is-one-quarter{flex:none;width:25%}.columns.is-mobile>.column.is-one-fifth{flex:none;width:20%}.columns.is-mobile>.column.is-two-fifths{flex:none;width:40%}.columns.is-mobile>.column.is-three-fifths{flex:none;width:60%}.columns.is-mobile>.column.is-four-fifths{flex:none;width:80%}.columns.is-mobile>.column.is-offset-three-quarters{margin-left:75%}.columns.is-mobile>.column.is-offset-two-thirds{margin-left:66.6666%}.columns.is-mobile>.column.is-offset-half{margin-left:50%}.columns.is-mobile>.column.is-offset-one-third{margin-left:33.3333%}.columns.is-mobile>.column.is-offset-one-quarter{margin-left:25%}.columns.is-mobile>.column.is-offset-one-fifth{margin-left:20%}.columns.is-mobile>.column.is-offset-two-fifths{margin-left:40%}.columns.is-mobile>.column.is-offset-three-fifths{margin-left:60%}.columns.is-mobile>.column.is-offset-four-fifths{margin-left:80%}.columns.is-mobile>.column.is-0{flex:none;width:0%}.columns.is-mobile>.column.is-offset-0{margin-left:0%}.columns.is-mobile>.column.is-1{flex:none;width:8.33333337%}.columns.is-mobile>.column.is-offset-1{margin-left:8.33333337%}.columns.is-mobile>.column.is-2{flex:none;width:16.66666674%}.columns.is-mobile>.column.is-offset-2{margin-left:16.66666674%}.columns.is-mobile>.column.is-3{flex:none;width:25%}.columns.is-mobile>.column.is-offset-3{margin-left:25%}.columns.is-mobile>.column.is-4{flex:none;width:33.33333337%}.columns.is-mobile>.column.is-offset-4{margin-left:33.33333337%}.columns.is-mobile>.column.is-5{flex:none;width:41.66666674%}.columns.is-mobile>.column.is-offset-5{margin-left:41.66666674%}.columns.is-mobile>.column.is-6{flex:none;width:50%}.columns.is-mobile>.column.is-offset-6{margin-left:50%}.columns.is-mobile>.column.is-7{flex:none;width:58.33333337%}.columns.is-mobile>.column.is-offset-7{margin-left:58.33333337%}.columns.is-mobile>.column.is-8{flex:none;width:66.66666674%}.columns.is-mobile>.column.is-offset-8{margin-left:66.66666674%}.columns.is-mobile>.column.is-9{flex:none;width:75%}.columns.is-mobile>.column.is-offset-9{margin-left:75%}.columns.is-mobile>.column.is-10{flex:none;width:83.33333337%}.columns.is-mobile>.column.is-offset-10{margin-left:83.33333337%}.columns.is-mobile>.column.is-11{flex:none;width:91.66666674%}.columns.is-mobile>.column.is-offset-11{margin-left:91.66666674%}.columns.is-mobile>.column.is-12{flex:none;width:100%}.columns.is-mobile>.column.is-offset-12{margin-left:100%}@media screen and (max-width: 768px){.column.is-narrow-mobile{flex:none;width:unset}.column.is-full-mobile{flex:none;width:100%}.column.is-three-quarters-mobile{flex:none;width:75%}.column.is-two-thirds-mobile{flex:none;width:66.6666%}.column.is-half-mobile{flex:none;width:50%}.column.is-one-third-mobile{flex:none;width:33.3333%}.column.is-one-quarter-mobile{flex:none;width:25%}.column.is-one-fifth-mobile{flex:none;width:20%}.column.is-two-fifths-mobile{flex:none;width:40%}.column.is-three-fifths-mobile{flex:none;width:60%}.column.is-four-fifths-mobile{flex:none;width:80%}.column.is-offset-three-quarters-mobile{margin-left:75%}.column.is-offset-two-thirds-mobile{margin-left:66.6666%}.column.is-offset-half-mobile{margin-left:50%}.column.is-offset-one-third-mobile{margin-left:33.3333%}.column.is-offset-one-quarter-mobile{margin-left:25%}.column.is-offset-one-fifth-mobile{margin-left:20%}.column.is-offset-two-fifths-mobile{margin-left:40%}.column.is-offset-three-fifths-mobile{margin-left:60%}.column.is-offset-four-fifths-mobile{margin-left:80%}.column.is-0-mobile{flex:none;width:0%}.column.is-offset-0-mobile{margin-left:0%}.column.is-1-mobile{flex:none;width:8.33333337%}.column.is-offset-1-mobile{margin-left:8.33333337%}.column.is-2-mobile{flex:none;width:16.66666674%}.column.is-offset-2-mobile{margin-left:16.66666674%}.column.is-3-mobile{flex:none;width:25%}.column.is-offset-3-mobile{margin-left:25%}.column.is-4-mobile{flex:none;width:33.33333337%}.column.is-offset-4-mobile{margin-left:33.33333337%}.column.is-5-mobile{flex:none;width:41.66666674%}.column.is-offset-5-mobile{margin-left:41.66666674%}.column.is-6-mobile{flex:none;width:50%}.column.is-offset-6-mobile{margin-left:50%}.column.is-7-mobile{flex:none;width:58.33333337%}.column.is-offset-7-mobile{margin-left:58.33333337%}.column.is-8-mobile{flex:none;width:66.66666674%}.column.is-offset-8-mobile{margin-left:66.66666674%}.column.is-9-mobile{flex:none;width:75%}.column.is-offset-9-mobile{margin-left:75%}.column.is-10-mobile{flex:none;width:83.33333337%}.column.is-offset-10-mobile{margin-left:83.33333337%}.column.is-11-mobile{flex:none;width:91.66666674%}.column.is-offset-11-mobile{margin-left:91.66666674%}.column.is-12-mobile{flex:none;width:100%}.column.is-offset-12-mobile{margin-left:100%}}@media screen and (min-width: 769px),print{.column.is-narrow,.column.is-narrow-tablet{flex:none;width:unset}.column.is-full,.column.is-full-tablet{flex:none;width:100%}.column.is-three-quarters,.column.is-three-quarters-tablet{flex:none;width:75%}.column.is-two-thirds,.column.is-two-thirds-tablet{flex:none;width:66.6666%}.column.is-half,.column.is-half-tablet{flex:none;width:50%}.column.is-one-third,.column.is-one-third-tablet{flex:none;width:33.3333%}.column.is-one-quarter,.column.is-one-quarter-tablet{flex:none;width:25%}.column.is-one-fifth,.column.is-one-fifth-tablet{flex:none;width:20%}.column.is-two-fifths,.column.is-two-fifths-tablet{flex:none;width:40%}.column.is-three-fifths,.column.is-three-fifths-tablet{flex:none;width:60%}.column.is-four-fifths,.column.is-four-fifths-tablet{flex:none;width:80%}.column.is-offset-three-quarters,.column.is-offset-three-quarters-tablet{margin-left:75%}.column.is-offset-two-thirds,.column.is-offset-two-thirds-tablet{margin-left:66.6666%}.column.is-offset-half,.column.is-offset-half-tablet{margin-left:50%}.column.is-offset-one-third,.column.is-offset-one-third-tablet{margin-left:33.3333%}.column.is-offset-one-quarter,.column.is-offset-one-quarter-tablet{margin-left:25%}.column.is-offset-one-fifth,.column.is-offset-one-fifth-tablet{margin-left:20%}.column.is-offset-two-fifths,.column.is-offset-two-fifths-tablet{margin-left:40%}.column.is-offset-three-fifths,.column.is-offset-three-fifths-tablet{margin-left:60%}.column.is-offset-four-fifths,.column.is-offset-four-fifths-tablet{margin-left:80%}.column.is-0,.column.is-0-tablet{flex:none;width:0%}.column.is-offset-0,.column.is-offset-0-tablet{margin-left:0%}.column.is-1,.column.is-1-tablet{flex:none;width:8.33333337%}.column.is-offset-1,.column.is-offset-1-tablet{margin-left:8.33333337%}.column.is-2,.column.is-2-tablet{flex:none;width:16.66666674%}.column.is-offset-2,.column.is-offset-2-tablet{margin-left:16.66666674%}.column.is-3,.column.is-3-tablet{flex:none;width:25%}.column.is-offset-3,.column.is-offset-3-tablet{margin-left:25%}.column.is-4,.column.is-4-tablet{flex:none;width:33.33333337%}.column.is-offset-4,.column.is-offset-4-tablet{margin-left:33.33333337%}.column.is-5,.column.is-5-tablet{flex:none;width:41.66666674%}.column.is-offset-5,.column.is-offset-5-tablet{margin-left:41.66666674%}.column.is-6,.column.is-6-tablet{flex:none;width:50%}.column.is-offset-6,.column.is-offset-6-tablet{margin-left:50%}.column.is-7,.column.is-7-tablet{flex:none;width:58.33333337%}.column.is-offset-7,.column.is-offset-7-tablet{margin-left:58.33333337%}.column.is-8,.column.is-8-tablet{flex:none;width:66.66666674%}.column.is-offset-8,.column.is-offset-8-tablet{margin-left:66.66666674%}.column.is-9,.column.is-9-tablet{flex:none;width:75%}.column.is-offset-9,.column.is-offset-9-tablet{margin-left:75%}.column.is-10,.column.is-10-tablet{flex:none;width:83.33333337%}.column.is-offset-10,.column.is-offset-10-tablet{margin-left:83.33333337%}.column.is-11,.column.is-11-tablet{flex:none;width:91.66666674%}.column.is-offset-11,.column.is-offset-11-tablet{margin-left:91.66666674%}.column.is-12,.column.is-12-tablet{flex:none;width:100%}.column.is-offset-12,.column.is-offset-12-tablet{margin-left:100%}}@media screen and (max-width: 1055px){.column.is-narrow-touch{flex:none;width:unset}.column.is-full-touch{flex:none;width:100%}.column.is-three-quarters-touch{flex:none;width:75%}.column.is-two-thirds-touch{flex:none;width:66.6666%}.column.is-half-touch{flex:none;width:50%}.column.is-one-third-touch{flex:none;width:33.3333%}.column.is-one-quarter-touch{flex:none;width:25%}.column.is-one-fifth-touch{flex:none;width:20%}.column.is-two-fifths-touch{flex:none;width:40%}.column.is-three-fifths-touch{flex:none;width:60%}.column.is-four-fifths-touch{flex:none;width:80%}.column.is-offset-three-quarters-touch{margin-left:75%}.column.is-offset-two-thirds-touch{margin-left:66.6666%}.column.is-offset-half-touch{margin-left:50%}.column.is-offset-one-third-touch{margin-left:33.3333%}.column.is-offset-one-quarter-touch{margin-left:25%}.column.is-offset-one-fifth-touch{margin-left:20%}.column.is-offset-two-fifths-touch{margin-left:40%}.column.is-offset-three-fifths-touch{margin-left:60%}.column.is-offset-four-fifths-touch{margin-left:80%}.column.is-0-touch{flex:none;width:0%}.column.is-offset-0-touch{margin-left:0%}.column.is-1-touch{flex:none;width:8.33333337%}.column.is-offset-1-touch{margin-left:8.33333337%}.column.is-2-touch{flex:none;width:16.66666674%}.column.is-offset-2-touch{margin-left:16.66666674%}.column.is-3-touch{flex:none;width:25%}.column.is-offset-3-touch{margin-left:25%}.column.is-4-touch{flex:none;width:33.33333337%}.column.is-offset-4-touch{margin-left:33.33333337%}.column.is-5-touch{flex:none;width:41.66666674%}.column.is-offset-5-touch{margin-left:41.66666674%}.column.is-6-touch{flex:none;width:50%}.column.is-offset-6-touch{margin-left:50%}.column.is-7-touch{flex:none;width:58.33333337%}.column.is-offset-7-touch{margin-left:58.33333337%}.column.is-8-touch{flex:none;width:66.66666674%}.column.is-offset-8-touch{margin-left:66.66666674%}.column.is-9-touch{flex:none;width:75%}.column.is-offset-9-touch{margin-left:75%}.column.is-10-touch{flex:none;width:83.33333337%}.column.is-offset-10-touch{margin-left:83.33333337%}.column.is-11-touch{flex:none;width:91.66666674%}.column.is-offset-11-touch{margin-left:91.66666674%}.column.is-12-touch{flex:none;width:100%}.column.is-offset-12-touch{margin-left:100%}}@media screen and (min-width: 1056px){.column.is-narrow-desktop{flex:none;width:unset}.column.is-full-desktop{flex:none;width:100%}.column.is-three-quarters-desktop{flex:none;width:75%}.column.is-two-thirds-desktop{flex:none;width:66.6666%}.column.is-half-desktop{flex:none;width:50%}.column.is-one-third-desktop{flex:none;width:33.3333%}.column.is-one-quarter-desktop{flex:none;width:25%}.column.is-one-fifth-desktop{flex:none;width:20%}.column.is-two-fifths-desktop{flex:none;width:40%}.column.is-three-fifths-desktop{flex:none;width:60%}.column.is-four-fifths-desktop{flex:none;width:80%}.column.is-offset-three-quarters-desktop{margin-left:75%}.column.is-offset-two-thirds-desktop{margin-left:66.6666%}.column.is-offset-half-desktop{margin-left:50%}.column.is-offset-one-third-desktop{margin-left:33.3333%}.column.is-offset-one-quarter-desktop{margin-left:25%}.column.is-offset-one-fifth-desktop{margin-left:20%}.column.is-offset-two-fifths-desktop{margin-left:40%}.column.is-offset-three-fifths-desktop{margin-left:60%}.column.is-offset-four-fifths-desktop{margin-left:80%}.column.is-0-desktop{flex:none;width:0%}.column.is-offset-0-desktop{margin-left:0%}.column.is-1-desktop{flex:none;width:8.33333337%}.column.is-offset-1-desktop{margin-left:8.33333337%}.column.is-2-desktop{flex:none;width:16.66666674%}.column.is-offset-2-desktop{margin-left:16.66666674%}.column.is-3-desktop{flex:none;width:25%}.column.is-offset-3-desktop{margin-left:25%}.column.is-4-desktop{flex:none;width:33.33333337%}.column.is-offset-4-desktop{margin-left:33.33333337%}.column.is-5-desktop{flex:none;width:41.66666674%}.column.is-offset-5-desktop{margin-left:41.66666674%}.column.is-6-desktop{flex:none;width:50%}.column.is-offset-6-desktop{margin-left:50%}.column.is-7-desktop{flex:none;width:58.33333337%}.column.is-offset-7-desktop{margin-left:58.33333337%}.column.is-8-desktop{flex:none;width:66.66666674%}.column.is-offset-8-desktop{margin-left:66.66666674%}.column.is-9-desktop{flex:none;width:75%}.column.is-offset-9-desktop{margin-left:75%}.column.is-10-desktop{flex:none;width:83.33333337%}.column.is-offset-10-desktop{margin-left:83.33333337%}.column.is-11-desktop{flex:none;width:91.66666674%}.column.is-offset-11-desktop{margin-left:91.66666674%}.column.is-12-desktop{flex:none;width:100%}.column.is-offset-12-desktop{margin-left:100%}}@media screen and (min-width: 1216px){.column.is-narrow-widescreen{flex:none;width:unset}.column.is-full-widescreen{flex:none;width:100%}.column.is-three-quarters-widescreen{flex:none;width:75%}.column.is-two-thirds-widescreen{flex:none;width:66.6666%}.column.is-half-widescreen{flex:none;width:50%}.column.is-one-third-widescreen{flex:none;width:33.3333%}.column.is-one-quarter-widescreen{flex:none;width:25%}.column.is-one-fifth-widescreen{flex:none;width:20%}.column.is-two-fifths-widescreen{flex:none;width:40%}.column.is-three-fifths-widescreen{flex:none;width:60%}.column.is-four-fifths-widescreen{flex:none;width:80%}.column.is-offset-three-quarters-widescreen{margin-left:75%}.column.is-offset-two-thirds-widescreen{margin-left:66.6666%}.column.is-offset-half-widescreen{margin-left:50%}.column.is-offset-one-third-widescreen{margin-left:33.3333%}.column.is-offset-one-quarter-widescreen{margin-left:25%}.column.is-offset-one-fifth-widescreen{margin-left:20%}.column.is-offset-two-fifths-widescreen{margin-left:40%}.column.is-offset-three-fifths-widescreen{margin-left:60%}.column.is-offset-four-fifths-widescreen{margin-left:80%}.column.is-0-widescreen{flex:none;width:0%}.column.is-offset-0-widescreen{margin-left:0%}.column.is-1-widescreen{flex:none;width:8.33333337%}.column.is-offset-1-widescreen{margin-left:8.33333337%}.column.is-2-widescreen{flex:none;width:16.66666674%}.column.is-offset-2-widescreen{margin-left:16.66666674%}.column.is-3-widescreen{flex:none;width:25%}.column.is-offset-3-widescreen{margin-left:25%}.column.is-4-widescreen{flex:none;width:33.33333337%}.column.is-offset-4-widescreen{margin-left:33.33333337%}.column.is-5-widescreen{flex:none;width:41.66666674%}.column.is-offset-5-widescreen{margin-left:41.66666674%}.column.is-6-widescreen{flex:none;width:50%}.column.is-offset-6-widescreen{margin-left:50%}.column.is-7-widescreen{flex:none;width:58.33333337%}.column.is-offset-7-widescreen{margin-left:58.33333337%}.column.is-8-widescreen{flex:none;width:66.66666674%}.column.is-offset-8-widescreen{margin-left:66.66666674%}.column.is-9-widescreen{flex:none;width:75%}.column.is-offset-9-widescreen{margin-left:75%}.column.is-10-widescreen{flex:none;width:83.33333337%}.column.is-offset-10-widescreen{margin-left:83.33333337%}.column.is-11-widescreen{flex:none;width:91.66666674%}.column.is-offset-11-widescreen{margin-left:91.66666674%}.column.is-12-widescreen{flex:none;width:100%}.column.is-offset-12-widescreen{margin-left:100%}}@media screen and (min-width: 1408px){.column.is-narrow-fullhd{flex:none;width:unset}.column.is-full-fullhd{flex:none;width:100%}.column.is-three-quarters-fullhd{flex:none;width:75%}.column.is-two-thirds-fullhd{flex:none;width:66.6666%}.column.is-half-fullhd{flex:none;width:50%}.column.is-one-third-fullhd{flex:none;width:33.3333%}.column.is-one-quarter-fullhd{flex:none;width:25%}.column.is-one-fifth-fullhd{flex:none;width:20%}.column.is-two-fifths-fullhd{flex:none;width:40%}.column.is-three-fifths-fullhd{flex:none;width:60%}.column.is-four-fifths-fullhd{flex:none;width:80%}.column.is-offset-three-quarters-fullhd{margin-left:75%}.column.is-offset-two-thirds-fullhd{margin-left:66.6666%}.column.is-offset-half-fullhd{margin-left:50%}.column.is-offset-one-third-fullhd{margin-left:33.3333%}.column.is-offset-one-quarter-fullhd{margin-left:25%}.column.is-offset-one-fifth-fullhd{margin-left:20%}.column.is-offset-two-fifths-fullhd{margin-left:40%}.column.is-offset-three-fifths-fullhd{margin-left:60%}.column.is-offset-four-fifths-fullhd{margin-left:80%}.column.is-0-fullhd{flex:none;width:0%}.column.is-offset-0-fullhd{margin-left:0%}.column.is-1-fullhd{flex:none;width:8.33333337%}.column.is-offset-1-fullhd{margin-left:8.33333337%}.column.is-2-fullhd{flex:none;width:16.66666674%}.column.is-offset-2-fullhd{margin-left:16.66666674%}.column.is-3-fullhd{flex:none;width:25%}.column.is-offset-3-fullhd{margin-left:25%}.column.is-4-fullhd{flex:none;width:33.33333337%}.column.is-offset-4-fullhd{margin-left:33.33333337%}.column.is-5-fullhd{flex:none;width:41.66666674%}.column.is-offset-5-fullhd{margin-left:41.66666674%}.column.is-6-fullhd{flex:none;width:50%}.column.is-offset-6-fullhd{margin-left:50%}.column.is-7-fullhd{flex:none;width:58.33333337%}.column.is-offset-7-fullhd{margin-left:58.33333337%}.column.is-8-fullhd{flex:none;width:66.66666674%}.column.is-offset-8-fullhd{margin-left:66.66666674%}.column.is-9-fullhd{flex:none;width:75%}.column.is-offset-9-fullhd{margin-left:75%}.column.is-10-fullhd{flex:none;width:83.33333337%}.column.is-offset-10-fullhd{margin-left:83.33333337%}.column.is-11-fullhd{flex:none;width:91.66666674%}.column.is-offset-11-fullhd{margin-left:91.66666674%}.column.is-12-fullhd{flex:none;width:100%}.column.is-offset-12-fullhd{margin-left:100%}}.columns{margin-left:-.75rem;margin-right:-.75rem;margin-top:-.75rem}.columns:last-child{margin-bottom:-.75rem}.columns:not(:last-child){margin-bottom:calc(1.5rem - .75rem)}.columns.is-centered{justify-content:center}.columns.is-gapless{margin-left:0;margin-right:0;margin-top:0}.columns.is-gapless>.column{margin:0;padding:0 !important}.columns.is-gapless:not(:last-child){margin-bottom:1.5rem}.columns.is-gapless:last-child{margin-bottom:0}.columns.is-mobile{display:flex}.columns.is-multiline{flex-wrap:wrap}.columns.is-vcentered{align-items:center}@media screen and (min-width: 769px),print{.columns:not(.is-desktop){display:flex}}@media screen and (min-width: 1056px){.columns.is-desktop{display:flex}}.columns.is-variable{--columnGap: 0.75rem;margin-left:calc(-1 * var(--columnGap));margin-right:calc(-1 * var(--columnGap))}.columns.is-variable>.column{padding-left:var(--columnGap);padding-right:var(--columnGap)}.columns.is-variable.is-0{--columnGap: 0rem}@media screen and (max-width: 768px){.columns.is-variable.is-0-mobile{--columnGap: 0rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-0-tablet{--columnGap: 0rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-0-tablet-only{--columnGap: 0rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-0-touch{--columnGap: 0rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-0-desktop{--columnGap: 0rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-0-desktop-only{--columnGap: 0rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-0-widescreen{--columnGap: 0rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-0-widescreen-only{--columnGap: 0rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-0-fullhd{--columnGap: 0rem}}.columns.is-variable.is-1{--columnGap: .25rem}@media screen and (max-width: 768px){.columns.is-variable.is-1-mobile{--columnGap: .25rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-1-tablet{--columnGap: .25rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-1-tablet-only{--columnGap: .25rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-1-touch{--columnGap: .25rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-1-desktop{--columnGap: .25rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-1-desktop-only{--columnGap: .25rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-1-widescreen{--columnGap: .25rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-1-widescreen-only{--columnGap: .25rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-1-fullhd{--columnGap: .25rem}}.columns.is-variable.is-2{--columnGap: .5rem}@media screen and (max-width: 768px){.columns.is-variable.is-2-mobile{--columnGap: .5rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-2-tablet{--columnGap: .5rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-2-tablet-only{--columnGap: .5rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-2-touch{--columnGap: .5rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-2-desktop{--columnGap: .5rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-2-desktop-only{--columnGap: .5rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-2-widescreen{--columnGap: .5rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-2-widescreen-only{--columnGap: .5rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-2-fullhd{--columnGap: .5rem}}.columns.is-variable.is-3{--columnGap: .75rem}@media screen and (max-width: 768px){.columns.is-variable.is-3-mobile{--columnGap: .75rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-3-tablet{--columnGap: .75rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-3-tablet-only{--columnGap: .75rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-3-touch{--columnGap: .75rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-3-desktop{--columnGap: .75rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-3-desktop-only{--columnGap: .75rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-3-widescreen{--columnGap: .75rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-3-widescreen-only{--columnGap: .75rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-3-fullhd{--columnGap: .75rem}}.columns.is-variable.is-4{--columnGap: 1rem}@media screen and (max-width: 768px){.columns.is-variable.is-4-mobile{--columnGap: 1rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-4-tablet{--columnGap: 1rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-4-tablet-only{--columnGap: 1rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-4-touch{--columnGap: 1rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-4-desktop{--columnGap: 1rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-4-desktop-only{--columnGap: 1rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-4-widescreen{--columnGap: 1rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-4-widescreen-only{--columnGap: 1rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-4-fullhd{--columnGap: 1rem}}.columns.is-variable.is-5{--columnGap: 1.25rem}@media screen and (max-width: 768px){.columns.is-variable.is-5-mobile{--columnGap: 1.25rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-5-tablet{--columnGap: 1.25rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-5-tablet-only{--columnGap: 1.25rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-5-touch{--columnGap: 1.25rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-5-desktop{--columnGap: 1.25rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-5-desktop-only{--columnGap: 1.25rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-5-widescreen{--columnGap: 1.25rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-5-widescreen-only{--columnGap: 1.25rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-5-fullhd{--columnGap: 1.25rem}}.columns.is-variable.is-6{--columnGap: 1.5rem}@media screen and (max-width: 768px){.columns.is-variable.is-6-mobile{--columnGap: 1.5rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-6-tablet{--columnGap: 1.5rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-6-tablet-only{--columnGap: 1.5rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-6-touch{--columnGap: 1.5rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-6-desktop{--columnGap: 1.5rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-6-desktop-only{--columnGap: 1.5rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-6-widescreen{--columnGap: 1.5rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-6-widescreen-only{--columnGap: 1.5rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-6-fullhd{--columnGap: 1.5rem}}.columns.is-variable.is-7{--columnGap: 1.75rem}@media screen and (max-width: 768px){.columns.is-variable.is-7-mobile{--columnGap: 1.75rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-7-tablet{--columnGap: 1.75rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-7-tablet-only{--columnGap: 1.75rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-7-touch{--columnGap: 1.75rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-7-desktop{--columnGap: 1.75rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-7-desktop-only{--columnGap: 1.75rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-7-widescreen{--columnGap: 1.75rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-7-widescreen-only{--columnGap: 1.75rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-7-fullhd{--columnGap: 1.75rem}}.columns.is-variable.is-8{--columnGap: 2rem}@media screen and (max-width: 768px){.columns.is-variable.is-8-mobile{--columnGap: 2rem}}@media screen and (min-width: 769px),print{.columns.is-variable.is-8-tablet{--columnGap: 2rem}}@media screen and (min-width: 769px) and (max-width: 1055px){.columns.is-variable.is-8-tablet-only{--columnGap: 2rem}}@media screen and (max-width: 1055px){.columns.is-variable.is-8-touch{--columnGap: 2rem}}@media screen and (min-width: 1056px){.columns.is-variable.is-8-desktop{--columnGap: 2rem}}@media screen and (min-width: 1056px) and (max-width: 1215px){.columns.is-variable.is-8-desktop-only{--columnGap: 2rem}}@media screen and (min-width: 1216px){.columns.is-variable.is-8-widescreen{--columnGap: 2rem}}@media screen and (min-width: 1216px) and (max-width: 1407px){.columns.is-variable.is-8-widescreen-only{--columnGap: 2rem}}@media screen and (min-width: 1408px){.columns.is-variable.is-8-fullhd{--columnGap: 2rem}}.tile{align-items:stretch;display:block;flex-basis:0;flex-grow:1;flex-shrink:1;min-height:min-content}.tile.is-ancestor{margin-left:-.75rem;margin-right:-.75rem;margin-top:-.75rem}.tile.is-ancestor:last-child{margin-bottom:-.75rem}.tile.is-ancestor:not(:last-child){margin-bottom:.75rem}.tile.is-child{margin:0 !important}.tile.is-parent{padding:.75rem}.tile.is-vertical{flex-direction:column}.tile.is-vertical>.tile.is-child:not(:last-child){margin-bottom:1.5rem !important}@media screen and (min-width: 769px),print{.tile:not(.is-child){display:flex}.tile.is-1{flex:none;width:8.33333337%}.tile.is-2{flex:none;width:16.66666674%}.tile.is-3{flex:none;width:25%}.tile.is-4{flex:none;width:33.33333337%}.tile.is-5{flex:none;width:41.66666674%}.tile.is-6{flex:none;width:50%}.tile.is-7{flex:none;width:58.33333337%}.tile.is-8{flex:none;width:66.66666674%}.tile.is-9{flex:none;width:75%}.tile.is-10{flex:none;width:83.33333337%}.tile.is-11{flex:none;width:91.66666674%}.tile.is-12{flex:none;width:100%}}.hero{align-items:stretch;display:flex;flex-direction:column;justify-content:space-between}.hero .navbar{background:none}.hero .tabs ul{border-bottom:none}.hero.is-white{background-color:#fff;color:#0a0a0a}.hero.is-white a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-white strong{color:inherit}.hero.is-white .title{color:#0a0a0a}.hero.is-white .subtitle{color:rgba(10,10,10,0.9)}.hero.is-white .subtitle a:not(.button),.hero.is-white .subtitle strong{color:#0a0a0a}@media screen and (max-width: 1055px){.hero.is-white .navbar-menu{background-color:#fff}}.hero.is-white .navbar-item,.hero.is-white .navbar-link{color:rgba(10,10,10,0.7)}.hero.is-white a.navbar-item:hover,.hero.is-white a.navbar-item.is-active,.hero.is-white .navbar-link:hover,.hero.is-white .navbar-link.is-active{background-color:#f2f2f2;color:#0a0a0a}.hero.is-white .tabs a{color:#0a0a0a;opacity:0.9}.hero.is-white .tabs a:hover{opacity:1}.hero.is-white .tabs li.is-active a{color:#fff !important;opacity:1}.hero.is-white .tabs.is-boxed a,.hero.is-white .tabs.is-toggle a{color:#0a0a0a}.hero.is-white .tabs.is-boxed a:hover,.hero.is-white .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-white .tabs.is-boxed li.is-active a,.hero.is-white .tabs.is-boxed li.is-active a:hover,.hero.is-white .tabs.is-toggle li.is-active a,.hero.is-white .tabs.is-toggle li.is-active a:hover{background-color:#0a0a0a;border-color:#0a0a0a;color:#fff}.hero.is-white.is-bold{background-image:linear-gradient(141deg, #e8e3e4 0%, #fff 71%, #fff 100%)}@media screen and (max-width: 768px){.hero.is-white.is-bold .navbar-menu{background-image:linear-gradient(141deg, #e8e3e4 0%, #fff 71%, #fff 100%)}}.hero.is-black{background-color:#0a0a0a;color:#fff}.hero.is-black a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-black strong{color:inherit}.hero.is-black .title{color:#fff}.hero.is-black .subtitle{color:rgba(255,255,255,0.9)}.hero.is-black .subtitle a:not(.button),.hero.is-black .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-black .navbar-menu{background-color:#0a0a0a}}.hero.is-black .navbar-item,.hero.is-black .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-black a.navbar-item:hover,.hero.is-black a.navbar-item.is-active,.hero.is-black .navbar-link:hover,.hero.is-black .navbar-link.is-active{background-color:#000;color:#fff}.hero.is-black .tabs a{color:#fff;opacity:0.9}.hero.is-black .tabs a:hover{opacity:1}.hero.is-black .tabs li.is-active a{color:#0a0a0a !important;opacity:1}.hero.is-black .tabs.is-boxed a,.hero.is-black .tabs.is-toggle a{color:#fff}.hero.is-black .tabs.is-boxed a:hover,.hero.is-black .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-black .tabs.is-boxed li.is-active a,.hero.is-black .tabs.is-boxed li.is-active a:hover,.hero.is-black .tabs.is-toggle li.is-active a,.hero.is-black .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#0a0a0a}.hero.is-black.is-bold{background-image:linear-gradient(141deg, #000 0%, #0a0a0a 71%, #181616 100%)}@media screen and (max-width: 768px){.hero.is-black.is-bold .navbar-menu{background-image:linear-gradient(141deg, #000 0%, #0a0a0a 71%, #181616 100%)}}.hero.is-light{background-color:#f5f5f5;color:rgba(0,0,0,0.7)}.hero.is-light a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-light strong{color:inherit}.hero.is-light .title{color:rgba(0,0,0,0.7)}.hero.is-light .subtitle{color:rgba(0,0,0,0.9)}.hero.is-light .subtitle a:not(.button),.hero.is-light .subtitle strong{color:rgba(0,0,0,0.7)}@media screen and (max-width: 1055px){.hero.is-light .navbar-menu{background-color:#f5f5f5}}.hero.is-light .navbar-item,.hero.is-light .navbar-link{color:rgba(0,0,0,0.7)}.hero.is-light a.navbar-item:hover,.hero.is-light a.navbar-item.is-active,.hero.is-light .navbar-link:hover,.hero.is-light .navbar-link.is-active{background-color:#e8e8e8;color:rgba(0,0,0,0.7)}.hero.is-light .tabs a{color:rgba(0,0,0,0.7);opacity:0.9}.hero.is-light .tabs a:hover{opacity:1}.hero.is-light .tabs li.is-active a{color:#f5f5f5 !important;opacity:1}.hero.is-light .tabs.is-boxed a,.hero.is-light .tabs.is-toggle a{color:rgba(0,0,0,0.7)}.hero.is-light .tabs.is-boxed a:hover,.hero.is-light .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-light .tabs.is-boxed li.is-active a,.hero.is-light .tabs.is-boxed li.is-active a:hover,.hero.is-light .tabs.is-toggle li.is-active a,.hero.is-light .tabs.is-toggle li.is-active a:hover{background-color:rgba(0,0,0,0.7);border-color:rgba(0,0,0,0.7);color:#f5f5f5}.hero.is-light.is-bold{background-image:linear-gradient(141deg, #dfd8d9 0%, #f5f5f5 71%, #fff 100%)}@media screen and (max-width: 768px){.hero.is-light.is-bold .navbar-menu{background-image:linear-gradient(141deg, #dfd8d9 0%, #f5f5f5 71%, #fff 100%)}}.hero.is-dark,.content kbd.hero{background-color:#363636;color:#fff}.hero.is-dark a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.content kbd.hero a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-dark strong,.content kbd.hero strong{color:inherit}.hero.is-dark .title,.content kbd.hero .title{color:#fff}.hero.is-dark .subtitle,.content kbd.hero .subtitle{color:rgba(255,255,255,0.9)}.hero.is-dark .subtitle a:not(.button),.content kbd.hero .subtitle a:not(.button),.hero.is-dark .subtitle strong,.content kbd.hero .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-dark .navbar-menu,.content kbd.hero .navbar-menu{background-color:#363636}}.hero.is-dark .navbar-item,.content kbd.hero .navbar-item,.hero.is-dark .navbar-link,.content kbd.hero .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-dark a.navbar-item:hover,.content kbd.hero a.navbar-item:hover,.hero.is-dark a.navbar-item.is-active,.content kbd.hero a.navbar-item.is-active,.hero.is-dark .navbar-link:hover,.content kbd.hero .navbar-link:hover,.hero.is-dark .navbar-link.is-active,.content kbd.hero .navbar-link.is-active{background-color:#292929;color:#fff}.hero.is-dark .tabs a,.content kbd.hero .tabs a{color:#fff;opacity:0.9}.hero.is-dark .tabs a:hover,.content kbd.hero .tabs a:hover{opacity:1}.hero.is-dark .tabs li.is-active a,.content kbd.hero .tabs li.is-active a{color:#363636 !important;opacity:1}.hero.is-dark .tabs.is-boxed a,.content kbd.hero .tabs.is-boxed a,.hero.is-dark .tabs.is-toggle a,.content kbd.hero .tabs.is-toggle a{color:#fff}.hero.is-dark .tabs.is-boxed a:hover,.content kbd.hero .tabs.is-boxed a:hover,.hero.is-dark .tabs.is-toggle a:hover,.content kbd.hero .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-dark .tabs.is-boxed li.is-active a,.content kbd.hero .tabs.is-boxed li.is-active a,.hero.is-dark .tabs.is-boxed li.is-active a:hover,.hero.is-dark .tabs.is-toggle li.is-active a,.content kbd.hero .tabs.is-toggle li.is-active a,.hero.is-dark .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#363636}.hero.is-dark.is-bold,.content kbd.hero.is-bold{background-image:linear-gradient(141deg, #1f191a 0%, #363636 71%, #46403f 100%)}@media screen and (max-width: 768px){.hero.is-dark.is-bold .navbar-menu,.content kbd.hero.is-bold .navbar-menu{background-image:linear-gradient(141deg, #1f191a 0%, #363636 71%, #46403f 100%)}}.hero.is-primary,.docstring>section>a.hero.docs-sourcelink{background-color:#4eb5de;color:#fff}.hero.is-primary a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.docstring>section>a.hero.docs-sourcelink a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-primary strong,.docstring>section>a.hero.docs-sourcelink strong{color:inherit}.hero.is-primary .title,.docstring>section>a.hero.docs-sourcelink .title{color:#fff}.hero.is-primary .subtitle,.docstring>section>a.hero.docs-sourcelink .subtitle{color:rgba(255,255,255,0.9)}.hero.is-primary .subtitle a:not(.button),.docstring>section>a.hero.docs-sourcelink .subtitle a:not(.button),.hero.is-primary .subtitle strong,.docstring>section>a.hero.docs-sourcelink .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-primary .navbar-menu,.docstring>section>a.hero.docs-sourcelink .navbar-menu{background-color:#4eb5de}}.hero.is-primary .navbar-item,.docstring>section>a.hero.docs-sourcelink .navbar-item,.hero.is-primary .navbar-link,.docstring>section>a.hero.docs-sourcelink .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-primary a.navbar-item:hover,.docstring>section>a.hero.docs-sourcelink a.navbar-item:hover,.hero.is-primary a.navbar-item.is-active,.docstring>section>a.hero.docs-sourcelink a.navbar-item.is-active,.hero.is-primary .navbar-link:hover,.docstring>section>a.hero.docs-sourcelink .navbar-link:hover,.hero.is-primary .navbar-link.is-active,.docstring>section>a.hero.docs-sourcelink .navbar-link.is-active{background-color:#39acda;color:#fff}.hero.is-primary .tabs a,.docstring>section>a.hero.docs-sourcelink .tabs a{color:#fff;opacity:0.9}.hero.is-primary .tabs a:hover,.docstring>section>a.hero.docs-sourcelink .tabs a:hover{opacity:1}.hero.is-primary .tabs li.is-active a,.docstring>section>a.hero.docs-sourcelink .tabs li.is-active a{color:#4eb5de !important;opacity:1}.hero.is-primary .tabs.is-boxed a,.docstring>section>a.hero.docs-sourcelink .tabs.is-boxed a,.hero.is-primary .tabs.is-toggle a,.docstring>section>a.hero.docs-sourcelink .tabs.is-toggle a{color:#fff}.hero.is-primary .tabs.is-boxed a:hover,.docstring>section>a.hero.docs-sourcelink .tabs.is-boxed a:hover,.hero.is-primary .tabs.is-toggle a:hover,.docstring>section>a.hero.docs-sourcelink .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-primary .tabs.is-boxed li.is-active a,.docstring>section>a.hero.docs-sourcelink .tabs.is-boxed li.is-active a,.hero.is-primary .tabs.is-boxed li.is-active a:hover,.hero.is-primary .tabs.is-toggle li.is-active a,.docstring>section>a.hero.docs-sourcelink .tabs.is-toggle li.is-active a,.hero.is-primary .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#4eb5de}.hero.is-primary.is-bold,.docstring>section>a.hero.is-bold.docs-sourcelink{background-image:linear-gradient(141deg, #1bc7de 0%, #4eb5de 71%, #5fa9e7 100%)}@media screen and (max-width: 768px){.hero.is-primary.is-bold .navbar-menu,.docstring>section>a.hero.is-bold.docs-sourcelink .navbar-menu{background-image:linear-gradient(141deg, #1bc7de 0%, #4eb5de 71%, #5fa9e7 100%)}}.hero.is-link{background-color:#2e63b8;color:#fff}.hero.is-link a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-link strong{color:inherit}.hero.is-link .title{color:#fff}.hero.is-link .subtitle{color:rgba(255,255,255,0.9)}.hero.is-link .subtitle a:not(.button),.hero.is-link .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-link .navbar-menu{background-color:#2e63b8}}.hero.is-link .navbar-item,.hero.is-link .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-link a.navbar-item:hover,.hero.is-link a.navbar-item.is-active,.hero.is-link .navbar-link:hover,.hero.is-link .navbar-link.is-active{background-color:#2958a4;color:#fff}.hero.is-link .tabs a{color:#fff;opacity:0.9}.hero.is-link .tabs a:hover{opacity:1}.hero.is-link .tabs li.is-active a{color:#2e63b8 !important;opacity:1}.hero.is-link .tabs.is-boxed a,.hero.is-link .tabs.is-toggle a{color:#fff}.hero.is-link .tabs.is-boxed a:hover,.hero.is-link .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-link .tabs.is-boxed li.is-active a,.hero.is-link .tabs.is-boxed li.is-active a:hover,.hero.is-link .tabs.is-toggle li.is-active a,.hero.is-link .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#2e63b8}.hero.is-link.is-bold{background-image:linear-gradient(141deg, #1b6098 0%, #2e63b8 71%, #2d51d2 100%)}@media screen and (max-width: 768px){.hero.is-link.is-bold .navbar-menu{background-image:linear-gradient(141deg, #1b6098 0%, #2e63b8 71%, #2d51d2 100%)}}.hero.is-info{background-color:#209cee;color:#fff}.hero.is-info a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-info strong{color:inherit}.hero.is-info .title{color:#fff}.hero.is-info .subtitle{color:rgba(255,255,255,0.9)}.hero.is-info .subtitle a:not(.button),.hero.is-info .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-info .navbar-menu{background-color:#209cee}}.hero.is-info .navbar-item,.hero.is-info .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-info a.navbar-item:hover,.hero.is-info a.navbar-item.is-active,.hero.is-info .navbar-link:hover,.hero.is-info .navbar-link.is-active{background-color:#1190e3;color:#fff}.hero.is-info .tabs a{color:#fff;opacity:0.9}.hero.is-info .tabs a:hover{opacity:1}.hero.is-info .tabs li.is-active a{color:#209cee !important;opacity:1}.hero.is-info .tabs.is-boxed a,.hero.is-info .tabs.is-toggle a{color:#fff}.hero.is-info .tabs.is-boxed a:hover,.hero.is-info .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-info .tabs.is-boxed li.is-active a,.hero.is-info .tabs.is-boxed li.is-active a:hover,.hero.is-info .tabs.is-toggle li.is-active a,.hero.is-info .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#209cee}.hero.is-info.is-bold{background-image:linear-gradient(141deg, #05a6d6 0%, #209cee 71%, #3287f5 100%)}@media screen and (max-width: 768px){.hero.is-info.is-bold .navbar-menu{background-image:linear-gradient(141deg, #05a6d6 0%, #209cee 71%, #3287f5 100%)}}.hero.is-success{background-color:#22c35b;color:#fff}.hero.is-success a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-success strong{color:inherit}.hero.is-success .title{color:#fff}.hero.is-success .subtitle{color:rgba(255,255,255,0.9)}.hero.is-success .subtitle a:not(.button),.hero.is-success .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-success .navbar-menu{background-color:#22c35b}}.hero.is-success .navbar-item,.hero.is-success .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-success a.navbar-item:hover,.hero.is-success a.navbar-item.is-active,.hero.is-success .navbar-link:hover,.hero.is-success .navbar-link.is-active{background-color:#1ead51;color:#fff}.hero.is-success .tabs a{color:#fff;opacity:0.9}.hero.is-success .tabs a:hover{opacity:1}.hero.is-success .tabs li.is-active a{color:#22c35b !important;opacity:1}.hero.is-success .tabs.is-boxed a,.hero.is-success .tabs.is-toggle a{color:#fff}.hero.is-success .tabs.is-boxed a:hover,.hero.is-success .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-success .tabs.is-boxed li.is-active a,.hero.is-success .tabs.is-boxed li.is-active a:hover,.hero.is-success .tabs.is-toggle li.is-active a,.hero.is-success .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#22c35b}.hero.is-success.is-bold{background-image:linear-gradient(141deg, #12a02c 0%, #22c35b 71%, #1fdf83 100%)}@media screen and (max-width: 768px){.hero.is-success.is-bold .navbar-menu{background-image:linear-gradient(141deg, #12a02c 0%, #22c35b 71%, #1fdf83 100%)}}.hero.is-warning{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.hero.is-warning a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-warning strong{color:inherit}.hero.is-warning .title{color:rgba(0,0,0,0.7)}.hero.is-warning .subtitle{color:rgba(0,0,0,0.9)}.hero.is-warning .subtitle a:not(.button),.hero.is-warning .subtitle strong{color:rgba(0,0,0,0.7)}@media screen and (max-width: 1055px){.hero.is-warning .navbar-menu{background-color:#ffdd57}}.hero.is-warning .navbar-item,.hero.is-warning .navbar-link{color:rgba(0,0,0,0.7)}.hero.is-warning a.navbar-item:hover,.hero.is-warning a.navbar-item.is-active,.hero.is-warning .navbar-link:hover,.hero.is-warning .navbar-link.is-active{background-color:#ffd83e;color:rgba(0,0,0,0.7)}.hero.is-warning .tabs a{color:rgba(0,0,0,0.7);opacity:0.9}.hero.is-warning .tabs a:hover{opacity:1}.hero.is-warning .tabs li.is-active a{color:#ffdd57 !important;opacity:1}.hero.is-warning .tabs.is-boxed a,.hero.is-warning .tabs.is-toggle a{color:rgba(0,0,0,0.7)}.hero.is-warning .tabs.is-boxed a:hover,.hero.is-warning .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-warning .tabs.is-boxed li.is-active a,.hero.is-warning .tabs.is-boxed li.is-active a:hover,.hero.is-warning .tabs.is-toggle li.is-active a,.hero.is-warning .tabs.is-toggle li.is-active a:hover{background-color:rgba(0,0,0,0.7);border-color:rgba(0,0,0,0.7);color:#ffdd57}.hero.is-warning.is-bold{background-image:linear-gradient(141deg, #ffae24 0%, #ffdd57 71%, #fffa71 100%)}@media screen and (max-width: 768px){.hero.is-warning.is-bold .navbar-menu{background-image:linear-gradient(141deg, #ffae24 0%, #ffdd57 71%, #fffa71 100%)}}.hero.is-danger{background-color:#da0b00;color:#fff}.hero.is-danger a:not(.button):not(.dropdown-item):not(.tag):not(.pagination-link.is-current),.hero.is-danger strong{color:inherit}.hero.is-danger .title{color:#fff}.hero.is-danger .subtitle{color:rgba(255,255,255,0.9)}.hero.is-danger .subtitle a:not(.button),.hero.is-danger .subtitle strong{color:#fff}@media screen and (max-width: 1055px){.hero.is-danger .navbar-menu{background-color:#da0b00}}.hero.is-danger .navbar-item,.hero.is-danger .navbar-link{color:rgba(255,255,255,0.7)}.hero.is-danger a.navbar-item:hover,.hero.is-danger a.navbar-item.is-active,.hero.is-danger .navbar-link:hover,.hero.is-danger .navbar-link.is-active{background-color:#c10a00;color:#fff}.hero.is-danger .tabs a{color:#fff;opacity:0.9}.hero.is-danger .tabs a:hover{opacity:1}.hero.is-danger .tabs li.is-active a{color:#da0b00 !important;opacity:1}.hero.is-danger .tabs.is-boxed a,.hero.is-danger .tabs.is-toggle a{color:#fff}.hero.is-danger .tabs.is-boxed a:hover,.hero.is-danger .tabs.is-toggle a:hover{background-color:rgba(10,10,10,0.1)}.hero.is-danger .tabs.is-boxed li.is-active a,.hero.is-danger .tabs.is-boxed li.is-active a:hover,.hero.is-danger .tabs.is-toggle li.is-active a,.hero.is-danger .tabs.is-toggle li.is-active a:hover{background-color:#fff;border-color:#fff;color:#da0b00}.hero.is-danger.is-bold{background-image:linear-gradient(141deg, #a70013 0%, #da0b00 71%, #f43500 100%)}@media screen and (max-width: 768px){.hero.is-danger.is-bold .navbar-menu{background-image:linear-gradient(141deg, #a70013 0%, #da0b00 71%, #f43500 100%)}}.hero.is-small .hero-body,#documenter .docs-sidebar form.docs-search>input.hero .hero-body{padding:1.5rem}@media screen and (min-width: 769px),print{.hero.is-medium .hero-body{padding:9rem 4.5rem}}@media screen and (min-width: 769px),print{.hero.is-large .hero-body{padding:18rem 6rem}}.hero.is-halfheight .hero-body,.hero.is-fullheight .hero-body,.hero.is-fullheight-with-navbar .hero-body{align-items:center;display:flex}.hero.is-halfheight .hero-body>.container,.hero.is-fullheight .hero-body>.container,.hero.is-fullheight-with-navbar .hero-body>.container{flex-grow:1;flex-shrink:1}.hero.is-halfheight{min-height:50vh}.hero.is-fullheight{min-height:100vh}.hero-video{overflow:hidden}.hero-video video{left:50%;min-height:100%;min-width:100%;position:absolute;top:50%;transform:translate3d(-50%, -50%, 0)}.hero-video.is-transparent{opacity:0.3}@media screen and (max-width: 768px){.hero-video{display:none}}.hero-buttons{margin-top:1.5rem}@media screen and (max-width: 768px){.hero-buttons .button{display:flex}.hero-buttons .button:not(:last-child){margin-bottom:0.75rem}}@media screen and (min-width: 769px),print{.hero-buttons{display:flex;justify-content:center}.hero-buttons .button:not(:last-child){margin-right:1.5rem}}.hero-head,.hero-foot{flex-grow:0;flex-shrink:0}.hero-body{flex-grow:1;flex-shrink:0;padding:3rem 1.5rem}@media screen and (min-width: 769px),print{.hero-body{padding:3rem 3rem}}.section{padding:3rem 1.5rem}@media screen and (min-width: 1056px){.section{padding:3rem 3rem}.section.is-medium{padding:9rem 4.5rem}.section.is-large{padding:18rem 6rem}}.footer{background-color:#fafafa;padding:3rem 1.5rem 6rem}h1 .docs-heading-anchor,h1 .docs-heading-anchor:hover,h1 .docs-heading-anchor:visited,h2 .docs-heading-anchor,h2 .docs-heading-anchor:hover,h2 .docs-heading-anchor:visited,h3 .docs-heading-anchor,h3 .docs-heading-anchor:hover,h3 .docs-heading-anchor:visited,h4 .docs-heading-anchor,h4 .docs-heading-anchor:hover,h4 .docs-heading-anchor:visited,h5 .docs-heading-anchor,h5 .docs-heading-anchor:hover,h5 .docs-heading-anchor:visited,h6 .docs-heading-anchor,h6 .docs-heading-anchor:hover,h6 .docs-heading-anchor:visited{color:#222}h1 .docs-heading-anchor-permalink,h2 .docs-heading-anchor-permalink,h3 .docs-heading-anchor-permalink,h4 .docs-heading-anchor-permalink,h5 .docs-heading-anchor-permalink,h6 .docs-heading-anchor-permalink{visibility:hidden;vertical-align:middle;margin-left:0.5em;font-size:0.7rem}h1 .docs-heading-anchor-permalink::before,h2 .docs-heading-anchor-permalink::before,h3 .docs-heading-anchor-permalink::before,h4 .docs-heading-anchor-permalink::before,h5 .docs-heading-anchor-permalink::before,h6 .docs-heading-anchor-permalink::before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f0c1"}h1:hover .docs-heading-anchor-permalink,h2:hover .docs-heading-anchor-permalink,h3:hover .docs-heading-anchor-permalink,h4:hover .docs-heading-anchor-permalink,h5:hover .docs-heading-anchor-permalink,h6:hover .docs-heading-anchor-permalink{visibility:visible}.docs-dark-only{display:none !important}pre{position:relative;overflow:hidden}pre code,pre code.hljs{padding:0 .75rem !important;overflow:auto;display:block}pre code:first-of-type,pre code.hljs:first-of-type{padding-top:0.5rem !important}pre code:last-of-type,pre code.hljs:last-of-type{padding-bottom:0.5rem !important}pre .copy-button{opacity:0.2;transition:opacity 0.2s;position:absolute;right:0em;top:0em;padding:0.5em;width:2.5em;height:2.5em;background:transparent;border:none;font-family:"Font Awesome 6 Free";color:#222;cursor:pointer;text-align:center}pre .copy-button:focus,pre .copy-button:hover{opacity:1;background:rgba(34,34,34,0.1);color:#2e63b8}pre .copy-button.success{color:#259a12;opacity:1}pre .copy-button.error{color:#cb3c33;opacity:1}pre:hover .copy-button{opacity:1}.admonition{background-color:#b5b5b5;border-style:solid;border-width:1px;border-color:#363636;border-radius:4px;font-size:1rem}.admonition strong{color:currentColor}.admonition.is-small,#documenter .docs-sidebar form.docs-search>input.admonition{font-size:.75rem}.admonition.is-medium{font-size:1.25rem}.admonition.is-large{font-size:1.5rem}.admonition.is-default{background-color:#b5b5b5;border-color:#363636}.admonition.is-default>.admonition-header{background-color:#363636;color:#fff}.admonition.is-default>.admonition-body{color:#fff}.admonition.is-info{background-color:#def0fc;border-color:#209cee}.admonition.is-info>.admonition-header{background-color:#209cee;color:#fff}.admonition.is-info>.admonition-body{color:rgba(0,0,0,0.7)}.admonition.is-success{background-color:#bdf4d1;border-color:#22c35b}.admonition.is-success>.admonition-header{background-color:#22c35b;color:#fff}.admonition.is-success>.admonition-body{color:rgba(0,0,0,0.7)}.admonition.is-warning{background-color:#fff3c5;border-color:#ffdd57}.admonition.is-warning>.admonition-header{background-color:#ffdd57;color:rgba(0,0,0,0.7)}.admonition.is-warning>.admonition-body{color:rgba(0,0,0,0.7)}.admonition.is-danger{background-color:#ffaba7;border-color:#da0b00}.admonition.is-danger>.admonition-header{background-color:#da0b00;color:#fff}.admonition.is-danger>.admonition-body{color:rgba(0,0,0,0.7)}.admonition.is-compat{background-color:#bdeff5;border-color:#1db5c9}.admonition.is-compat>.admonition-header{background-color:#1db5c9;color:#fff}.admonition.is-compat>.admonition-body{color:rgba(0,0,0,0.7)}.admonition-header{color:#fff;background-color:#363636;align-items:center;font-weight:700;justify-content:space-between;line-height:1.25;padding:0.5rem .75rem;position:relative}.admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;margin-right:.75rem;content:"\f06a"}details.admonition.is-details>.admonition-header{list-style:none}details.admonition.is-details>.admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f055"}details.admonition.is-details[open]>.admonition-header:before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f056"}.admonition-body{color:#222;padding:0.5rem .75rem}.admonition-body pre{background-color:#f5f5f5}.admonition-body code{background-color:rgba(0,0,0,0.05)}.docstring{margin-bottom:1em;background-color:rgba(0,0,0,0);border:1px solid #dbdbdb;box-shadow:2px 2px 3px rgba(10,10,10,0.1);max-width:100%}.docstring>header{cursor:pointer;display:flex;flex-grow:1;align-items:stretch;padding:0.5rem .75rem;background-color:#f5f5f5;box-shadow:0 0.125em 0.25em rgba(10,10,10,0.1);box-shadow:none;border-bottom:1px solid #dbdbdb;overflow:auto}.docstring>header code{background-color:transparent}.docstring>header .docstring-article-toggle-button{min-width:1.1rem;padding:0.2rem 0.2rem 0.2rem 0}.docstring>header .docstring-binding{margin-right:0.3em}.docstring>header .docstring-category{margin-left:0.3em}.docstring>section{position:relative;padding:.75rem .75rem;border-bottom:1px solid #dbdbdb}.docstring>section:last-child{border-bottom:none}.docstring>section>a.docs-sourcelink{transition:opacity 0.3s;opacity:0;position:absolute;right:.375rem;bottom:.375rem}.docstring>section>a.docs-sourcelink:focus{opacity:1 !important}.docstring:hover>section>a.docs-sourcelink{opacity:0.2}.docstring:focus-within>section>a.docs-sourcelink{opacity:0.2}.docstring>section:hover a.docs-sourcelink{opacity:1}.documenter-example-output{background-color:#fff}.outdated-warning-overlay{position:fixed;top:0;left:0;right:0;box-shadow:0 0 10px rgba(0,0,0,0.3);z-index:999;background-color:#ffaba7;color:rgba(0,0,0,0.7);border-bottom:3px solid #da0b00;padding:10px 35px;text-align:center;font-size:15px}.outdated-warning-overlay .outdated-warning-closer{position:absolute;top:calc(50% - 10px);right:18px;cursor:pointer;width:12px}.outdated-warning-overlay a{color:#2e63b8}.outdated-warning-overlay a:hover{color:#363636}.content pre{border:1px solid #dbdbdb}.content code{font-weight:inherit}.content a code{color:#2e63b8}.content h1 code,.content h2 code,.content h3 code,.content h4 code,.content h5 code,.content h6 code{color:#222}.content table{display:block;width:initial;max-width:100%;overflow-x:auto}.content blockquote>ul:first-child,.content blockquote>ol:first-child,.content .admonition-body>ul:first-child,.content .admonition-body>ol:first-child{margin-top:0}pre,code{font-variant-ligatures:no-contextual}.breadcrumb a.is-disabled{cursor:default;pointer-events:none}.breadcrumb a.is-disabled,.breadcrumb a.is-disabled:hover{color:#222}.hljs{background:initial !important}.katex .katex-mathml{top:0;right:0}.katex-display,mjx-container,.MathJax_Display{margin:0.5em 0 !important}html{-moz-osx-font-smoothing:auto;-webkit-font-smoothing:auto}li.no-marker{list-style:none}#documenter .docs-main>article{overflow-wrap:break-word}#documenter .docs-main>article .math-container{overflow-x:auto;overflow-y:hidden}@media screen and (min-width: 1056px){#documenter .docs-main{max-width:52rem;margin-left:20rem;padding-right:1rem}}@media screen and (max-width: 1055px){#documenter .docs-main{width:100%}#documenter .docs-main>article{max-width:52rem;margin-left:auto;margin-right:auto;margin-bottom:1rem;padding:0 1rem}#documenter .docs-main>header,#documenter .docs-main>nav{max-width:100%;width:100%;margin:0}}#documenter .docs-main header.docs-navbar{background-color:#fff;border-bottom:1px solid #dbdbdb;z-index:2;min-height:4rem;margin-bottom:1rem;display:flex}#documenter .docs-main header.docs-navbar .breadcrumb{flex-grow:1;overflow-x:hidden}#documenter .docs-main header.docs-navbar .docs-sidebar-button{display:block;font-size:1.5rem;padding-bottom:0.1rem;margin-right:1rem}#documenter .docs-main header.docs-navbar .docs-right{display:flex;white-space:nowrap;gap:1rem;align-items:center}#documenter .docs-main header.docs-navbar .docs-right .docs-icon,#documenter .docs-main header.docs-navbar .docs-right .docs-label{display:inline-block}#documenter .docs-main header.docs-navbar .docs-right .docs-label{padding:0;margin-left:0.3em}@media screen and (max-width: 1055px){#documenter .docs-main header.docs-navbar .docs-right .docs-navbar-link{margin-left:0.4rem;margin-right:0.4rem}}#documenter .docs-main header.docs-navbar>*{margin:auto 0}@media screen and (max-width: 1055px){#documenter .docs-main header.docs-navbar{position:sticky;top:0;padding:0 1rem;transition-property:top, box-shadow;-webkit-transition-property:top, box-shadow;transition-duration:0.3s;-webkit-transition-duration:0.3s}#documenter .docs-main header.docs-navbar.headroom--not-top{box-shadow:.2rem 0rem .4rem #bbb;transition-duration:0.7s;-webkit-transition-duration:0.7s}#documenter .docs-main header.docs-navbar.headroom--unpinned.headroom--not-top.headroom--not-bottom{top:-4.5rem;transition-duration:0.7s;-webkit-transition-duration:0.7s}}#documenter .docs-main section.footnotes{border-top:1px solid #dbdbdb}#documenter .docs-main section.footnotes li .tag:first-child,#documenter .docs-main section.footnotes li .docstring>section>a.docs-sourcelink:first-child,#documenter .docs-main section.footnotes li .content kbd:first-child,.content #documenter .docs-main section.footnotes li kbd:first-child{margin-right:1em;margin-bottom:0.4em}#documenter .docs-main .docs-footer{display:flex;flex-wrap:wrap;margin-left:0;margin-right:0;border-top:1px solid #dbdbdb;padding-top:1rem;padding-bottom:1rem}@media screen and (max-width: 1055px){#documenter .docs-main .docs-footer{padding-left:1rem;padding-right:1rem}}#documenter .docs-main .docs-footer .docs-footer-nextpage,#documenter .docs-main .docs-footer .docs-footer-prevpage{flex-grow:1}#documenter .docs-main .docs-footer .docs-footer-nextpage{text-align:right}#documenter .docs-main .docs-footer .flexbox-break{flex-basis:100%;height:0}#documenter .docs-main .docs-footer .footer-message{font-size:0.8em;margin:0.5em auto 0 auto;text-align:center}#documenter .docs-sidebar{display:flex;flex-direction:column;color:#0a0a0a;background-color:#f5f5f5;border-right:1px solid #dbdbdb;padding:0;flex:0 0 18rem;z-index:5;font-size:1rem;position:fixed;left:-18rem;width:18rem;height:100%;transition:left 0.3s}#documenter .docs-sidebar.visible{left:0;box-shadow:.4rem 0rem .8rem #bbb}@media screen and (min-width: 1056px){#documenter .docs-sidebar.visible{box-shadow:none}}@media screen and (min-width: 1056px){#documenter .docs-sidebar{left:0;top:0}}#documenter .docs-sidebar .docs-logo{margin-top:1rem;padding:0 1rem}#documenter .docs-sidebar .docs-logo>img{max-height:6rem;margin:auto}#documenter .docs-sidebar .docs-package-name{flex-shrink:0;font-size:1.5rem;font-weight:700;text-align:center;white-space:nowrap;overflow:hidden;padding:0.5rem 0}#documenter .docs-sidebar .docs-package-name .docs-autofit{max-width:16.2rem}#documenter .docs-sidebar .docs-package-name a,#documenter .docs-sidebar .docs-package-name a:hover{color:#0a0a0a}#documenter .docs-sidebar .docs-version-selector{border-top:1px solid #dbdbdb;display:none;padding:0.5rem}#documenter .docs-sidebar .docs-version-selector.visible{display:flex}#documenter .docs-sidebar ul.docs-menu{flex-grow:1;user-select:none;border-top:1px solid #dbdbdb;padding-bottom:1.5rem}#documenter .docs-sidebar ul.docs-menu>li>.tocitem{font-weight:bold}#documenter .docs-sidebar ul.docs-menu>li li{font-size:.95rem;margin-left:1em;border-left:1px solid #dbdbdb}#documenter .docs-sidebar ul.docs-menu input.collapse-toggle{display:none}#documenter .docs-sidebar ul.docs-menu ul.collapsed{display:none}#documenter .docs-sidebar ul.docs-menu input:checked~ul.collapsed{display:block}#documenter .docs-sidebar ul.docs-menu label.tocitem{display:flex}#documenter .docs-sidebar ul.docs-menu label.tocitem .docs-label{flex-grow:2}#documenter .docs-sidebar ul.docs-menu label.tocitem .docs-chevron{display:inline-block;font-style:normal;font-variant:normal;text-rendering:auto;line-height:1;font-size:.75rem;margin-left:1rem;margin-top:auto;margin-bottom:auto}#documenter .docs-sidebar ul.docs-menu label.tocitem .docs-chevron::before{font-family:"Font Awesome 6 Free";font-weight:900;content:"\f054"}#documenter .docs-sidebar ul.docs-menu input:checked~label.tocitem .docs-chevron::before{content:"\f078"}#documenter .docs-sidebar ul.docs-menu .tocitem{display:block;padding:0.5rem 0.5rem}#documenter .docs-sidebar ul.docs-menu .tocitem,#documenter .docs-sidebar ul.docs-menu .tocitem:hover{color:#0a0a0a;background:#f5f5f5}#documenter .docs-sidebar ul.docs-menu a.tocitem:hover,#documenter .docs-sidebar ul.docs-menu label.tocitem:hover{color:#0a0a0a;background-color:#ebebeb}#documenter .docs-sidebar ul.docs-menu li.is-active{border-top:1px solid #dbdbdb;border-bottom:1px solid #dbdbdb;background-color:#fff}#documenter .docs-sidebar ul.docs-menu li.is-active .tocitem,#documenter .docs-sidebar ul.docs-menu li.is-active .tocitem:hover{background-color:#fff;color:#0a0a0a}#documenter .docs-sidebar ul.docs-menu li.is-active ul.internal .tocitem:hover{background-color:#ebebeb;color:#0a0a0a}#documenter .docs-sidebar ul.docs-menu>li.is-active:first-child{border-top:none}#documenter .docs-sidebar ul.docs-menu ul.internal{margin:0 0.5rem 0.5rem;border-top:1px solid #dbdbdb}#documenter .docs-sidebar ul.docs-menu ul.internal li{font-size:.85rem;border-left:none;margin-left:0;margin-top:0.5rem}#documenter .docs-sidebar ul.docs-menu ul.internal .tocitem{width:100%;padding:0}#documenter .docs-sidebar ul.docs-menu ul.internal .tocitem::before{content:"⚬";margin-right:0.4em}#documenter .docs-sidebar form.docs-search{margin:auto;margin-top:0.5rem;margin-bottom:0.5rem}#documenter .docs-sidebar form.docs-search>input{width:14.4rem}#documenter .docs-sidebar #documenter-search-query{color:#707070;width:14.4rem;box-shadow:inset 0 1px 2px rgba(10,10,10,0.1)}@media screen and (min-width: 1056px){#documenter .docs-sidebar ul.docs-menu{overflow-y:auto;-webkit-overflow-scroll:touch}#documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar{width:.3rem;background:none}#documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar-thumb{border-radius:5px 0px 0px 5px;background:#e0e0e0}#documenter .docs-sidebar ul.docs-menu::-webkit-scrollbar-thumb:hover{background:#ccc}}@media screen and (max-width: 1055px){#documenter .docs-sidebar{overflow-y:auto;-webkit-overflow-scroll:touch}#documenter .docs-sidebar::-webkit-scrollbar{width:.3rem;background:none}#documenter .docs-sidebar::-webkit-scrollbar-thumb{border-radius:5px 0px 0px 5px;background:#e0e0e0}#documenter .docs-sidebar::-webkit-scrollbar-thumb:hover{background:#ccc}}kbd.search-modal-key-hints{border-radius:0.25rem;border:1px solid rgba(0,0,0,0.6);box-shadow:0 2px 0 1px rgba(0,0,0,0.6);cursor:default;font-size:0.9rem;line-height:1.5;min-width:0.75rem;text-align:center;padding:0.1rem 0.3rem;position:relative;top:-1px}.search-min-width-50{min-width:50%}.search-min-height-100{min-height:100%}.search-modal-card-body{max-height:calc(100vh - 15rem)}.search-result-link{border-radius:0.7em;transition:all 300ms}.search-result-link:hover,.search-result-link:focus{background-color:rgba(0,128,128,0.1)}.search-result-link .property-search-result-badge,.search-result-link .search-filter{transition:all 300ms}.property-search-result-badge,.search-filter{padding:0.15em 0.5em;font-size:0.8em;font-style:italic;text-transform:none !important;line-height:1.5;color:#f5f5f5;background-color:rgba(51,65,85,0.501961);border-radius:0.6rem}.search-result-link:hover .property-search-result-badge,.search-result-link:hover .search-filter,.search-result-link:focus .property-search-result-badge,.search-result-link:focus .search-filter{color:#f1f5f9;background-color:#333}.search-filter{color:#333;background-color:#f5f5f5;transition:all 300ms}.search-filter:hover,.search-filter:focus{color:#333}.search-filter-selected{color:#f5f5f5;background-color:rgba(139,0,139,0.5)}.search-filter-selected:hover,.search-filter-selected:focus{color:#f5f5f5}.search-result-highlight{background-color:#ffdd57;color:black}.search-divider{border-bottom:1px solid #dbdbdb}.search-result-title{width:85%;color:#333}.search-result-code-title{font-size:0.875rem;font-family:"JuliaMono","SFMono-Regular","Menlo","Consolas","Liberation Mono","DejaVu Sans Mono",monospace}#search-modal .modal-card-body::-webkit-scrollbar,#search-modal .filter-tabs::-webkit-scrollbar{height:10px;width:10px;background-color:transparent}#search-modal .modal-card-body::-webkit-scrollbar-thumb,#search-modal .filter-tabs::-webkit-scrollbar-thumb{background-color:gray;border-radius:1rem}#search-modal .modal-card-body::-webkit-scrollbar-track,#search-modal .filter-tabs::-webkit-scrollbar-track{-webkit-box-shadow:inset 0 0 6px rgba(0,0,0,0.6);background-color:transparent}.w-100{width:100%}.gap-2{gap:0.5rem}.gap-4{gap:1rem}.gap-8{gap:2rem}.ansi span.sgr1{font-weight:bolder}.ansi span.sgr2{font-weight:lighter}.ansi span.sgr3{font-style:italic}.ansi span.sgr4{text-decoration:underline}.ansi span.sgr7{color:#fff;background-color:#222}.ansi span.sgr8{color:transparent}.ansi span.sgr8 span{color:transparent}.ansi span.sgr9{text-decoration:line-through}.ansi span.sgr30{color:#242424}.ansi span.sgr31{color:#a7201f}.ansi span.sgr32{color:#066f00}.ansi span.sgr33{color:#856b00}.ansi span.sgr34{color:#2149b0}.ansi span.sgr35{color:#7d4498}.ansi span.sgr36{color:#007989}.ansi span.sgr37{color:gray}.ansi span.sgr40{background-color:#242424}.ansi span.sgr41{background-color:#a7201f}.ansi span.sgr42{background-color:#066f00}.ansi span.sgr43{background-color:#856b00}.ansi span.sgr44{background-color:#2149b0}.ansi span.sgr45{background-color:#7d4498}.ansi span.sgr46{background-color:#007989}.ansi span.sgr47{background-color:gray}.ansi span.sgr90{color:#616161}.ansi span.sgr91{color:#cb3c33}.ansi span.sgr92{color:#0e8300}.ansi span.sgr93{color:#a98800}.ansi span.sgr94{color:#3c5dcd}.ansi span.sgr95{color:#9256af}.ansi span.sgr96{color:#008fa3}.ansi span.sgr97{color:#f5f5f5}.ansi span.sgr100{background-color:#616161}.ansi span.sgr101{background-color:#cb3c33}.ansi span.sgr102{background-color:#0e8300}.ansi span.sgr103{background-color:#a98800}.ansi span.sgr104{background-color:#3c5dcd}.ansi span.sgr105{background-color:#9256af}.ansi span.sgr106{background-color:#008fa3}.ansi span.sgr107{background-color:#f5f5f5}code.language-julia-repl>span.hljs-meta{color:#066f00;font-weight:bolder}/*! + Theme: Default + Description: Original highlight.js style + Author: (c) Ivan Sagalaev + Maintainer: @highlightjs/core-team + Website: https://highlightjs.org/ + License: see project LICENSE + Touched: 2021 +*/pre code.hljs{display:block;overflow-x:auto;padding:1em}code.hljs{padding:3px 5px}.hljs{background:#F3F3F3;color:#444}.hljs-comment{color:#697070}.hljs-tag,.hljs-punctuation{color:#444a}.hljs-tag .hljs-name,.hljs-tag .hljs-attr{color:#444}.hljs-keyword,.hljs-attribute,.hljs-selector-tag,.hljs-meta .hljs-keyword,.hljs-doctag,.hljs-name{font-weight:bold}.hljs-type,.hljs-string,.hljs-number,.hljs-selector-id,.hljs-selector-class,.hljs-quote,.hljs-template-tag,.hljs-deletion{color:#880000}.hljs-title,.hljs-section{color:#880000;font-weight:bold}.hljs-regexp,.hljs-symbol,.hljs-variable,.hljs-template-variable,.hljs-link,.hljs-selector-attr,.hljs-operator,.hljs-selector-pseudo{color:#ab5656}.hljs-literal{color:#695}.hljs-built_in,.hljs-bullet,.hljs-code,.hljs-addition{color:#397300}.hljs-meta{color:#1f7199}.hljs-meta .hljs-string{color:#38a}.hljs-emphasis{font-style:italic}.hljs-strong{font-weight:bold}.gap-4{gap:1rem} diff --git a/previews/PR663/assets/themeswap.js b/previews/PR663/assets/themeswap.js new file mode 100644 index 000000000..9f5eebe6a --- /dev/null +++ b/previews/PR663/assets/themeswap.js @@ -0,0 +1,84 @@ +// Small function to quickly swap out themes. Gets put into the tag.. +function set_theme_from_local_storage() { + // Initialize the theme to null, which means default + var theme = null; + // If the browser supports the localstorage and is not disabled then try to get the + // documenter theme + if (window.localStorage != null) { + // Get the user-picked theme from localStorage. May be `null`, which means the default + // theme. + theme = window.localStorage.getItem("documenter-theme"); + } + // Check if the users preference is for dark color scheme + var darkPreference = + window.matchMedia("(prefers-color-scheme: dark)").matches === true; + // Initialize a few variables for the loop: + // + // - active: will contain the index of the theme that should be active. Note that there + // is no guarantee that localStorage contains sane values. If `active` stays `null` + // we either could not find the theme or it is the default (primary) theme anyway. + // Either way, we then need to stick to the primary theme. + // + // - disabled: style sheets that should be disabled (i.e. all the theme style sheets + // that are not the currently active theme) + var active = null; + var disabled = []; + var primaryLightTheme = null; + var primaryDarkTheme = null; + for (var i = 0; i < document.styleSheets.length; i++) { + var ss = document.styleSheets[i]; + // The tag of each style sheet is expected to have a data-theme-name attribute + // which must contain the name of the theme. The names in localStorage much match this. + var themename = ss.ownerNode.getAttribute("data-theme-name"); + // attribute not set => non-theme stylesheet => ignore + if (themename === null) continue; + // To distinguish the default (primary) theme, it needs to have the data-theme-primary + // attribute set. + if (ss.ownerNode.getAttribute("data-theme-primary") !== null) { + primaryLightTheme = themename; + } + // Check if the theme is primary dark theme so that we could store its name in darkTheme + if (ss.ownerNode.getAttribute("data-theme-primary-dark") !== null) { + primaryDarkTheme = themename; + } + // If we find a matching theme (and it's not the default), we'll set active to non-null + if (themename === theme) active = i; + // Store the style sheets of inactive themes so that we could disable them + if (themename !== theme) disabled.push(ss); + } + var activeTheme = null; + if (active !== null) { + // If we did find an active theme, we'll (1) add the theme--$(theme) class to + document.getElementsByTagName("html")[0].className = "theme--" + theme; + activeTheme = theme; + } else { + // If we did _not_ find an active theme, then we need to fall back to the primary theme + // which can either be dark or light, depending on the user's OS preference. + var activeTheme = darkPreference ? primaryDarkTheme : primaryLightTheme; + // In case it somehow happens that the relevant primary theme was not found in the + // preceding loop, we abort without doing anything. + if (activeTheme === null) { + console.error("Unable to determine primary theme."); + return; + } + // When switching to the primary light theme, then we must not have a class name + // for the tag. That's only for non-primary or the primary dark theme. + if (darkPreference) { + document.getElementsByTagName("html")[0].className = + "theme--" + activeTheme; + } else { + document.getElementsByTagName("html")[0].className = ""; + } + } + for (var i = 0; i < document.styleSheets.length; i++) { + var ss = document.styleSheets[i]; + // The tag of each style sheet is expected to have a data-theme-name attribute + // which must contain the name of the theme. The names in localStorage much match this. + var themename = ss.ownerNode.getAttribute("data-theme-name"); + // attribute not set => non-theme stylesheet => ignore + if (themename === null) continue; + // we'll disable all the stylesheets, except for the active one + ss.disabled = !(themename == activeTheme); + } +} +set_theme_from_local_storage(); diff --git a/previews/PR663/assets/warner.js b/previews/PR663/assets/warner.js new file mode 100644 index 000000000..3f6f5d008 --- /dev/null +++ b/previews/PR663/assets/warner.js @@ -0,0 +1,52 @@ +function maybeAddWarning() { + // DOCUMENTER_NEWEST is defined in versions.js, DOCUMENTER_CURRENT_VERSION and DOCUMENTER_STABLE + // in siteinfo.js. + // If either of these are undefined something went horribly wrong, so we abort. + if ( + window.DOCUMENTER_NEWEST === undefined || + window.DOCUMENTER_CURRENT_VERSION === undefined || + window.DOCUMENTER_STABLE === undefined + ) { + return; + } + + // Current version is not a version number, so we can't tell if it's the newest version. Abort. + if (!/v(\d+\.)*\d+/.test(window.DOCUMENTER_CURRENT_VERSION)) { + return; + } + + // Current version is newest version, so no need to add a warning. + if (window.DOCUMENTER_NEWEST === window.DOCUMENTER_CURRENT_VERSION) { + return; + } + + // Add a noindex meta tag (unless one exists) so that search engines don't index this version of the docs. + if (document.body.querySelector('meta[name="robots"]') === null) { + const meta = document.createElement("meta"); + meta.name = "robots"; + meta.content = "noindex"; + + document.getElementsByTagName("head")[0].appendChild(meta); + } + + const div = document.createElement("div"); + div.classList.add("outdated-warning-overlay"); + const closer = document.createElement("button"); + closer.classList.add("outdated-warning-closer", "delete"); + closer.addEventListener("click", function () { + document.body.removeChild(div); + }); + const href = window.documenterBaseURL + "/../" + window.DOCUMENTER_STABLE; + div.innerHTML = + 'This documentation is not for the latest stable release, but for either the development version or an older release.
Click here to go to the documentation for the latest stable release.'; + div.appendChild(closer); + document.body.appendChild(div); +} + +if (document.readyState === "loading") { + document.addEventListener("DOMContentLoaded", maybeAddWarning); +} else { + maybeAddWarning(); +} diff --git a/previews/PR663/design/changing_the_primal.html b/previews/PR663/design/changing_the_primal.html new file mode 100644 index 000000000..18e8434c5 --- /dev/null +++ b/previews/PR663/design/changing_the_primal.html @@ -0,0 +1,104 @@ + +Changing the Primal · ChainRules
GitHub

Design Notes: Why can you change the primal computation?

These design notes are to help you understand ChainRules.jl's rrule function. It explains why we have a rrule function that returns both the primal result (i.e. the output for the forward pass) and the pullback as a closure. It might be surprising to some AD authors, who might expect just a function that performs the pullback, that the rrule function computes the primal result as well as the pullback. In particularly, rrule allows you to change how the primal result is computed. We will illustrate in this document why being able to change the computation of the primal is crucial for efficient AD.

What about `frule`?

Discussion here is focused on on reverse mode and rrule. Similar concerns do apply to forward mode and frule. In forward mode these concerns lead to the fusing of the pushforward into frule. All the examples given here also apply in forward mode. In fact in forward mode there are even more opportunities to take advantage of sharing work between the primal and derivative computations. A particularly notable example is in efficiently calculating the pushforward of solving a differential equation via expanding the system of equations to also include the derivatives before solving it.

The Journey to rrule

Let's imagine a different system for rules, one that doesn't let you define the computation of the primal. This system is what a lot of AD systems have. It is what Nabla.jl had originally.[1] We will have a primal (i.e. forward) pass that directly executes the primal function and just records the primal function, its inputs and its output onto the tape.[2]. Then during the gradient (i.e. reverse) pass it has a function which receives those records from the tape along with the sensitivity of the output, and gives back the sensitivity of the input. We will call this function pullback_at, as it pulls back the sensitivity at a given primal point. To make this concrete:

y = f(x)  # primal program
+x̄ = pullback_at(f, x, y, ȳ)

Let's illustrate this with examples for sin and for the logistic sigmoid.

Example for `sin`
y = sin(x)
+pullback_at(::typeof(sin), x, y, ȳ) = ȳ * cos(x)

pullback_at uses the primal input x, and the sensitivity being pulled back ȳ.

Example for the logistic sigmoid
σ(x) = 1/(1 + exp(-x))  # = exp(x) / (1 + exp(x))
+y = σ(x)
+pullback_at(::typeof(σ), x, y, ȳ) = ȳ * y * σ(-x)  # = ȳ * σ(x) * σ(-x)

Notice that in pullback_at we are not only using input x but also using the primal output y . This is a nice bit of symmetry that shows up around exp.

Now let's consider why we implement rrules in the first place. One key reason is to insert domain knowledge so as to compute the derivative more efficiently than AD would just by breaking everything down into +, *, etc.[3] What insights do we have about sin and cos? What about using sincos?

Example for `sin`
julia> using BenchmarkTools
+
+julia> @btime sin(x) setup=(x=rand());
+  3.838 ns (0 allocations: 0 bytes)
+
+julia> @btime cos(x) setup=(x=rand());
+  4.795 ns (0 allocations: 0 bytes)
+
+julia> 3.838 + 4.795
+8.633

vs computing both together:

julia> @btime sincos(x) setup=(x=rand());
+  6.028 ns (0 allocations: 0 bytes)

What about the logistic sigmoid? We note that the two values we need are σ(x) and σ(-x) If we write these as: $\sigma(x) = \frac{e^x}{1+e^x}$ and $\sigma(-x) = \frac{1}{1+e^x}$ then we see they have the common term $e^x$. exp(x) is a much more expensive operation than + and /. So we can save time, if we can reuse that exp(x).

Example for the logistic sigmoid

If we have to computing separately:

julia> @btime 1/(1+exp(x)) setup=(x=rand());
+  5.622 ns (0 allocations: 0 bytes)
+
+julia> @btime 1/(1+exp(-x)) setup=(x=rand());
+  6.036 ns (0 allocations: 0 bytes)
+
+julia> 5.622 + 6.036
+11.658

vs reusing exp(x):

julia> @btime exp(x) setup=(x=rand());
+  5.367 ns (0 allocations: 0 bytes)
+
+julia> @btime ex/(1+ex) setup=(ex=exp(rand()));
+  1.255 ns (0 allocations: 0 bytes)
+
+julia> @btime 1/(1+ex) setup=(ex=exp(rand()));
+  1.256 ns (0 allocations: 0 bytes)
+
+julia> 5.367 + 1.255 + 1.256
+7.878

So we are talking about a 30-40% speed-up from these optimizations.[4]

It is faster to compute sin and cos at the same time via sincos than it is to compute them one after the other. And it is faster to reuse the exp(x) in computing σ(x) and σ(-x). How can we incorporate this insight into our system? We know we can compute both of these in the primal — because they only depend on x and not on ȳ — but there is nowhere to put them that is accessible both to the primal pass and the gradient pass code.

What if we introduced some variable called intermediates that is also recorded onto the tape during the primal pass? We would need to be able to modify the primal pass to do this, so that we can actually put the data into the intermediates. So we will introduce a function: augmented_primal, that will return the primal output plus the intermediates that we want to reuse in the gradient pass. Then we will make our AD system replace calls to the primal with calls to the augmented_primal of the primal function and take care of all the bookkeeping. So that would look like:

y = f(x)  # primal program
+y, intermediates = augmented_primal(f, x)
+x̄ = pullback_at(f, x, y, ȳ, intermediates)
Example for `sin`
function augmented_primal(::typeof(sin), x)
+  y, cx = sincos(x)
+  return y, (; cx=cx)  # use a NamedTuple for the intermediates
+end
+
+pullback_at(::typeof(sin), x, y, ȳ, intermediates) = ȳ * intermediates.cx
Example for the logistic sigmoid
function augmented_primal(::typeof(σ), x)
+  ex = exp(x)
+  y = ex / (1 + ex)
+  return y, (; ex=ex)  # use a NamedTuple for the intermediates
+end
+
+pullback_at(::typeof(σ), x, y, ȳ, intermediates) = ȳ * y / (1 + intermediates.ex)

Cool! That lets us do what we wanted. We net decreased the time it takes to run the primal and gradient passes. We have now demonstrated the title question of why we want to be able to modify the primal pass. We will go into that more later and have some more usage examples, but first let's continue to see how we go from augmented_primal and pullback_at to rrule.

One thing we notice when looking at pullback_at is it really is starting to have a lot of arguments. It had a fair few already, and now we are adding intermediates as well, making it even more unwieldy. Not to mention these are fairly simple example, the sin and σ functions have 1 input and no keyword arguments. Furthermore, we often don't even use all of the arguments to pullback_at. The new code for pulling back sin — which uses sincos and intermediates — no longer needs x, and it never needed y (though sigmoid σ does). And storing all these things on the tape — inputs, outputs, sensitivities, intermediates — is using up extra memory. What if we generalized the idea of the intermediate named tuple, and had augmented_primal return a struct that just held anything we might want put on the tape.

struct PullbackMemory{P, S}
+  primal_function::P
+  state::S
+end
+# convenience constructor:
+PullbackMemory(primal_function; state...) = PullbackMemory(primal_function, state)
+# convenience accessor so that `m.x` is same as `m.state.x`
+Base.getproperty(m::PullbackMemory, propname) = getproperty(getfield(m, :state), propname)

So changing our API we have:

y = f(x)  # primal program
+y, pb = augmented_primal(f, x)
+x̄ = pullback_at(pb, ȳ)

which is much cleaner.

Example for `sin`
function augmented_primal(::typeof(sin), x)
+  y, cx = sincos(x)
+  return y, PullbackMemory(sin; cx=cx)
+end
+
+pullback_at(pb::PullbackMemory{typeof(sin)}, ȳ) = ȳ * pb.cx
Example for the logistic sigmoid
function augmented_primal(::typeof(σ), x)
+  ex = exp(x)
+  y = ex / (1 + ex)
+  return y, PullbackMemory(σ; y=y, ex=ex)
+end
+
+pullback_at(pb::PullbackMemory{typeof(σ)}, ȳ) = ȳ * pb.y / (1 + pb.ex)

That now looks much simpler; pullback_at only ever has 2 arguments.

One way we could make it nicer to use is by making PullbackMemory a callable object. Conceptually, for a particular evaluation of an operation, the PullbackMemory is fixed. It is fully determined by the end of the primal pass. The during the gradient (reverse) pass the PullbackMemory is used to successively compute the ȳ argument. So it makes sense to make PullbackMemory a callable object that acts on the sensitivity. We can do that via call overloading:

y = f(x)  # primal program
+y, pb = augmented_primal(f, x)
+x̄ = pb(ȳ)
Example for `sin`
function augmented_primal(::typeof(sin), x)
+  y, cx = sincos(x)
+  return y, PullbackMemory(sin; cx=cx)
+end
+(pb::PullbackMemory{typeof(sin)})(ȳ) = ȳ * pb.cx
Example for the logistic sigmoid
function augmented_primal(::typeof(σ), x)
+  ex = exp(x)
+  y = ex / (1 + ex)
+  return y, PullbackMemory(σ; y=y, ex=ex)
+end
+
+(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y / (1 + pb.ex)

Let's recap what we have done here. We now have an object pb that acts on the cotangent of the output of the primal ȳ to give us the cotangent of the input of the primal function x̄. pb is not just the memory of state required for the pullback, it is the pullback.

We have one final thing to do, which is to think about how we make the code easy to modify. Let's go back and think about the changes we would have make to go from our original way of writing that only used the inputs/outputs, to one that used the intermediate state.

Example for `sin`

To rewrite that original formulation in the new pullback form we have:

function augmented_primal(::typeof(sin), x)
+  y = sin(x)
+  return y, PullbackMemory(sin; x=x)
+end
+(pb::PullbackMemory)(ȳ) = ȳ * cos(pb.x)

To go from that to:

function augmented_primal(::typeof(sin), x)
+  y, cx = sincos(x)
+  return y, PullbackMemory(sin; cx=cx)
+end
+(pb::PullbackMemory)(ȳ) = ȳ * pb.cx
Example for the logistic sigmoid
function augmented_primal(::typeof(σ), x)
+  y = σ(x)
+  return y, PullbackMemory(σ; y=y, x=x)
+end
+(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y * σ(-pb.x)

to get to:

function augmented_primal(::typeof(σ), x)
+  ex = exp(x)
+  y = ex/(1 + ex)
+  return y, PullbackMemory(σ; y=y, ex=ex)
+end
+(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y/(1 + pb.ex)

We should think about how we might want to make future changes to this code.[6]

We need to make a series of changes:

  • update what work is done in the primal, to compute the intermediate values.
  • update what is stored in the PullbackMemory.
  • update the function that applies the pullback so it uses the new thing that was stored.

It's important these parts all stay in sync. It's not too bad for this simple example with just one or two things to remember. For more complicated multi-argument functions, which we will show below, you often end up needing to remember half a dozen things, like sizes and indices relating to each input/output, so it gets a little more fiddly to make sure you remember all the things you need to and give them the same name in both places. Is there a way we can automatically just have all the things we use remembered for us? Surprisingly for such a specific request, there actually is: a closure.

A closure in Julia is a callable structure that automatically contains a field for every object from its parent scope that is used in its body. There are incredible ways to abuse this; but here we can use closures exactly as they are intended. Replacing PullbackMemory with a closure that works the same way lets us avoid having to manually control what is remembered and lets us avoid separately writing the call overload.

Example for `sin`
function augmented_primal(::typeof(sin), x)
+  y, cx = sincos(x)
+  pb = ȳ -> cx * ȳ  # pullback closure. closes over `cx`
+  return y, pb
+end
Example for the logistic sigmoid
function augmented_primal(::typeof(σ), x)
+  ex = exp(x)
+  y = ex / (1 + ex)
+  pb = ȳ -> ȳ * y / (1 + ex)  # pullback closure. closes over `y` and `ex`
+  return y, pb
+end

This is pretty clean now.

Our augmented_primal is now within spitting distance of rrule. All that is left is a rename and some extra conventions around multiple outputs and gradients with respect to callable objects.

This has been a journey into how we get to rrule as it is defined in ChainRulesCore. We started with an unaugmented primal function and a pullback_at function that only saw the inputs and outputs of the primal. We realized a key limitation of this was that we couldn't share computational work between the primal and gradient passes. To solve this we introduced the notation of some intermediate that is shared from the primal to the pullback. We successively improved that idea, first by making it a type that held everything that is needed for the pullback: the PullbackMemory, which we then made callable, so it was itself the pullback. Finally, we replaced that separate callable structure with a closure, which kept everything in one place and made it more convenient.

More Shared Work Examples

sin and the logistic sigmoid are nice, simple examples of when it is useful to share work between the primal and the pullback. There are many others though. It is actually surprising that in so many cases it is reasonable to write the rules where the only shared information between the primal and the pullback is the primal inputs (like our original sin), or primal outputs (like our original logistic sigmoid). Under our formulation above, those primal inputs/outputs are shared information just like any other. Beyond this, there are a number of other decent applications.

getindex

In Julia (and many other numerical languages) indexing can take many more arguments than simply a couple of integers, such as boolean masking arrays (logical indexing), ranges for slices, etc. Converting the arguments to plain integers, arrays of integers, and ranges with Base.to_indices is the first thing that getindex does. It then re-calls getindex with these simpler types to get the result.

The result of pulling back the getindex operation is always an array that is all zeros, except for the elements that are selected, which are set to the appropriate sensitivities being pulled back. To identify which actual positions in the array are being gotten/set is common work to both primal and gradient computations. We really don't want to deal with fancy indexing types during the pullback, because there are weird edge cases like indexing in such a way that the same element is output twice (and thus we have 2 sensitivities we need to add to it). We can pull the to_indices out of the primal computation and remember the plain indexes used, then can reuse them to set gradients during the pullback.

See the code for this in ChainRules.jl

exp(::Matrix)

Matrix Functions are generalizations of scalar functions to operate on matrices. Note that this is distinct from simply element-wise application of the function to the matrix's elements. The Matrix Exponential exp(::Matrix) is a particularly important matrix function.

Al-Mohy and Higham (2009)[7], published a method for computing the pullback of exp(::Matrix). It is pretty complex and very cool. To quote its abstract (emphasis mine):

The algorithm is derived from the scaling and squaring method by differentiating the Padé approximants and the squaring recurrence, re-using quantities computed during the evaluation of the Padé approximant, and intertwining the recurrences in the squaring phase.

Julia does in fact use a Padé approximation to compute exp(::Matrix). So we can extract the code for that into our augmented primal, and add remembering the intermediate quantities that are to be used. See the code for this in ChainRules.jl

An interesting scenario here that may be of concern to some: if Julia changes the algorithm it uses to compute exp(::Matrix), then during an AD primal pass, it will continue to use the old Padé approximation based algorithm. This may actually happen, as there are many other algorithms that can compute the matrix exponential. Further, perhaps there might be an improvement to the exact coefficient or cut-offs used by Julia's current Padé approximation. If Julia made this change it would not be considered breaking. Exact floating point numerical values are not generally considered part of the SemVer-bound API. Rather only the general accuracy of the computed value relative to the true mathematical value (e.g. for common scalar operations Julia promises 1 ULP).

This change will result in the output of the AD primal pass not being exactly equal to what would be seen from just running the primal code. It will still be accurate because the current implementation is accurate, but it will be different. It is our argument that in general this should be considered acceptable, as long as the AD primal pass is in general about as accurate as the unaugmented primal. E.g. it might overshoot for some values the unaugmented primal undershoots for.

eigvals

eigvals is a real case where the algorithm for the augmented primal and the original primal is already different today. To compute the pullback of eigvals you need to know not only the eigenvalues but also the eigenvectors. The eigen function computes both, so that is used in the augmented primal. See the code for this in ChainRules.jl. If we could not compute and remember the eigenvectors in the primal pass, we would have to call eigen in the gradient pass anyway and fully recompute eigenvectors and eigenvalues, more than doubling the total work.

However, if you trace this down, it actually uses a different algorithm.

eigvals basically wraps LAPACK.syevr!('N', ...), which goes through DSYEVR and eventually calls DSTERF, which uses "Pal-Walker-Kahan variant of the QL or QR algorithm." to compute eigenvalues

In contrast, eigen wraps LAPACK.syevr!('V',...) which also goes through DSYEVR but eventually calls DSTEMR, which calculates eigenvalues "either by bisection or the dqds algorithm.".

Both of these are very good algorithms. LAPACK has had decades of work by experts and is one of the most trusted libraries for linear algebra. But they are different algorithms that give different results. The differences in practice are around $10^{-15}$, which while very small on absolute terms are as far as Float64 is concerned a very real difference.

Matrix Division

Roughly speaking: Y=A\B is the function that finds the least-square solution to YA ≈ B. When solving such a system, the efficient way to do so is to factorize A into an appropriate factorized form such as Cholesky or QR, then perform the \ operation on the factorized form. The pullback of A\B with respect to B is Ȳ -> A' \ Ȳ. It should be noted that this involves computing the factorization of A' (the adjoint of A).[8] In this computation the factorization of the original A can reused. Doing so can give a 4x speed-up.

We don't have this in ChainRules.jl yet, because Julia is missing some definitions of adjoint of factorizations (JuliaLang/julia#38293).[8] We have been promised them for Julia v1.7 though. You can see what the code would look like in PR #302.

Conclusion

This document has explained why rrule is the way it is. In particular it has highlighted why the primal computation is able to be changed from simply calling the function. Further, it has explained why rrule returns a closure for the pullback, rather than it being a separate function. It has highlighted several places in ChainRules.jl where this has allowed us to significantly improve performance. Being able to change the primal computation is practically essential for a high performance AD system.

  • 1I am not just picking on Nabla randomly. Many of the core developers of ChainRules worked on Nabla prior. It's a good AD, but ChainRules incorporates lessons learned from working on Nabla.
  • 2which may be an explicit tape or an implicit tape that is actually incorporated into generated code (à la Zygote)
  • 3Another key reason is if the operation is a primitive that is not defined in terms of more basic operations. In many languages this is the case for sin; where the actual implementation is in some separate libm.so. But actually sin in Julia is defined in terms of a polynomial. It's fairly vanilla Julia code. It shouldn't be too hard for an AD that only knows about basic operations like + and * to AD through it. In any case, that is another discussion for another day.
  • 4Sure, this is small fries and depending on Julia version might just get solved by the optimizer[5], but go with it for the sake of example.
  • 5To be precise, this is very likely to be solved by the optimizer inlining both and then performing common subexpression elimination, with the result that it generates the code for sincos just from having sin and cos inside the same function. However, this actually doesn't apply in the case of AD, as it is not possible to inline code called in the gradient pass into the primal pass. Those are separate functions called at very different times. This is something opaque closures should help solve.
  • 6One change we might consider is to have logistic sigmoid to only remember one thing. Rather than remembering y and ex to use in the pullback, we could compute y / (1 + ex) during the augmented primal, and just remember that.
  • 7Al-Mohy, Awad H. and Higham, Nicholas J. (2009) Computing the Fréchet Derivative of the Matrix Exponential, with an application to Condition Number Estimation. SIAM Journal On Matrix Analysis and Applications., 30 (4). pp. 1639-1657. ISSN 1095-7162
  • 8To be clear here we mean adjoint as in the conjugate transpose of a matrix, rather than in the sense of reverse mode AD.
diff --git a/previews/PR663/design/many_tangents.html b/previews/PR663/design/many_tangents.html new file mode 100644 index 000000000..ff0d99113 --- /dev/null +++ b/previews/PR663/design/many_tangents.html @@ -0,0 +1,33 @@ + +Many Tangent Types · ChainRules
GitHub

Design Notes: The many-to-many relationship between tangent types and primal types

ChainRules has a system where one primal type (the type having its derivative taken) can have multiple possible tangent types (the type of the derivative); and where one tangent type can correspond to multiple primal types. This is in-contrast to the Swift AD efforts, which has one tangent type per primal type (Swift uses the term associated tangent type).

tangent and associated tangent type

The use of “associated tangent type” in AD is not technically correct, as they live in the cotangent plane instead of the tangent plane. However it is often reasonable for AD to treat the cotangent plane and tangent plane as the same thing, and this was an intentional choice by the Swift team. In ChainRules we use the term “tangent type” to refer to both tangents and cotangents.

One thing to understand about tangents is that they have to form a vector space (or something very like them). They need to support addition to each other, they need a zero which doesn't change what it is added to, and they need to support scalar multiplication (this isn't really required, but it is handy for things like gradient descent). Beyond being a vector space, tangents need to be able to be added to a primal value to get back another primal value. Or roughly equivalently a tangent is a difference between two primal values.

One thing to note in this example is that the primal does not have to be a vector. As an example, consider DateTime. A DateTime is not a vector space: there is no origin point, and DateTimes cannot be added to each other. The corresponding tangent type is any subtype of Period, such as Millisecond, Hour, Day etc.

Natural tangent

For a given primal type, we say a natural tangent type is one which people would intuitively think of as representing the difference between two primal values. It tends to already exist outside of the context of AD. So Millisecond, Hour, Day etc. are examples of natural tangents for the DateTime primal.

Note here that we already have a one primal type to many tangent types relationship. We have Millisecond and Hour and Day all being valid tangent types for DateTime. In this case we could convert them all to a single tangent type, such as Nanoseconds, but that is not always a reasonable decision: we may run in to overflow, or lots of allocations if we need to use a BigInt to represent the number of Nanosecond since the start of the universe. For types with more complex semantics, such as array types, these considerations are much more important.

Natural tangent types are the types people tend to think in, and thus the type they tend to write custom sensitivity rules in. An important special case of natural tangents is when the primal type is a vector space (e.g. Real,AbstractMatrix) in which case it is common for the natural tangent type to be the same as the primal type. One exception to this is getindex. The ideal choice of tangent type for getindex on a dense array would be some type of sparse array, due to the fact the derivative will have only one non-zero element. This actually further brings us to a weirdness of tangent types not actually being closed under addition, as it would be ideal for the sparse array to become a dense array if summed over all elements.

Structural tangent types

AD cannot automatically determine the natural tangent types for a primal. For some types we may be able to declare manually their natural tangent type. Other types will not have natural tangent types at all - e.g. NamedTuple, Tuple, WebServer, Flux.Dense - so we are destined to make some up. So beyond natural tangent types, we also have structural tangent types. ChainRules uses Tangent{P, <:NamedTuple} to represent a structural tangent type corresponding to primal type P. Zygote v0.4 uses NamedTuple.

Structural tangents are derived from the structure of the input. Either automatically, as part of the AD, or manually, as part of a custom rule.

Consider the structure of DateTime:

julia> dump(now())
+DateTime
+  instant: UTInstant{Millisecond}
+    periods: Millisecond
+      value: Int64 63719890305605

The corresponding structural tangent is:

Tangent{DateTime}(
+    instant::Tangent{UTInstant{Millisecond}}(
+        periods::Tangent{Millisecond}(
+            value::Int64
+        )
+    )
+)
One must be allowed to take derivatives of integer arguments

This brings up another contrast to Swift. In Swift Int is considered non-differentiable, which is quite reasonable; it doesn’t have a very good definition of the limit of a small step (as that would be some floating/fixed point type). Int is intrinsically discrete. It is commonly used for indexing, and if one takes a gradient step, say turning x[2] into x[2.1] then that is an error. However, disallowing Int to be used as a tangent means we cannot handle cases like DateTime having an inner field of milliseconds counted as an integer from the unix epoch or other cases where an integer is used as a convenience for computational efficiency. In the case where a custom sensitivity rule claims that there is a non-zero derivative for an Int argument that is being used for indexing, that code is simply wrong. We can’t handle incorrect code and trying to is a path toward madness. Julia, unlike Swift, is not well suited to handling rules about what you can and can’t do with particular types.

So the structural tangent is another type of tangent. We must support both natural and structural tangents because AD can only create structural tangents (unless using custom sensitivity rules) and all custom sensitivities are only written in terms of natural tangents, as that is what is used in papers about derivatives.

Semi-structural tangents

Where there is no natural tangent type for the outermost type but there is for some of its fields, we call this a "semi-structural" tangent.

Consider if we had a representation of a country's GDP as output by some continuous time model like a Gaussian Process, where that representation is as a sequence of TimeSamples structured as follows:

julia> struct TimeSample
+           time::DateTime
+           value::Float64
+       end

We can look at its structure:

julia> dump(TimeSample(now(), 2.6e9))
+TimeSample
+  time: DateTime
+    instant: Dates.UTInstant{Millisecond}
+      periods: Millisecond
+        value: Int64 63720043490844
+  value: Float64 2.6e9

Thus we see the that structural tangent would be:

Tangent{TimeSample}(
+    time::Tangent{DateTime}(
+        instant::Tangent{UTInstant{Millisecond}}(
+            periods::Tangent{Millisecond}(
+                value::Int64
+            )
+        )
+    ),
+    value::Float64
+)

But instead in the custom sensitivity rule we would write a semi-structured tangent type. Since there is not a natural tangent type for TimeSample but there is for DateTime.

Tangent{TimeSample}(
+    time::Day,
+    value::Float64
+)

So the rule author has written a structural tangent with some fields that are natural tangents.

Another related case is for types that overload getproperty such as SVD and QR. In this case the structural tangent will be based on the fields, but those fields do not always have an easy relation to what is actually used in math. For example, the QR type has fields factors and t, but we would more naturally think in terms of the properties Q and R. So most rule authors would want to write semi-structural tangents based on the properties.

To return to the question of why ChainRules has Tangent{P, <:NamedTuple} whereas Zygote v0.4 just has NamedTuple, it relates to semi-structural derivatives, and being able to overload things more generally. If one knows that one has a semi-structural derivative based on property names, like Tangent{QR}(Q=..., R=...), and one is adding it to the true structural derivative based on field names Tangent{QR}(factors=..., τ=...), then we need to overload the addition operator to perform that correctly. We cannot happily overload similar things for NamedTuple since we don't know the primal type, only the names of the values contained. In fact we can't actually overload addition at all for NamedTuple as that would be type-piracy, so have to use Zygote.accum instead.

Another use of the primal being a type parameter is to catch errors. ChainRules disallows the addition of Tangent{SVD} to Tangent{QR} since in a correctly differentiated program that can never occur.

Tangent types for computational efficiency

There is another kind of unnatural tangent. One that is for computational efficiency. ChainRules has Thunks and InplaceableThunks, which wrap the computation of a derivative and delays that work until it is needed, either via the derivative being added to something or being unthunked manually, thus saving time if it is never used.

Another tangent type used for efficiency is ZeroTangent which represents the hard zero (in Zygote v0.4 this is nothing). For example the derivative of f(x, y)=2x with respect to y is ZeroTangent(). Add ZeroTangent() to anything, and one gets back the original thing without change. We noted that all tangents need to be a vector space. ZeroTangent() is the trivial vector space. Further, add ZeroTangent() to any primal value (no matter the type) and you get back another value of the same primal type (the same value in fact). So it meets the requirements of a tangent type for all primal types. ZeroTangent can save on memory (since we can avoid allocating anything) and on time (since performing the multiplication ZeroTangent and Thunk are both examples of a tangent type that is valid for multiple primal types.

Conclusion

Now, you have seen examples of both tangent types that work for multiple primal types, and primal types that have multiple valid tangent types. Semantically we can handle these very easily in julia. Just put in a few more dispatching on +. Multiple-dispatch is great like that. The down-side is our type-inference becomes hard. If you have exactly 1 tangent type for each primal type, you can very easily work out what all the types on your reverse pass will be - you don't really need type inference - but you lose so much expressiveness.

Appendix: What Swift does

I don't know how Swift is handling thunks, maybe they are not, maybe they have an optimizing compiler that can just slice out code-paths that don't lead to values that get used; maybe they have a language built in for lazy computation.

They are, as I understand it, handling ZeroTangent by requiring every tangent type to define a zero method – which it has since it is a vector space. This costs memory and time, but probably not actually all that much. With regards to handling multiple different tangent types for one primal, like natural and structural derivatives, everything needs to be converted to the canonical tangent type of that primal.

As I understand it, things can be automatically converted by defining conversion protocols or something like that, so rule authors can return anything that has a conversion protocol to the canonical tangent type of the primal.

However, it seems like this will run into problems. Recall that the natural tangent in the case of getindex on an AbstractArray was a sparse array. But for say the standard dense Array, the only reasonable canonical tangent type is also a dense Array. But if you convert a sparse array into a dense array you do giant allocations to fill in all the other entries with zero.

So this is the story about why we have many-to-many tangent types in ChainRules.

diff --git a/previews/PR663/index.html b/previews/PR663/index.html new file mode 100644 index 000000000..a54268c65 --- /dev/null +++ b/previews/PR663/index.html @@ -0,0 +1,65 @@ + +Introduction · ChainRules
GitHub

ChainRules

Automatic differentiation (AD) is a set of techniques for obtaining derivatives of arbitrary functions. There are surprisingly many packages for doing AD in Julia. ChainRules isn't one of these packages.

The AD packages essentially combine derivatives of simple functions into derivatives of more complicated functions. They differ in the way they break down complicated functions into simple ones, but they all require a common set of derivatives of simple functions (rules).

ChainRules is an AD-independent set of rules, and a system for defining and testing rules.

What is a rule?

A rule encodes knowledge about propagating derivatives, e.g. that the derivative with respect to x of a*x is a, and the derivative of sin(x) is cos(x), etc.

ChainRules ecosystem organisation

The ChainRules ecosystem comprises:

AD systems depend on ChainRulesCore.jl to get access to tangent types and the core rule definition functionality (frule and rrule), and on ChainRules.jl to benefit from the collection of rules for Julia Base and the standard libraries.

Packages that just want to define rules only need to depend on ChainRulesCore.jl, which is an exceptionally light dependency. They should also have a test-only dependency on ChainRulesTestUtils.jl to test the rules using finite differences.

Note that the packages with rules do not have to depend on AD systems, and neither do the AD systems have to depend on individual packages.

ChainRules roll-out status

Numerous packages depend on ChainRulesCore to define rules for their functions.

6 AD engines currently use ChainRules to get access to rules:

Zygote.jl is a reverse-mode AD that supports using rrules, calling back into AD, and opting out of rules. However, its own ZygoteRules.jl primitives (@adjoints) take precedence before rrules when both are defined – even if the @adjoint is less specific than the rrule. Internally it uses its own set of tangent types, e.g. nothing instead of NoTangent/ZeroTangent. It also unthunks every tangent.

Diffractor.jl is a forward- and reverse-mode AD that fully supports ChainRules, including calling back into AD, opting out of rules, and uses tangent types internally.

Yota is a reverse-mode AD that fully supports ChainRules, including calling back into AD, opting out of rules, and uses tangent types internally.

ReverseDiff is a reverse-mode AD that supports using rrules, but not calling back into AD and opting out of rules.

Nabla.jl is a reverse-mode AD that supports using rrules, but not opting out of rules, nor calling back into AD.

ReversePropagation.jl is a reverse-mode AD that supports using rrules for scalar functions, but not calling back into AD and opting out of rules.

On the other hand, ForwardDiff.jl is NOT natively compatible with ChainRules. You can use the package ForwardDiffChainRules.jl to bridge this gap.

Key functionality

Consider a relationship $y = f(x)$, where $f$ is some function. Computing $y$ from $x$ is the original problem, called the primal computation, in contrast to the problem of computing derivatives. We say that the primal function $f$ takes a primal input $x$ and returns the primal output $y$.

ChainRules rules are concerned with propagating tangents of primal inputs to tangents of primal outputs (frule, from forwards mode AD), and propagating cotangents of primal outputs to cotangents of primal inputs (rrule, from reverse mode AD). To be able to do that, ChainRules also defines a small number of tangent types to represent tangents and cotangents.

Tangents and cotangents

Strictly speaking tangents, $ẋ = \frac{dx}{da}$, are propagated in frules, and cotangents, $x̄ = \frac{da}{dx}$, are propagated in rrules. However, in practice there is rarely a need to distinguish between the two: both are represented by the same tangent types. Thus, except when the detail might clarify, we refer to both as tangents.

`frule` and `rrule`

frule and rrule are ChainRules specific terms. Their exact functioning is fairly ChainRules specific, though other tools have similar functions. The core notion is sometimes called custom AD primitives, custom adjoints, custom gradients, custom sensitivities. The whole field is a mess for terminology.

Forward-mode AD rules (frules)

If we know the value of $ẋ = \frac{dx}{da}$ for some $a$ and we want to know $ẏ = \frac{dy}{da}$, the chain rule tells us that $ẏ = \frac{dy}{dx} ẋ$. Intuitively, we are pushing the derivative forward. This is the basis for forward-mode AD.

frule

The frule for $f$ encodes how to propagate the tangent of the primal input ($ẋ$) to the tangent of the primal output ($ẏ$).

The frule signature for a function foo(args...; kwargs...) is

function frule((Δself, Δargs...), ::typeof(foo), args...; kwargs...)
+    ...
+    return y, ∂Y
+end

where y = foo(args; kwargs...) is the primal output, and ∂Y is the result of propagating the input tangents Δself, Δargs... forwards at the point in the domain of foo described by args. This propagation is called the pushforward. Often we will think of the frule as having the primal computation y = foo(args...; kwargs...), and the pushforward ∂Y = pushforward(Δself, Δargs...), even though they are not present in separate forms in the code.

For example, the frule for sin(x) is:

function frule((_, Δx), ::typeof(sin), x)
+    return sin(x), cos(x) * Δx
+end

Reverse-mode AD rules (rrules)

If we know the value of $ȳ = \frac{da}{dy}$ for some $a$ and we want to know $x̄ = \frac{da}{dx}$, the chain rule tells us that $x̄ =ȳ \frac{dy}{dx}$. Intuitively, we are pushing the derivative backward. This is the basis for reverse-mode AD.

rrule

The rrule for $f$ encodes how to propagate the cotangents of the primal output ($ȳ$) to the cotangent of the primal input ($x̄$).

The rrule signature for a function foo(args...; kwargs...) is

function rrule(::typeof(foo), args...; kwargs...)
+    ...
+    return y, pullback
+end

where y (the primal output) must be equal to foo(args...; kwargs...). pullback is a function to propagate the derivative information backwards at the point in the domain of foo described by args. That pullback function is used like: ∂self, ∂args... = pullback(Δy) Almost always the pullback will be declared locally within the rrule, and will be a closure over some of the other arguments, and potentially over the primal result too.

For example, the rrule for sin(x) is:

function rrule(::typeof(sin), x)
+    sin_pullback(Δy) = (NoTangent(), cos(x)' * Δy)
+    return sin(x), sin_pullback
+end
Why `rrule` returns a pullback but `frule` doesn't return a pushforward

While rrule takes only the arguments to the original function (the primal arguments) and returns a function (the pullback) that operates with the derivative information, the frule does it all at once. This is because the frule fuses the primal computation and the pushforward. This is an optimization that allows frules to contain single large operations that perform both the primal computation and the pushforward at the same time (for example solving an ODE). This operation is only possible in forward mode (where frule is used) because the derivative information needed by the pushforward available with the frule is invoked – it is about the primal function's inputs. In contrast, in reverse mode the derivative information needed by the pullback is about the primal function's output. Thus the reverse mode returns the pullback function which the caller (usually an AD system) keeps hold of until derivative information about the output is available.

Tangent types

The types of (co)-tangents depend on the types of the primals. Scalar primals are represented by scalar tangents (e.g. Float64 tangent for a Float64 primal). Vector, matrix, and higher rank tensor primals can be represented by vector, matrix and tensor tangents.

ChainRules defines a Tangent tangent type to represent tangents of structs, Tuples, NamedTuples, and Dicts.

Additionally, for signalling semantics, we distinguish between two tangent types representing a zero tangent. NoTangent type represent situations in which the tangent space does not exist, e.g. an index into an array can not be perturbed. ZeroTangent is used for cases where the tangent happens to be zero, e.g. because the primal argument is not used in the computation.

We also define Thunks to allow certain optimisation. Thunks are a wrapper over a computation that can potentially be avoided, depending on the downstream use.

See the section on tangent types for more details.

Example of using ChainRules directly

While ChainRules is largely intended as a backend for autodiff systems, it can be used directly. In fact, this can be very useful if you can constrain the code you need to differentiate to only use things that have rules defined for. This was once how all neural network code worked.

Using ChainRules directly also helps get a feel for it.

using ChainRulesCore
+
+function foo(x)
+    a = sin(x)
+    b = 0.2 + a
+    c = asin(b)
+    return c
+end
+
+# Define rules (alternatively get them for free via `using ChainRules`)
+@scalar_rule(sin(x), cos(x))
+@scalar_rule(+(x, y), (1.0, 1.0))
+@scalar_rule(asin(x), inv(sqrt(1 - x^2)))
#### Find dfoo/dx via rrules
+#### First the forward pass, gathering up the pullbacks
+x = 3;
+a, a_pullback = rrule(sin, x);
+b, b_pullback = rrule(+, 0.2, a);
+c, c_pullback = rrule(asin, b)
+
+#### Then the backward pass calculating gradients
+c̄ = 1;                    # ∂c/∂c
+_, b̄ = c_pullback(c̄);     # ∂c/∂b = ∂c/∂b ⋅ ∂c/∂c
+_, _, ā = b_pullback(b̄);  # ∂c/∂a = ∂c/∂b ⋅ ∂b/∂a
+_, x̄ = a_pullback(ā);     # ∂c/∂x = ∂c/∂a ⋅ ∂a/∂x
+x̄                         # ∂c/∂x = ∂foo/∂x
+# output
+-1.0531613736418153
#### Find dfoo/dx via frules
+x = 3;
+ẋ = 1;              # ∂x/∂x
+nofields = ZeroTangent();  # ∂self/∂self
+
+a, ȧ = frule((nofields, ẋ), sin, x);                    # ∂a/∂x = ∂a/∂x ⋅ ∂x/∂x 
+b, ḃ = frule((nofields, ZeroTangent(), ȧ), +, 0.2, a);  # ∂b/∂x = ∂b/∂a ⋅ ∂a/∂x
+c, ċ = frule((nofields, ḃ), asin, b);                   # ∂c/∂x = ∂c/∂b ⋅ ∂b/∂x
+ċ                                                       # ∂c/∂x = ∂foo/∂x
+# output
+-1.0531613736418153
#### Find dfoo/dx via FiniteDifferences.jl
+using FiniteDifferences
+central_fdm(5, 1)(foo, x)
+# output
+-1.0531613736418257
+
+#### Find dfoo/dx via ForwardDiff.jl
+using ForwardDiff
+ForwardDiff.derivative(foo, x)
+# output
+-1.0531613736418153
+
+#### Find dfoo/dx via Zygote.jl
+using Zygote
+Zygote.gradient(foo, x)
+# output
+(-1.0531613736418153,)
diff --git a/previews/PR663/maths/arrays.html b/previews/PR663/maths/arrays.html new file mode 100644 index 000000000..7e345a778 --- /dev/null +++ b/previews/PR663/maths/arrays.html @@ -0,0 +1,328 @@ + +Deriving array rules · ChainRules
GitHub

Deriving Array Rules

One of the goals of the ChainRules interface is to make it easy to define your own rules for a function. This tutorial attempts to demystify deriving and implementing custom rules for arrays with real and complex entries, with examples. The approach we use is similar to the one succinctly explained and demonstrated in [Giles2008] and its extended work [Giles2008ext], but we generalize it to support functions of multidimensional arrays with both real and complex entries.

Throughout this tutorial, we will use the following type alias:

const RealOrComplex = Union{Real,Complex}

Forward-mode rules

Approach

Consider a function

Ω = f(X::Array{<:RealOrComplex}...)::Array{<:RealOrComplex}

or in math notation

\[f: (\ldots, X_m, \ldots) \mapsto \Omega,\]

where the components of $X_m$ are written as $(X_m)_{i,\ldots,j}$. The variables $X_m$ and $\Omega$ are intermediates in a larger program (function) that, by considering only a single real input $t$ and real output $s$ can always be written as

\[t \mapsto (\ldots, X_m, \ldots) \mapsto \Omega \mapsto s,\]

where $t$ and $s$ are real numbers. If we know the partial derivatives of $X_m$ with respect to $t$, $\frac{dX_m}{dt} = \dot{X}_m$, the chain rule gives the pushforward of $f$ as:

\[\begin{equation} \label{pf} +\dot{\Omega} + = f_*(\ldots, \dot{X}_m, \ldots) + = \sum_m \sum_{i, \ldots, j} + \frac{\partial \Omega}{ \partial (X_m)_{i,\ldots,j} } (\dot{X}_m)_{i,\ldots,j} +\end{equation}\]

That's ugly, but in practice we can often write it more simply by using forward mode rules for simpler functions, as we'll see below. The forward-mode rules for arrays follow directly from the usual scalar chain rules.

Array addition

Ω = A + B

This one is easy:

\[\Omega = A + B\]

\[\dot{\Omega} = \dot{A} + \dot{B}\]

We can implement the frule in ChainRules's notation:

function frule(
+    (_, ΔA, ΔB),
+    ::typeof(+),
+    A::Array{<:RealOrComplex},
+    B::Array{<:RealOrComplex},
+)
+    Ω = A + B
+    ∂Ω = ΔA + ΔB
+    return (Ω, ∂Ω)
+end

Matrix multiplication

Ω = A * B

\[\Omega = A B\]

First we write in component form:

\[\Omega_{ij} = \sum_k A_{ik} B_{kj}\]

Then we use the product rule to get the pushforward for each scalar entry:

\[\begin{align*} +\dot{\Omega}_{ij} + &= \sum_k \left( \dot{A}_{ik} B_{kj} + A_{ik} \dot{B}_{kj} \right) + && \text{apply scalar product rule } + \frac{d}{dt}(x y) = \frac{dx}{dt} y + x \frac{dy}{dt} \\ + &= \sum_k \dot{A}_{ik} B_{kj} + \sum_k A_{ik} \dot{B}_{kj} + && \text{split sum} +\end{align*}\]

But the last expression is just the component form of a sum of matrix products:

\[\begin{equation}\label{diffprod} +\dot{\Omega} = \dot{A} B + A \dot{B} +\end{equation}\]

This is the matrix product rule, and we write its frule as

function frule(
+    (_, ΔA, ΔB),
+    ::typeof(*),
+    A::Matrix{<:RealOrComplex},
+    B::Matrix{<:RealOrComplex},
+)
+    Ω = A * B
+    ∂Ω = ΔA * B + A * ΔB
+    return (Ω, ∂Ω)
+end

Matrix inversion

Ω = inv(A)

\[\Omega = A^{-1}\]

It's easiest to derive this rule from either of the two constraints:

\[\begin{align*} +\Omega A &= A^{-1} ~A = I\\ +A \Omega &= A~ A^{-1} = I, +\end{align*}\]

where $I$ is the identity matrix.

We use the matrix product rule to differentiate the first constraint:

\[\dot{\Omega} A + \Omega \dot{A} = 0\]

Then, right-multiply both sides by $A^{-1}$ to isolate $\dot{\Omega}$:

\[\begin{align} +0 &= \dot{\Omega}~ A~ A^{-1} + \Omega ~\dot{A}~ A^{-1} \nonumber\\ + &= \dot{\Omega}~ I + \Omega ~\dot{A}~ A^{-1} + && \text{use } A~ A^{-1} = I \nonumber\\ + &= \dot{\Omega} + \Omega \dot{A} \Omega + && \text{substitute } A^{-1} = \Omega \nonumber\\ +\dot{\Omega} + &= -\Omega \dot{A} \Omega + && \text{solve for } \dot{\Omega} \label{invdiff} +\end{align}\]

We write the frule as

function frule((_, ΔA), ::typeof(inv), A::Matrix{<:RealOrComplex})
+    Ω = inv(A)
+    ∂Ω = -Ω * ΔA * Ω
+    return (Ω, ∂Ω)
+end

Other useful identities

These identities are particularly useful:

\[\begin{align*} +\frac{d}{dt} \left( \Re(A) \right) &= \Re(\dot{A})\\ +\frac{d}{dt} \left( A^* \right) &= \dot{A}^*\\ +\frac{d}{dt} \left( A^\mathsf{T} \right) &= \dot{A}^\mathsf{T}\\ +\frac{d}{dt} \left( A^\mathsf{H} \right) &= \dot{A}^\mathsf{H}\\ +\frac{d}{dt} \left( \sum_{j} A_{i \ldots j \ldots k} \right) &= + \sum_{j} \dot{A}_{i \ldots j \ldots k}, +\end{align*}\]

where $\cdot^*$ is the complex conjugate (conj), and $\cdot^\mathsf{H} = \left(\cdot^\mathsf{T}\right)^*$ is the conjugate transpose (the adjoint function).

Reverse-mode rules

Approach

Reverse-mode rules are a little less intuitive, but we can re-use our pushforwards to simplify their derivation. Recall our program:

\[t \mapsto (\ldots, X_m, \ldots) \mapsto \Omega \mapsto s,\]

At any step in the program, if we have intermediates $X_m$, we can write down the derivative $\frac{ds}{dt}$ in terms of the tangents $\dot{X}_m = \frac{dX_m}{dt}$ and adjoints $\overline{X}_m = \frac{\partial s}{\partial X_m}$

\[\begin{align*} +\frac{ds}{dt} + &= \sum_m \Re\left( \sum_{i,\ldots,j} + \left( \frac{\partial s}{\partial (X_m)_{i,\ldots,j}} \right)^* + \frac{d (X_m)_{i,\ldots,j}}{dt} + \right)\\ + &= \sum_m \Re\left( \sum_{i,\ldots,j} + (\overline{X}_m)_{i,\ldots,j}^* + (\dot{X}_m)_{i,\ldots,j} + \right)\\ + &= \sum_m \Re\ip{ \overline{X}_m }{ \dot{X}_m }, +\end{align*}\]

where $\Re(\cdot)$ is the real part of a number (real), and $\ip{\cdot}{\cdot}$ is the Frobenius inner product (LinearAlgebra.dot). Because this equation follows at any step of the program, we can equivalently write

\[\frac{ds}{dt} = \Re\ip{ \overline{\Omega} }{ \dot{\Omega} },\]

which gives the identity

\[\begin{equation} \label{pbident} +\Re\ip{ \overline{\Omega} }{ \dot{\Omega} } = \sum_m \Re\ip{ \overline{X}_m }{ \dot{X}_m }. +\end{equation}\]

For matrices and vectors, $\ip{A}{B} = \tr(A^\mathsf{H} B)$, and the identity simplifies to:

\[\begin{equation} \label{pbidentmat} +\Re\left( \tr\left( + \overline{\Omega}^\mathsf{H} \dot{\Omega} +\right) \right) = +\sum_m \Re \left( \tr \left( + \overline{X}_m^\mathsf{H} \dot{X}_m +\right) \right), +\end{equation}\]

where $\tr(\cdot)$ is the matrix trace (LinearAlgebra.tr) function. However, it is often cleaner and more general to work with the inner product.

Our approach for deriving the adjoints $\overline{X}_m$ is then:

  1. Derive the pushforward ($\dot{\Omega}$ in terms of $\dot{X}_m$) using \eqref{pf}.
  2. Substitute this expression for $\dot{\Omega}$ into the left-hand side of \eqref{pbident}.
  3. Manipulate until it looks like the right-hand side of \eqref{pbident}.
  4. Solve for each $\overline{X}_m$.

Note that the final expressions for the adjoints will not contain any $\dot{X}_m$ terms.

Note

Why do we conjugate, and why do we only use the real part of the dot product in \eqref{pbident}? Recall from Complex Numbers that we treat a complex number as a pair of real numbers. These identities are a direct consequence of this convention. Consider $\frac{ds}{dt}$ for a scalar function $f: (x + i y) \mapsto (u + i v)$:

\[\begin{align*} +\frac{ds}{dt} + &= \Re\ip{ \overline{x} + i \overline{y} }{ \dot{x} + i \dot{y} } \\ + &= \Re\left( + \left( \overline{x} + i \overline{y} \right)^* + \left( \dot{x} + i \dot{y} \right) + \right) \\ + &= \Re\left( + \left( \overline{x} - i \overline{y} \right) + \left( \dot{x} + i \dot{y} \right) + \right) \\ + &= \Re\left( + \left( \overline{x} \dot{x} + \overline{y} \dot{y} \right) + + i \left( \overline{x} \dot{y} - \overline{y} \dot{x} \right) + \right)\\ + &= \overline{x} \dot{x} + \overline{y} \dot{y}\\ +\end{align*}\]

which is exactly what the identity would produce if we had written the function as $f: (x, y) \mapsto (u, v)$.

Useful properties of the inner product

Several properties of the Frobenius inner product come in handy. First, it is linear in its second argument and conjugate linear in its first. That is, for arrays $A, B, C, D$ and scalars $a$ and $b$,

\[\begin{align} +\ip{A+B}{C+D} &= \ip{A}{C} + \ip{B}{C} + \ip{A}{D} + \ip{B}{D} \label{iplinear}\\ +\ip{aA}{bB} &= a^* b \ip{A}{B} \nonumber +\end{align}\]

Second, swapping arguments is equivalent to conjugating the inner product:

\[\begin{equation} +\ip{A}{B} = \ip{B}{A}^* \label{ipconj} +\end{equation}\]

Third, for matrices and vectors $A$, $B$, and $C$, we can move arguments from the left or right of one side to the other using the matrix adjoint:

\[\begin{equation} +\ip{A}{BCD} = \ip{B^\mathsf{H} A}{CD} = \ip{B^\mathsf{H} A D^\mathsf{H}}{C} \label{ipperm} +\end{equation}\]

Fourth, the inner product of two arrays $A$ and $B$ is equivalent to the sum of the elementwise inner products of the two arrays:

\[\begin{equation} +\ip{A}{B} = \sum_{i,\ldots,k} \ip{A_{i,\ldots,k}}{B_{i,\ldots,k}} = \sum_{i,\ldots,k} A_{i,\ldots,k}^* B_{i,\ldots,k} +\end{equation}\]

As a result, only elements that are nonzero on both sides contribute to the inner product. This property is especially useful when deriving rules involving structurally sparse arrays.

Now let's derive a few pullbacks using this approach.

Matrix multiplication

Ω = A * B

We above derived in \eqref{diffprod} the pushforward

\[\dot{\Omega} = \dot{A} B + A \dot{B}\]

Using \eqref{pbidentmat}, we now multiply by $\overline{\Omega}^\mathsf{H}$ and take the real trace:

\[\begin{align*} +\Re\ip{\overline{\Omega}}{\dot{\Omega}} + &= \Re \ip{\overline \Omega}{\dot{A} B + A \dot{B}} + && \text{substitute } \dot{\Omega} \text{ from } \eqref{diffprod}\\ + &= \Re \ip{\overline \Omega}{\dot{A} B} + \Re \ip{\overline \Omega}{A \dot{B}} + && \text{expand using } \eqref{iplinear} \\ + &= \Re \ip{\overline \Omega B^\mathsf{H}}{\dot{A}} + \Re \ip{A^\mathsf{H} \overline \Omega}{\dot{B}} + && \text{rearrange the left term using } \eqref{ipperm}\\ + &= \Re \ip{\overline A}{\dot{A}} + \Re \ip{\overline B}{\dot{B}} + && \text{right-hand side of } \eqref{pbidentmat} +\end{align*}\]

That's it! The expression is in the desired form to solve for the adjoints by comparing the last two lines:

\[\overline A = \overline \Omega B^\mathsf{H}, \qquad \overline B = A^\mathsf{H} \overline \Omega\]

Using ChainRules's notation, we would implement the rrule as

function rrule(::typeof(*), A::Matrix{<:RealOrComplex}, B::Matrix{<:RealOrComplex})
+    function times_pullback(ΔΩ)
+        ∂A = @thunk(ΔΩ * B')
+        ∂B = @thunk(A' * ΔΩ)
+        return (NoTangent(), ∂A, ∂B)
+    end
+    return A * B, times_pullback
+end

Matrix inversion

Ω = inv(A)

In \eqref{invdiff}, we derived the pushforward as

\[\dot{\Omega} = -\Omega \dot{A} \Omega\]

Using \eqref{pbidentmat},

\[\begin{align*} +\Re\ip{\overline{\Omega}}{\dot{\Omega}} + &= \Re\ip{\overline{\Omega}}{-\Omega \dot{A} \Omega} + && \text{substitute } \eqref{invdiff}\\ + &= \Re\ip{-\Omega^\mathsf{H} \overline{\Omega} \Omega^\mathsf{H}}{\dot{A}} + && \text{rearrange using } \eqref{ipperm}\\ + &= \Re\ip{\overline{A}}{\dot{A}} + && \text{right-hand side of } \eqref{pbidentmat} +\end{align*}\]

we can now solve for $\overline{A}$:

\[\overline{A} = -\Omega^\mathsf{H} \overline{\Omega} \Omega^\mathsf{H}\]

We can implement the resulting rrule as

function rrule(::typeof(inv), A::Matrix{<:RealOrComplex})
+    Ω = inv(A)
+    function inv_pullback(ΔΩ)
+        ∂A = -Ω' * ΔΩ * Ω'
+        return (NoTangent(), ∂A)
+    end
+    return Ω, inv_pullback
+end

A multidimensional array example

We presented the approach for deriving pushforwards and pullbacks for arrays of arbitrary dimensions, so let's cover an example. For multidimensional arrays, it's often easier to work in component form. Consider the following function:

Ω = sum(abs2, X::Array{<:RealOrComplex,3}; dims=2)::Array{<:Real,3}

which we write as

\[\Omega_{i1k} = \sum_{j} |X_{ijk}|^2 + = \sum_{j} \Re \ip{X_{ijk}}{X_{ijk}}\]

The pushforward from \eqref{pf} is

\[\begin{align} +\dot{\Omega}_{i1k} + &= \sum_j \Re\ip{\dot{X}_{ijk}}{X_{ijk}} + \ip{X_{ijk}}{\dot{X}_{ijk}} \nonumber\\ + &= \sum_j \Re\ip{X_{ijk}}{\dot{X}_{ijk}}^* + \ip{X_{ijk}}{\dot{X}_{ijk}} \nonumber\\ + &= \sum_j 2 \Re\ip{X_{ijk}}{\dot{X}_{ijk}}, \label{sumabspf} +\end{align}\]

where in the last step we have used the fact that for all real $a$ and $b$,

\[(a + i b) + (a + i b)^* + = (a + i b) + (a - i b) + = 2 a + = 2 \Re (a + i b).\]

Because none of this derivation depended on the index (or indices), we implement frule generically as

function frule(
+    (_, _, ΔX),
+    ::typeof(sum),
+    ::typeof(abs2),
+    X::Array{<:RealOrComplex};
+    dims = :,
+)
+    Ω = sum(abs2, X; dims = dims)
+    ∂Ω = sum(2 .* real.(conj.(X) .* ΔX); dims = dims)
+    return (Ω, ∂Ω)
+end

We can now derive the reverse-mode rule. The elementwise form of \eqref{pbident} is

\[\begin{align*} +\Re\ip{ \overline{\Omega} }{ \dot{\Omega} } + &= \Re \left( \sum_{ik} \overline{\Omega}_{i1k}^* + \dot{\Omega}_{i1k} \right) + && \text{expand left-hand side of } \eqref{pbident}\\ + &= \Re \left(\sum_{ijk} \overline{\Omega}_{i1k}^* + 2 \Re\left( X_{ijk}^* \dot{X}_{ijk} \right) + \right) + && \text{substitute } \eqref{sumabspf}\\ + &= \Re \left( \sum_{ijk} + \left( + 2 \Re \left( \overline{\Omega}_{i1k} \right) + X_{ijk}^* + \right) \dot{X}_{ijk} + \right) + && \text{bring } \dot{X}_{ijk} \text{ outside of } \Re\\ + &= \sum_{ijk} \Re\ip{2 \Re \left( \overline{\Omega}_{i1k} \right) X_{ijk}}{\dot{X}_{ijk}} + && \text{rewrite as an inner product}\\ + &= \sum_{ijk} \Re\ip{\overline{X}_{ijk}}{\dot{X}_{i1k}} + && \text{right-hand side of } \eqref{pbident} +\end{align*}\]

We now solve for $\overline{X}$:

\[\overline{X}_{ijk} = 2\Re \left( \overline{\Omega}_{i1k} \right) X_{ijk}\]

Like the frule, this rrule can be implemented generically:

function rrule(::typeof(sum), ::typeof(abs2), X::Array{<:RealOrComplex}; dims = :)
+    function sum_abs2_pullback(ΔΩ)
+        ∂abs2 = NoTangent()
+        ∂X = @thunk(2 .* real.(ΔΩ) .* X)
+        return (NoTangent(), ∂abs2, ∂X)
+    end
+    return sum(abs2, X; dims = dims), sum_abs2_pullback
+end

Functions that return a tuple

Every Julia function returns a single output. For example, let's look at LinearAlgebra.logabsdet, the logarithm of the absolute value of the determinant of a matrix, which returns $\log |\det(A)|$ and $\operatorname{sign}(\det A) = \frac{\det A}{| \det A |}$:

(l, s) = logabsdet(A)

The return type is actually a single output, a tuple of scalars, but when deriving, we treat them as multiple outputs. The left-hand side of \eqref{pbident} then becomes a sum over terms, just like the right-hand side.

Let's derive the forward- and reverse-mode rules for logabsdet.

\[\begin{align*} +l &= \log |\det(A)|\\ +s &= \operatorname{sign}(\det(A)), +\end{align*}\]

where $\operatorname{sign}(x) = \frac{x}{|x|}$.

Forward-mode rule

To make this easier, let's break the computation into more manageable steps:

\[\begin{align*} +d &= \det(A)\\ +a &= |d| = \sqrt{\Re \left( d^* d \right)}\\ +l &= \log a\\ +s &= \frac{d}{a} +\end{align*}\]

We'll make frequent use of the identities:

\[d = a s\]

\[s^* s = \frac{d^* d}{a^2} = \frac{a^2}{a^2} = 1\]

It will also be useful to define $b = \tr\left( A^{-1} \dot{A} \right)$.

For $\dot{d}$, we use the pushforward for the determinant given in section 2.2.4 of [Giles2008ext]:

\[\dot{d} = d b\]

Now we'll compute the pushforwards for the remaining steps.

\[\begin{align*} +\dot{a} &= \frac{1}{2 a} \frac{d}{dt} + \Re\left( d^* d \right)\\ + &= \frac{2}{2 a} \Re \left( d^* \dot{d} \right)\\ + &= \Re \left( s^* \dot{d} \right) + && \text{use } d = a s \\ + &= \Re \left( s^* d b \right) + && \text{substitute } \dot{d} \\ +\dot{l} &= a^{-1} \dot{a}\\ + &= a^{-1} \Re \left( s^* d b \right) + && \text{substitute } \dot{a}\\ + &= \Re \left( s^* s b \right) + && \text{use } d = a s \\ + &= \Re \left(b \right) + && \text{use } s^* s = 1\\ +\dot{s} &= a^{-1} \dot{d} - a^{-2} d \dot{a}\\ + &= a^{-1} \left( \dot{d} - \dot{a} s \right) + && \text{use } d = a s \\ + &= a^{-1} \left( + \dot{d} - \Re \left( s^* \dot{d} \right) s + \right) + && \text{substitute } \dot{a}\\ + &= a^{-1} \left( + \dot{d} - \left( + s^* \dot{d} - + i \Im \left( s^* \dot{d} \right) + \right) s + \right) + && \text{use } \Re(x) = x - i \Im(x)\\ + &= a^{-1} \left( + \dot{d} - \left( s^* s \right) \dot{d} + + i \Im \left( s^* \dot{d} \right) s + \right)\\ + &= i a^{-1} \Im \left( s^* \dot{d} \right) s + && \text{use } s^* s = 1\\ + &= i a^{-1} \Im \left( s^* d b \right) s + && \text{substitute } \dot{d}\\ + &= i \Im \left( s^* s b \right) s + && \text{use } d = a s \\ + &= i \Im(b) s + && \text{use } s^* s = 1 +\end{align*}\]

Note that the term $b$ is reused. In summary, after all of that work, the final pushforward is quite simple:

\[\begin{align} +b &= \tr \left( A^{-1} \dot{A} \right) \label{logabsdet_b} \\ +\dot{l} &= \Re(b) \label{logabsdet_ldot}\\ +\dot{s} &= i \Im(b) s \label{logabsdet_sdot}\\ +\end{align}\]

We can define the frule as:

function frule((_, ΔA), ::typeof(logabsdet), A::Matrix{<:RealOrComplex})
+    # The primal function uses the lu decomposition to compute logabsdet
+    # we reuse this decomposition to compute inv(A) * ΔA
+    F = lu(A, check = false)
+    Ω = logabsdet(F)  # == logabsdet(A)
+    b = tr(F \ ΔA)  # == tr(inv(A) * ΔA)
+    s = last(Ω)
+    ∂l = real(b)
+    # for real A, ∂s will always be zero (because imag(b) = 0)
+    # this is type-stable because the eltype is known
+    ∂s = eltype(A) <: Real ? ZeroTangent() : im * imag(b) * s
+    # tangents of tuples are of type Tangent{<:Tuple}
+    ∂Ω = Tangent{typeof(Ω)}(∂l, ∂s)
+    return (Ω, ∂Ω)
+end

Reverse-mode rule

\[\begin{align*} +&\Re\ip{\overline{l}}{\dot{l}} + \Re\ip{\overline{s}}{\dot{s}} + && \text{left-hand side of } \eqref{pbidentmat}\\ +&= \Re\left( \overline{l}^* \dot{l} + \overline{s}^* \dot{s} \right) \\ +&= \Re\left( + \overline{l}^* \Re(b) + i \overline{s}^* s \Im(b) + \right) + && \text{substitute } \eqref{logabsdet_ldot} \text{ and } \eqref{logabsdet_sdot} \\ +&= \Re\left( + \Re\left( \overline{l} \right) \Re(b) - + \Im \left( \overline{s}^* s \right) \Im(b) + \right) + && \text{discard imaginary parts} \\ +&= \Re\left( + \left( + \Re \left( \overline{l} \right) + + i \Im \left( \overline{s}^* s \right) + \right) b + \right) + && \text{gather parts of } b \\ +&= \Re\left( + \left( + \Re \left( \overline{l} \right) + + i \Im \left( \overline{s}^* s \right) + \right) + \tr(A^{-1} \dot{A}) + \right) + && \text{substitute } b \text{ from } \eqref{logabsdet_b} \\ +&= \Re\left( \tr \left( + \left( + \Re \left( \overline{l} \right) + + i \Im \left( \overline{s}^* s \right) + \right) + A^{-1} \dot{A} + \right) \right) + && \text{bring scalar within } \tr \\ +&= \Re\ip{ + \left( + \Re \left( \overline{l} \right) + i \Im \left( s^* \overline{s} \right) + \right) A^{-\mathsf{H}} + }{\dot{A}} && \text{rewrite as inner product}\\ +&= \Re\ip{\overline{A}}{\dot{A}} && \text{right-hand side of } \eqref{pbidentmat}\\ +\end{align*}\]

Now we solve for $\overline{A}$:

\[\begin{align*} +\overline{A} = \left( + \Re \left( \overline{l} \right) + + i \Im \left( s^* \overline{s} \right) +\right) A^{-\mathsf{H}} +\end{align*}\]

The rrule can be implemented as

function rrule(::typeof(logabsdet), A::Matrix{<:RealOrComplex})
+    # The primal function uses the lu decomposition to compute logabsdet
+    # we reuse this decomposition to compute inv(A)
+    F = lu(A, check = false)
+    Ω = logabsdet(F)  # == logabsdet(A)
+    s = last(Ω)
+    function logabsdet_pullback(ΔΩ)
+        (Δl, Δs) = ΔΩ
+        f = conj(s) * Δs
+        imagf = f - real(f)  # 0 for real A and Δs, im * imag(f) for complex A and/or Δs
+        g = real(Δl) + imagf
+        ∂A = g * inv(F)'  # == g * inv(A)'
+        return (NoTangent(), ∂A)
+    end
+    return (Ω, logabsdet_pullback)
+end
Note

It's a good idea when deriving pushforwards and pullbacks to verify that they make sense. For the pushforward, since $l$ is real, it follows that $\dot{l}$ is too.

What about $\dot{s}$? Well, $s = \frac{d}{|d|}$ is point on the unit circle in the complex plane. Multiplying a complex number by $i$ rotates it counter-clockwise by 90°. So the expression for $\dot{s}$ takes a real number, $\Im(b)$, multiplies by $s$ to make it parallel to $s$, then multiplies by $i$ to make it perpendicular to $s$, that is, perfectly tangent to the unit complex circle at $s$.

For the pullback, it again follows that only the real part of $\overline{l}$ is pulled back.

$s^*$ rotates a number parallel to $s$ to the real line. So $s^* \overline{s}$ rotates $\overline{s}$ so that its imaginary part is the part that was tangent to the complex circle at $s$, while the real part is the part that was not tangent. Then the pullback isolates the imaginary part, which effectively is a projection. That is, any part of the adjoint $\overline{s}$ that is not tangent to the complex circle at $s$ will not contribute to $\overline{A}$.

Implicit functions

Sometimes a function is only defined implicitly, and internally some solver or iterative algorithm is used to compute the result. We can still in some cases derive rules by considering only the implicit functions and not the internals. One example is the solution $X$ to the Sylvester equation

\[A X + X B = -C\]

for inputs $A$, $B$, and $C$. We can also write this solution as $X = \operatorname{sylvester}(A, B, C)$, which in Julia is computed using LinearAlgebra.sylvester(A, B, C).

Forward-mode Rule

We start by differentiating the implicit function:

\[\dot{A} X + A \dot{X} + \dot{X} B + X \dot{B} = -\dot{C}\]

Then we isolate the terms with $\dot{X}$ on one side:

\[\begin{align} +A \dot{X} + \dot{X} B + &= -\dot{C} - \dot{A} X - X \dot{B} \label{sylpfimplicit}\\ + &= -(\dot{C} + \dot{A} X + X \dot{B}) \nonumber +\end{align}\]

So the pushforward is the solution to a different Sylvester equation:

\[\dot{X} = \operatorname{sylvester}(A, B, \dot{C} + \dot{A} X + X \dot{B})\]

The frule can be implemented as

function frule((_, ΔA, ΔB, ΔC), ::typeof(sylvester), A, B, C)
+    X = sylvester(A, B, C)
+    return X, sylvester(A, B, ΔC + ΔA * X + X * ΔB)
+end

Reverse-mode Rule

Like with the pushforward, it's easiest to work with the implicit function. We start by introducing some dummy $-Z$ and taking its inner product with both sides of \eqref{sylpfimplicit}:

\[\ip{-Z}{A \dot{X} + \dot{X} B} = \ip{-Z}{-\dot{C} - \dot{A} X - X \dot{B}}.\]

Then we expand

\[\ip{-Z}{A \dot{X}} + \ip{-Z}{\dot{X} B} = \ip{Z}{\dot{C}} + \ip{Z}{\dot{A} X} + \ip{Z}{X \dot{B}}.\]

Now permute:

\[\ip{-A^\mathsf{H} Z}{\dot{X}} + \ip{-Z B^\mathsf{H}}{\dot{X}} = \ip{Z}{\dot{C}} + \ip{Z X^\mathsf{H}}{\dot{A}} + \ip{X^\mathsf{H} Z}{\dot{B}}.\]

Then combine:

\[\ip{-(A^\mathsf{H} Z + Z B^\mathsf{H})}{\dot{X}} = \ip{Z X^\mathsf{H}}{\dot{A}} + \ip{X^\mathsf{H} Z}{\dot{B}} + \ip{Z}{\dot{C}}.\]

This is almost exactly the identity we need to solve for $\overline{A}$, $\overline{B}$, and $\overline{C}$. To manipulate it to the right form, we need only define $A^\mathsf{H} Z + Z B^\mathsf{H} = -\overline{X}$. This yet another Sylvester equation, so letting $Z = \overline{C}$, our final pullback is:

\[\begin{align*} +\overline{C} &= \operatorname{sylvester}(A^\mathsf{H}, B^\mathsf{H}, \overline{X})\\ + &= \operatorname{sylvester}(B, A, \overline{X}^\mathsf{H})^\mathsf{H}\\ +\overline{A} &= \overline{C} X^\mathsf{H}\\ +\overline{B} &= X^\mathsf{H} \overline{C}\\ +\end{align*}\]

The rrule can be implemented as

function rrule(::typeof(sylvester), A, B, C)
+    X = sylvester(A, B, C)
+    function sylvester_pullback(ΔX)
+        ∂C = copy(sylvester(B, A, copy(ΔX'))')
+        return NoTangent(), @thunk(∂C * X'), @thunk(X' * ∂C), ∂C
+    end
+    return X, sylvester_pullback
+end

Note, however, that the Sylvester equation is usually solved using the Schur decomposition of $A$ and $B$. These Schur decompositions can be reused to solve the Sylvester equations in the pushforward and pullback. See the implementation in ChainRules for details.

More examples

For more instructive examples of array rules, see [Giles2008ext] (real vector and matrix rules) and the LinearAlgebra rules in ChainRules. For differentiating the LU decomposition, see this blog post by Seth Axen.

References

  • Giles2008

    Giles M. B. Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation. In: Advances in Automatic Differentiation. Lecture Notes in Computational Science and Engineering, vol 64: pp 35-44. Springer, Berlin (2008). doi: 10.1007/978-3-540-68942-3_4. pdf

  • Giles2008ext

    Giles M. B. An Extended Collection of Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation. (unpublished). pdf

diff --git a/previews/PR663/maths/complex.html b/previews/PR663/maths/complex.html new file mode 100644 index 000000000..1b77ff8f6 --- /dev/null +++ b/previews/PR663/maths/complex.html @@ -0,0 +1,52 @@ + +Complex numbers · ChainRules
GitHub

How do chain rules work for complex functions?

ChainRules follows the convention that frule applied to a function $f(x + i y) = u(x,y) + i v(x,y)$ with perturbation $\Delta x + i \Delta y$ returns the value and

\[\tfrac{\partial u}{\partial x} \, \Delta x + \tfrac{\partial u}{\partial y} \, \Delta y + i \, \Bigl( \tfrac{\partial v}{\partial x} \, \Delta x + \tfrac{\partial v}{\partial y} \, \Delta y \Bigr) +.\]

Similarly, rrule applied to the same function returns the value and a pullback function which, when applied to the adjoint $\Delta u + i \Delta v$, returns

\[\Delta u \, \tfrac{\partial u}{\partial x} + \Delta v \, \tfrac{\partial v}{\partial x} + i \, \Bigl(\Delta u \, \tfrac{\partial u }{\partial y} + \Delta v \, \tfrac{\partial v}{\partial y} \Bigr) +.\]

If we interpret complex numbers as vectors in $\mathbb{R}^2$, then frule (rrule) corresponds to multiplication with the (transposed) Jacobian of $f(z)$, i.e. frule corresponds to

\[\begin{pmatrix} +\tfrac{\partial u}{\partial x} \, \Delta x + \tfrac{\partial u}{\partial y} \, \Delta y +\\ +\tfrac{\partial v}{\partial x} \, \Delta x + \tfrac{\partial v}{\partial y} \, \Delta y +\end{pmatrix} += +\begin{pmatrix} +\tfrac{\partial u}{\partial x} & \tfrac{\partial u}{\partial y} \\ +\tfrac{\partial v}{\partial x} & \tfrac{\partial v}{\partial y} \\ +\end{pmatrix} +\begin{pmatrix} +\Delta x \\ \Delta y +\end{pmatrix} +\]

and rrule corresponds to

\[\begin{pmatrix} +\tfrac{\partial u}{\partial x} \, \Delta u + \tfrac{\partial v}{\partial x} \, \Delta v +\\ +\tfrac{\partial u}{\partial y} \, \Delta u + \tfrac{\partial v}{\partial y} \, \Delta v +\end{pmatrix} += +\begin{pmatrix} +\tfrac{\partial u}{\partial x} & \tfrac{\partial u}{\partial y} \\ +\tfrac{\partial v}{\partial x} & \tfrac{\partial v}{\partial y} \\ +\end{pmatrix}^\mathsf{T} +\begin{pmatrix} +\Delta u \\ \Delta v +\end{pmatrix} +.\]

The Jacobian of $f:\mathbb{C} \to \mathbb{C}$ interpreted as a function $\mathbb{R}^2 \to \mathbb{R}^2$ can hence be evaluated using either of the following functions.

function jacobian_via_frule(f,z)
+    du_dx, dv_dx = reim(frule((ZeroTangent(), 1),f,z)[2])
+    du_dy, dv_dy = reim(frule((ZeroTangent(),im),f,z)[2])
+    return [
+        du_dx  du_dy
+        dv_dx  dv_dy
+    ]
+end
function jacobian_via_rrule(f,z)
+    _, pullback = rrule(f,z)
+    du_dx, du_dy = reim(pullback( 1)[2])
+    dv_dx, dv_dy = reim(pullback(im)[2])
+    return [
+        du_dx  du_dy
+        dv_dx  dv_dy
+    ]
+end

If $f(z)$ is holomorphic, then the derivative part of frule can be implemented as $f'(z) \, \Delta z$ and the derivative part of rrule can be implemented as $\bigl(f'(z)\bigr)^* \, \Delta f$, where $\cdot^*$ is the complex conjugate. Consequently, holomorphic derivatives can be evaluated using either of the following functions.

function holomorphic_derivative_via_frule(f,z)
+    fz,df_dz = frule((ZeroTangent(),1),f,z)
+    return df_dz
+end
function holomorphic_derivative_via_rrule(f,z)
+    fz, pullback = rrule(f,z)
+    dself, conj_df_dz = pullback(1)
+    return conj(conj_df_dz)
+end
Note

There are various notions of complex derivatives (holomorphic and Wirtinger derivatives, Jacobians, gradients, etc.) which differ in subtle but important ways. The goal of ChainRules is to provide the basic differentiation rules upon which these derivatives can be implemented, but it does not implement these derivatives itself. It is recommended that you carefully check how the above definitions of frule and rrule translate into your specific notion of complex derivative, since getting this wrong will quietly give you wrong results.

diff --git a/previews/PR663/maths/nondiff_points-359f64e4.svg b/previews/PR663/maths/nondiff_points-359f64e4.svg new file mode 100644 index 000000000..3219d70f3 --- /dev/null +++ b/previews/PR663/maths/nondiff_points-359f64e4.svg @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-a4828821.svg b/previews/PR663/maths/nondiff_points-a4828821.svg new file mode 100644 index 000000000..054d3d22a --- /dev/null +++ b/previews/PR663/maths/nondiff_points-a4828821.svg @@ -0,0 +1,37 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-bd317c35.svg b/previews/PR663/maths/nondiff_points-bd317c35.svg new file mode 100644 index 000000000..004bc8a6c --- /dev/null +++ b/previews/PR663/maths/nondiff_points-bd317c35.svg @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-c6064fd4.svg b/previews/PR663/maths/nondiff_points-c6064fd4.svg new file mode 100644 index 000000000..4a5c28057 --- /dev/null +++ b/previews/PR663/maths/nondiff_points-c6064fd4.svg @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-d1529e0d.svg b/previews/PR663/maths/nondiff_points-d1529e0d.svg new file mode 100644 index 000000000..b21acdbfa --- /dev/null +++ b/previews/PR663/maths/nondiff_points-d1529e0d.svg @@ -0,0 +1,41 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-f10bd981.svg b/previews/PR663/maths/nondiff_points-f10bd981.svg new file mode 100644 index 000000000..3c12383da --- /dev/null +++ b/previews/PR663/maths/nondiff_points-f10bd981.svg @@ -0,0 +1,39 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points-fea88105.svg b/previews/PR663/maths/nondiff_points-fea88105.svg new file mode 100644 index 000000000..00d72e7e2 --- /dev/null +++ b/previews/PR663/maths/nondiff_points-fea88105.svg @@ -0,0 +1,43 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/previews/PR663/maths/nondiff_points.html b/previews/PR663/maths/nondiff_points.html new file mode 100644 index 000000000..b90540976 --- /dev/null +++ b/previews/PR663/maths/nondiff_points.html @@ -0,0 +1,2 @@ + +Non-differentiable Points · ChainRules
GitHub

What to return for non-differentiable points

What is the short version?

If the function is not-differentiable choose to return something useful rather than erroring. For a branch a function is not differentiable due to e.g. a branch, like abs, your rule can reasonably claim the derivative at that point is the value from either branch, or any value in-between. In particular for local optima (like in the case of abs) claiming the derivative is 0 is a good idea. Similarly, if derivative is from one side is not defined, or is not finite, return the derivative from the other side. Throwing an error, or returning NaN is generally the least useful option.

However, contrary to what calculus says most autodiff systems will return an answer for such functions. For example for: abs_left(x) = (x <= 0) ? -x : x, AD will say the derivative at x=0 is -1. Alternatively for: abs_right(x) = (x < 0) ? -x : x, AD will say the derivative at x=0 is 1. Those two examples are weird since they are equal at all points, but AD claims different derivatives at x=0. The way to fix autodiff systems being weird is to write custom rules. So what rule should we write for this case?

The obvious answer, would be to write a rule that throws an error if input at a point where calculus says the derivative is not defined. Another option is to return some error signally value like NaN. Which you can do. However, there is no where to go with an error, the user still wants a derivative; so this is not useful.

Let us explore what is useful:

Case Studies

Derivative is defined in usual sense

plot(x->x^3)
Example block output

This is the standard case, one can return the derivative that is defined according to school room calculus. Here we would reasonably say that at x=0 the derivative is 3*0^2=0.

Local Minima / Maxima

plot(abs)
Example block output

abs is the classic example of a function where the derivative is not defined, as the limit from above is not equal to the limit from below.

\[\operatorname{abs}'(0) = \lim_{h \to 0^-} \dfrac{\operatorname{abs}(0)-\operatorname{abs}(0-h)}{0-h} = -1\]

\[\operatorname{abs}'(0) = \lim_{h \to 0^+} \dfrac{\operatorname{abs}(0)-\operatorname{abs}(0-h)}{0-h} = 1\]

Now, as discussed in the introduction, the AD system would on it's own choose either 1 or -1, depending on implementation.

We however have a potentially much nicer answer available to use: 0.

This has a number of advantages.

  • It follows the rule that derivatives are zero at local minima (and maxima).
  • If you leave a gradient descent optimizer running it will eventually actually converge absolutely to the point – where as with it being 1 or -1 it would never outright converge it would always flee.

Further:

  • It is a perfectly nice member of the subderivative.
  • It is the mean of the derivative on each side; which means that it will agree with central finite differencing at the point.

Piecewise slope change

plot(x-> x < 0 ? x : 5x)
Example block output

Here we have 3 main options, all are good.

We could say the derivative at 0 is:

  • 1: which agrees with backwards finite differencing
  • 5: which agrees with forwards finite differencing
  • 3: which is the mean of [1, 5], and agrees with central finite differencing

All of these options are perfectly nice members of the subderivative. 3 is the arguably the nicest, but it is also the most expensive to compute. In general all are acceptable.

Derivative zero almost everywhere

plot(ceil)
Example block output

Here it is most useful to say the derivative is zero everywhere. The limits are zero from both sides.

The other option for x->ceil(x) would be to relax the problem into x->x, and thus say it is 1 everywhere. But that it too weird, if the user wanted a relaxation of the problem then they would provide one. We can not be imposing that relaxation on to ceil, as it is not reasonable for everyone.

Not defined on one-side

plot(x->exp(2log(x)))
Example block output

We do not have to worry about what to return for the side where it is not defined. As we will never be asked for the derivative at e.g. x=-2.5 since the primal function errors. But we do need to worry about at the boundary – if that boundary point doesn't error.

Since we will never be asked about the left-hand side (as the primal errors), we can use just the right-hand side derivative. In this case giving 0.0.

Also nice in this case is that it agrees with the symbolic simplification of x->exp(2log(x)) into x->x^2.

Derivative nonfinite and same on both sides

plot(cbrt)
Example block output

Here we have no real choice but to say the derivative at 0 is Inf. We could consider as an alternative saying some large but finite value. However, if too large it will just overflow rapidly anyway; and if too small it will not dominate over finite terms. It is not possible to find a given value that is always large enough. Our alternatives would be to consider the derivative at nextfloat(0.0) or prevfloat(0.0). But this is more or less the same as choosing some large value – in this case an extremely large value that will rapidly overflow.

Derivative on-finite and different on both sides

plot(x-> sign(x) * cbrt(x))
Example block output

In this example, the primal is defined and finite, so we would like a derivative to be defined. We are back in the case of a local minimum like we were for abs. We can make most of the same arguments as we made there to justify saying the derivative is zero.

Conclusion

From the case studies a few general rules can be seen for how to choose a value that is useful. These rough rules are:

  • Say the derivative is 0 at local optima.
  • If the derivative from one side is defined and the other isn't, say it is the derivative taken from the defined side.
  • If the derivative from one side is finite and the other isn't, say it is the derivative taken from the finite side.
  • When derivative from each side is not equal, strongly consider reporting the average.

Our goal as always, is to get a pragmatically useful result for everyone, which must by necessity also avoid a pathological result for anyone.

diff --git a/previews/PR663/maths/propagators.html b/previews/PR663/maths/propagators.html new file mode 100644 index 000000000..060a0cd99 --- /dev/null +++ b/previews/PR663/maths/propagators.html @@ -0,0 +1,31 @@ + +The propagators: pushforward and pullback · ChainRules
GitHub

The propagators: pushforward and pullback

pushforward and pullback

Pushforward and pullback are fancy words that the autodiff community recently adopted from Differential Geometry. They are broadly in agreement with the use of pullback and pushforward in differential geometry. But any geometer will tell you these are the super-boring flat cases. Some will also frown at you. They are also sometimes described in terms of the jacobian: The pushforward is jacobian vector product (jvp), and pullback is jacobian transpose vector product (j'vp). Other terms that may be used include for pullback the backpropagator, and by analogy for pushforward the forwardpropagator, thus these are the propagators. These are also good names because effectively they propagate wiggles and wobbles through them, via the chain rule. (the term backpropagator may originate with "Lambda The Ultimate Backpropagator" by Pearlmutter and Siskind, 2008)

Core Idea

Less formally

  • The pushforward takes a wiggle in the input space, and tells what wobble you would create in the output space, by passing it through the function.
  • The pullback takes wobbliness information with respect to the function's output, and tells the equivalent wobbliness with respect to the functions input.

More formally

The pushforward of $f$ takes the sensitivity of the input of $f$ to a quantity, and gives the sensitivity of the output of $f$ to that quantity.

The pullback of $f$ takes the sensitivity of a quantity to the output of $f$, and gives the sensitivity of that quantity to the input of $f$.

Math

This is all a bit simplified by talking in 1D.

Lighter Math

For a chain of expressions:

a = f(x)
+b = g(a)
+c = h(b)

The pullback of g, which incorporates the knowledge of ∂b/∂a, applies the chain rule to go from ∂c/∂b to ∂c/∂a.

The pushforward of g, which also incorporates the knowledge of ∂b/∂a, applies the chain rule to go from ∂a/∂x to ∂b/∂x.

Geometric interpretation of reverse and forwards mode AD

Let us think of our types geometrically. In other words, elements of a type form a manifold. This document will explain this point of view in some detail.

Some terminology/conventions

Let $p$ be an element of type $M$, which is defined by some assignment of numbers $x_1, \dots, x_m$, say $(x_1, \dots, x_m) = (a_1, \dots, a_m)$

A function $f:M \to K$ on $M$ is (for simplicity) a polynomial $K[x_1, \dots, x_m]$

The tangent space $T_pM$ of $M$ at point $p$ is the $K$-vector space spanned by derivations $d/dx$. The tangent space acts linearly on the space of functions. They act as usual on functions. Our starting point is that we know how to write down $d/dx(f) = df/dx$.

The collection of tangent spaces ${T_pM}$ for $p\in M$ is called the tangent bundle of $M$.

Let $df$ denote the first order information of $f$ at each point. This is called the differential of $f$. If the derivatives of $f$ and $g$ agree at $p$, we say that $df$ and $dg$ represent the same cotangent at $p$. The covectors $dx_1, \dots, dx_m$ form the basis of the cotangent space $T^*_pM$ at $p$. Notice that this vector space is dual to $T_pM$.

The collection of cotangent spaces ${T^*_pM}$ for $p\in M$ is called the cotangent bundle of $M$.

Push-forwards and pullbacks

Let $N$ be another type, defined by numbers $y_1,...,y_n$, and let $g:M \to N$ be a map, that is, an $n$-dimensional vector $(g_1, ..., g_m)$ of functions on $M$.

We define the push-forward $g_*:TM \to TN$ between tangent bundles by $g_*(X)(h) = X(g\circ h)$ for any tangent vector $X$ and function $f$. We have $g_*(d/dx_i)(y_j) = dg_j/dx_i$, so the push-forward corresponds to the Jacobian, given a chosen basis.

Similarly, the pullback of the differential $df$ is defined by $g^*(df) = d(f\circ g)$. So for a coordinate differential $dy_j$, we have $g^*(dy_j) = d(g_j)$. Notice that this is a covector, and we could have defined the pullback by its action on vectors by $g^*(dh)(X) = g_*(X)(dh) = X(g\circ h)$ for any function $f$ on $N$ and $X\in TM$. In particular, $g^*(dy_j)(d/dx_i) = d(g_j)/dx_i$. If you work out the action in a basis of the cotangent space, you see that it acts by the adjoint of the Jacobian.

Notice that the pullback of a differential and the pushforward of a vector have a very different meaning, and this should be reflected on how they are used in code.

The information contained in the push-forward map is exactly what does my function do to tangent vectors. Pullbacks, acting on differentials of functions, act by taking the total derivative of a function. This works in a coordinate invariant way, and works without the notion of a metric. Gradients recall are vectors, yet they should contain the same information of the differential $df$. Assuming we use the standard euclidean metric, we can identify $df$ and $\nabla f$ as vectors. But pulling back gradients still should not be a thing.

If the goal is to evaluate the gradient of a function $f=g\circ h:M \to N \to K$, where $g$ is a map and $h$ is a function, we have two obvious options: First, we may push-forward a basis of $M$ to $TK$ which we identify with K itself. This results in $m$ scalars, representing components of the gradient. Step-by-step in coordinates:

  1. Compute the push-forward of the basis of $T_pM$, i.e. just the columns of the Jacobian $dg_i/dx_j$.
  2. Compute the push-forward of the function $h$ (consider it as a map, K is also a manifold!) to get $h_*(g_*T_pM) = \sum_j dh/dy_i (dg_i/dx_j)$

Second, we pull back the differential $dh$:

  1. compute $dh = dh/dy_1,...,dh/dy_n$ in coordinates.
  2. pull back by (in coordinates) multiplying with the adjoint of the Jacobian, resulting in $g_*(dh) = \sum_i(dg_i/dx_j)(dh/dy_i)$.

The anatomy of pullback and pushforward

For our function foo(args...; kwargs...) = y:

function pullback(Δy)
+    ...
+    return ∂self, ∂args...
+end

The input to the pullback is often called the seed. If the function is y = f(x) often the pullback will be written s̄elf, x̄ = pullback(ȳ).

Note

The pullback returns one ∂arg per arg to the original function, plus one ∂self for the fields of the function itself (explained below).

perturbation, seed, sensitivity

Sometimes perturbation, seed, and even sensitivity will be used interchangeably. They are not generally synonymous, and ChainRules shouldn't mix them up. One must be careful when reading literature. At the end of the day, they are all wiggles or wobbles.

The pushforward is a part of the frule function. Considered alone it would look like:

function pushforward(Δself, Δargs...)
+    ...
+    return ∂y
+end

But because it is fused into frule we see it as part of:

function frule((Δself, Δargs...), ::typeof(foo), args...; kwargs...)
+    ...
+    return y, ∂y
+end

The input to the pushforward is often called the perturbation. If the function is y = f(x) often the pushforward will be written ẏ = last(frule((ṡelf, ẋ), f, x)). ẏ is commonly used to represent the perturbation for y.

Note

In the frule/pushforward, there is one Δarg per arg to the original function. The Δargs are similar in type/structure to the corresponding inputs args (Δself is explained below). The ∂y are similar in type/structure to the original function's output Y. In particular if that function returned a tuple then ∂y will be a tuple of the same size.

Self derivative Δself, ∂self, s̄elf, ṡelf etc

Δself, ∂self, s̄elf, ṡelf

It is the derivatives with respect to the internal fields of the function. To the best of our knowledge there is no standard terminology for this. Other good names might be Δinternal/∂internal.

From the mathematical perspective, one may have been wondering what all this Δself, ∂self is. Given that a function with two inputs, say f(a, b), only has two partial derivatives: $\dfrac{∂f}{∂a}$, $\dfrac{∂f}{∂b}$. Why then does a pushforward take in this extra Δself, and why does a pullback return this extra ∂self?

The reason is that in Julia the function f may itself have internal fields. For example a closure has the fields it closes over; a callable object (i.e. a functor) like a Flux.Dense has the fields of that object.

Thus every function is treated as having the extra implicit argument self, which captures those fields. So every pushforward takes in an extra argument, which is ignored unless the original function has fields. It is common to write function foo_pushforward(_, Δargs...) in the case when foo does not have fields. Similarly every pullback returns an extra ∂self, which for things without fields is NoTangent(), indicating there are no fields within the function itself.

Here's an example showing how to define ∂self in an rrule when the primal function has internal fields (implicit arguments):

struct Multiplier{T}
+    x::T
+end
+(m::Multiplier)(y) = m.x * y
+
+function ChainRulesCore.rrule(m::Multiplier, y)
+    product = m(y)
+    function pullback(Δproduct) 
+        ∂self = Tangent{Multiplier}(; x = Δproduct * y')
+        ∂y = m.x' * Δproduct
+        return ∂self, ∂y
+    end 
+    return product, pullback
+end

Pushforward / Pullback summary

  • Pullback

    • returned by rrule
    • takes output space wobbles, gives input space wiggles
    • Argument structure matches structure of primal function output
    • If primal function returns a tuple, then pullback takes in a tuple of differentials.
    • 1 return per original function argument + 1 for the function itself

  • Pushforward:

    • part of frule
    • takes input space wiggles, gives output space wobbles
    • Argument structure matches primal function argument structure, but passed as a tuple at start of frule
    • 1 argument per original function argument + 1 for the function itself
    • 1 return per original function return

Pullback/Pushforward and Directional Derivative/Gradient

The most trivial use of the pushforward from within frule is to calculate the directional derivative:

If we would like to know the directional derivative of f for an input change of (1.5, 0.4, -1)

direction = (1.5, 0.4, -1) # (ȧ, ḃ, ċ)
+y, ẏ = frule((ZeroTangent(), direction...), f, a, b, c)

On the basis directions one gets the partial derivatives of y:

y, ∂y_∂a = frule((ZeroTangent(), 1, 0, 0), f, a, b, c)
+y, ∂y_∂b = frule((ZeroTangent(), 0, 1, 0), f, a, b, c)
+y, ∂y_∂c = frule((ZeroTangent(), 0, 0, 1), f, a, b, c)

Similarly, the most trivial use of rrule and returned pullback is to calculate the gradient:

y, f_pullback = rrule(f, a, b, c)
+∇f = f_pullback(1)  # for appropriate `1`-like seed.
+s̄elf, ā, b̄, c̄ = ∇f

Then we have that ∇f is the gradient of f at (a, b, c). And we thus have the partial derivatives $\overline{\mathrm{self}} = \dfrac{∂f}{∂\mathrm{self}}$, $\overline{a} = \dfrac{∂f}{∂a}$, $\overline{b} = \dfrac{∂f}{∂b}$, $\overline{c} = \dfrac{∂f}{∂c}$, including the self-partial derivative, $\overline{\mathrm{self}}$.

diff --git a/previews/PR663/rule_author/converting_zygoterules.html b/previews/PR663/rule_author/converting_zygoterules.html new file mode 100644 index 000000000..be3c14dac --- /dev/null +++ b/previews/PR663/rule_author/converting_zygoterules.html @@ -0,0 +1,13 @@ + +Converting ZygoteRules.@adjoint to rrules · ChainRules
GitHub

Converting ZygoteRules.@adjoint to rrules

ZygoteRules.jl is a legacy package similar to ChainRulesCore but supporting Zygote.jl only.

If you have some rules written with ZygoteRules it is a good idea to upgrade them to use ChainRules instead. Zygote will still be able to use them, but so will other AD systems, and you will get access to some more advanced features. Some of these features are currently ignored by Zygote, but could be supported in the future.

Example

Consider the function

struct Foo
+    a::Float64,
+    b::Float64
+end
+
+f(x, y::Foo, z) = 2*x + y.a

ZygoteRules

@adjoint function f(x, y::Foo, z)
+    f_pullback(Ω̄) = (2Ω̄, NamedTuple(;a=Ω̄, b=nothing), nothing)
+    return f(x, y, z), f_pullback
+end

ChainRules

function rrule(::typeof(f), x, y::Foo, z)
+    f_pullback(Ω̄) = (NoTangent(), 2Ω̄, Tangent{Foo}(;a=Ω̄), ZeroTangent())
+    return f(x, y, z), f_pullback
+end

Write as a rrule(::typeof(f), ...)

No magic macro here, rrule is the function that it is. The function it is the rule for is the first argument, or second argument if you need to take a RuleConfig.

Note that when writing the rule for constructor you will need to use ::Type{Foo}, not typeof(Foo). See docs on Constructors.

Include the derivative with respect to the function object itself

The ZygoteRules.@adjoint macro automagically[1] inserts an extra nothing in the return for the function it generates to represent the derivative of output with respect to the function object. ChainRules as a philosophy avoids magic as much as possible, and thus require you to return it explicitly. If it is a plain function (like typeof(sin)), then the tangent will be NoTangent.

Tangent Type changes

ChainRules uses tangent types that must represent vector spaces (i.e. tangent spaces). They need to have things like + defined on them. ZygoteRules takes a more adhoc approach to this.

nothing becomes an AbstractZero

ZygoteRules uses nothing to represent some sense of zero, in a primal type agnostic way. There are many senses of zero. ChainRules represents two of them, as subtypes of AbstractZero.

ZeroTangent for the case that there is no relationship between the primal output and the primal input. NoTangent for the case where conceptually the tangent space doesn't exist. e.g. what is the Tangent to a String or an index: those can't be perturbed.

See FAQ on the difference between ZeroTangent and NoTangent. At the end of the day it doesn't matter too much if you get them wrong. NoTangent and ZeroTangent more or less act identically.

Tuples and NamedTuples become Tangent{T}s

Zygote uses Tuples and NamedTuples to represent the structural tangents for Tuples and structs respectively. ChainRules core provides a generic Tangent{T} to represent the structural tangent of a primal type T. It takes positional arguments if representing tangent for a Tuple. Or keyword argument to represent the tangent for a struct or a NamedTuple. When representing a struct you only need to list the nonzero fields – any not given are implicit considered to be ZeroTangent.

When we say structural tangent we mean tangent types that are based only on the structure of the primal. This is in contrast to a natural tangent which captures some knowledge based on what the primal type represents. (E.g. for arrays a natural tangent is often the same kind of array). For more details see the the design docs on the many tangent types

Calling back into AD (ZygoteRules.pullback)

Rules that need to call back into the AD system, e.g, for higher order functions like map(f, xs), need to be changed. In ZygoteRules you can use ZygoteRules.pullback or ZygoteRules._pullback, which will always result in calling into Zygote. Since ChainRules is AD agnostic, you can't do that. Instead you use a RuleConfig to specify requirements of an AD system e.g ::RuleConfig{>:HasReverseMode} work for Zygote, and then use rrule_via_ad.

See the docs on calling back into AD for more details.

Consider adding some thunks

A feature ChainRulesCore offers that ZygoteRules doesn't is support for thunks. Thunks delay work until it is needed, and avoid it if it never is. See docs on @thunk, Thunk, InplaceableThunk.

You don't have to use thunks, though. It is easy to go overboard with using thunks.

Testing Changes

One of the advantages of using ChainRules is that you can easily and robustly test your rules with ChainRulesTestUtils.jl. This uses finite differencing to test the accuracy of derivative, as well as checks the correctness of the API. It should catch anything you might have gotten wrong referred to in this page.

The test for the above example is test_rrule(f, 2.5, Foo(9.9, 7.2), 31.0). You can see it looks a lot like an example call to rrule, just with the prefix test_ added to the start.

@nograd becomes @non_differentiable

Probably more or less with no changes. @non_differentiable also lets you specify a signature in case you want to restrict non-differentiability to a certain subset of argument types.

No such thing as literal_getproperty

That is just getproperty, it takes Symbol. It should constant-fold. It likely doesn't though as Zygote doesn't play nice with the optimizer.

Take embedded spaces and types seriously

Traditionally Zygote has taken a very laissez-faire attitude towards types and mathematical spaces. Sometimes treating Reals as embedded in the Complex plane; sometimes not. Sometimes treating sparse and structuredly-sparse matrix as embedded in the space of dense matrices. Writing rules that apply to any Array{T} which perhaps are only applicable for Array{<:Real} and not so much for Array{Quaternion}. Traditionally ChainRules takes a much more considered approach.

See for example our docs on how to handle complex numbers correctly. (The outcome of several long long long discussions with a number of experts in our community)

Now, I am not here to tell you what to do in your package, but this is a good time to reconsider how seriously you take these things in the rules you are converting.

What if I miss something

It is not great, but it probably OK. Zygote's ChainRules interface is fairly forgiving. Other AD systems may not be. If you test with ChainRulesTestUtils.jl then you can be confident that you didn't miss anything.

  • 1unless you write it in functor form (i.e. @adjoint (f::MyType)(args...)=...), in that case like for rrule you need to include it explicitly.
diff --git a/previews/PR663/rule_author/debug_mode.html b/previews/PR663/rule_author/debug_mode.html new file mode 100644 index 000000000..ada173c63 --- /dev/null +++ b/previews/PR663/rule_author/debug_mode.html @@ -0,0 +1,2 @@ + +Debug mode · ChainRules
GitHub

Debug Mode

ChainRulesCore supports a debug mode which you can use while writing new rules. It provides better error messages. If you are developing some new rules, and you get a weird error message, it is worth enabling debug mode.

There is some overhead to having it enabled, so it is disabled by default.

To enable, redefine the ChainRulesCore.debug_mode function to return true.

ChainRulesCore.debug_mode() = true

Features of Debug Mode:

  • If you add a Tangent to a primal value, and it was unable to construct a new primal values, then a better error message will be displayed detailing what overloads need to be written to fix this.
  • during add!!, if an InplaceThunk is used, and it runs the code that is supposed to run in place, but the return result is not the input (with updated values), then an error is thrown. Rather than silently using what ever values were returned.
diff --git a/previews/PR663/rule_author/example.html b/previews/PR663/rule_author/example.html new file mode 100644 index 000000000..2fa8fbaba --- /dev/null +++ b/previews/PR663/rule_author/example.html @@ -0,0 +1,26 @@ + +Pedagogical example · ChainRules
GitHub

Pedagogical Example

This pedagogical example will show you how to write an rrule. See On writing good rrule / frule methods section for more tips and gotchas. If you want to learn about frules, you should still read and understand this example as many concepts are shared, and then look for real world frule examples in ChainRules.jl.

The primal

We define a struct Foo

struct Foo{T}
+    A::Matrix{T}
+    c::Float64
+end

and a function that multiplies Foo with an AbstractArray:

function foo_mul(foo::Foo, b::AbstractArray)
+    return foo.A * b
+end

Note that field c is ignored in the calculation.

The rrule

The rrule method for our primal computation should extend the ChainRulesCore.rrule function.

function ChainRulesCore.rrule(::typeof(foo_mul), foo::Foo{T}, b::AbstractArray) where T
+    y = foo_mul(foo, b)
+    function foo_mul_pullback(ȳ)
+        f̄ = NoTangent()
+        f̄oo = Tangent{Foo{T}}(; A=ȳ * b', c=ZeroTangent())
+        b̄ = @thunk(foo.A' * ȳ)
+        return f̄, f̄oo, b̄
+    end
+    return y, foo_mul_pullback
+end

We can check this rule against a finite-differences approach using ChainRulesTestUtils:

julia> using ChainRulesTestUtils
+julia> test_rrule(foo_mul, Foo(rand(3, 3), 3.0), rand(3, 3))
+Test Summary:                                       | Pass  Total
+test_rrule: foo_mul on Foo{Float64},Matrix{Float64} |   10     10
+Test.DefaultTestSet("test_rrule: foo_mul on Foo{Float64},Matrix{Float64}", Any[], 10, false, false)

Now let's examine the rule in more detail:

function ChainRulesCore.rrule(::typeof(foo_mul), foo::Foo, b::AbstractArray)
+    ...
+    return y, foo_mul_pullback
+end

The rrule dispatches on the typeof of the function we are writing the rrule for, as well as the types of its arguments. Read more about writing rules for constructors and callable objects here. The rrule returns the primal result y, and the pullback function. It is a very good idea to name your pullback function, so that they are helpful when appearing in the stacktrace.

y = foo_mul(foo, b)

Computes the primal result. It is possible to change the primal computation so that work can be shared between the primal and the pullback. See e.g. the rule for sort, where the sorting is done only once.

function foo_mul_pullback(ȳ)
+    ...
+    return f̄, f̄oo, b̄
+end

The pullback function takes in the tangent of the primal output (ȳ) and returns the tangents of the primal inputs. Note that it returns a tangent for the primal function in addition to the tangents of primal arguments.

Finally, computing the tangents of primal inputs:

f̄ = NoTangent()

The function foo_mul has no fields (i.e. it is not a closure) and can not be perturbed. Therefore its tangent (f̄) is a NoTangent.

f̄oo = Tangent{Foo}(; A=ȳ * b', c=ZeroTangent())

The struct foo::Foo gets a Tangent{Foo} structural tangent, which stores the tangents of fields of foo.

The tangent of the field A is ȳ * b',

The tangent of the field c is ZeroTangent(), because c can be perturbed but has no effect on the primal output.

b̄ = @thunk(foo.A' * ȳ)

The tangent of b is foo.A' * ȳ, but we have wrapped it into a Thunk, a tangent type that represents delayed computation. The idea is that in case the tangent is not used anywhere, the computation never happens. Use InplaceableThunk if you are interested in accumulating gradients in-place. Note that in practice one would also @thunk the f̄oo.A tangent, but it was omitted in this example for clarity.

As a final note, since b is an AbstractArray, its tangent b̄ should be projected to the right subspace. See the ProjectTo the primal subspace section for more information and an example that motivates the projection operation.

diff --git a/previews/PR663/rule_author/intro.html b/previews/PR663/rule_author/intro.html new file mode 100644 index 000000000..c94987231 --- /dev/null +++ b/previews/PR663/rule_author/intro.html @@ -0,0 +1,2 @@ + +Introduction · ChainRules
GitHub

Introduction

This section of the docs will tell you everything you need to know about writing rules for your package.

It will help with understanding tangent types, the anatomy of the frule and the rrule, and provide tips on writing good rules, as well as how to test them easily using finite differences.

This section also outlines some ChainRules superpowers that can be considered advanced usage. Most users can ignore these. However:

  • If you are writing rules with abstractly typed arguments, read about ProjectTo.
  • If you want to opt out of using the abstractly typed rule for certain argument types, read about @opt_out.
  • If you are writing rules for higher order functions, read about calling back into AD.
  • If you want to accumulate gradients in-place to avoid extra allocations, read about gradient accumulation.
diff --git a/previews/PR663/rule_author/rule_definition_tools.html b/previews/PR663/rule_author/rule_definition_tools.html new file mode 100644 index 000000000..e85922a82 --- /dev/null +++ b/previews/PR663/rule_author/rule_definition_tools.html @@ -0,0 +1,9 @@ + +Rule definition tools · ChainRules
GitHub

Using rule definition tools

Rule definition tools can help you write more frules and the rrules with less lines of code.

@non_differentiable

For non-differentiable functions the @non_differentiable macro can be used. For example, instead of manually defining the frule and the rrule for string concatenation *(String..), the macro call

@non_differentiable *(String...)

defines the following frule and rrule automatically

function ChainRulesCore.frule(var"##_#1600", ::Core.Typeof(*), String::Any...; kwargs...)
+    return (*(String...; kwargs...), NoTangent())
+end
+function ChainRulesCore.rrule(::Core.Typeof(*), String::Any...; kwargs...)
+    return (*(String...; kwargs...), function var"*_pullback"(_)
+        (ZeroTangent(), ntuple((_->NoTangent()), 0 + length(String))...)
+    end)
+end

Note that the types of arguments are propagated to the frule and rrule definitions. This is needed in case the function differentiable for some but not for other types of arguments. For example *(1, 2, 3) is differentiable, and is not defined with the macro call above.

@scalar_rule

For functions involving only scalars, i.e. subtypes of Number (no structs, Strings...), both the frule and the rrule can be defined using a single @scalar_rule macro call.

Note that the function does not have to be $\mathbb{R} \rightarrow \mathbb{R}$. In fact, any number of scalar arguments is supported, as is returning a tuple of scalars.

See docstrings for the comprehensive usage instructions.

diff --git a/previews/PR663/rule_author/superpowers/gradient_accumulation.html b/previews/PR663/rule_author/superpowers/gradient_accumulation.html new file mode 100644 index 000000000..92dd478d1 --- /dev/null +++ b/previews/PR663/rule_author/superpowers/gradient_accumulation.html @@ -0,0 +1,18 @@ + +Gradient accumulation · ChainRules
GitHub

Gradient Accumulation

Consider some function $f(x) = g(x) + h(x)$. If we would like the derivative of $f$ with respect to $x$ we must compute it for each part and then sum them, i.e. $\frac{\partial f}{\partial x} = \frac{\partial g}{\partial x} + \frac{\partial h}{\partial x}$. In general, we must accumulate (sum) gradients from each sub-part of a program where a variable is used.

Consider for example:

function sum_first_and_second(X::Array{Float64})
+    a = X[1]
+    b = X[2]
+    y = a + b
+    return y
+end

The AD software must transform that into something which repeatedly sums up the gradient of each part: X̄ = ā + b̄.

This requires that all tangent types D must implement +: +(::D, ::D)::D.

We can note that in this particular case ā and b̄ will both be arrays. This operation (X̄ = ā + b̄) will allocate one array to hold ā, another one to hold b̄, and a third one to hold ā + b̄. This is three allocations. Allocations are not free, they increase the time the program takes to run by a nontrivial amount, even with a good allocator and a good garbage collector.

Maybe-mutating accumulation (add!!)

We can note that in the above that neither ā nor b̄ are ever used again after accumulating to get X̄. Furthermore, Arrays are mutable. That means we could over-write either ā or b̄ and use the result as X̄:

ā .+= b̄
+X̄ = ā

This cuts our allocations down to 2, just ā and b̄.

However, we have a bit of a problem that not all types are mutable, so this pattern is hard to apply in general. To deal with that ChainRulesCore provides add!!. Per the BangBang.jl convention, this is a maybe mutating addition. It may mutate its first argument (if it is mutable), but it will definitely return the correct result. We would write using that as X̄ = add!!(ā, b̄): which would in this case give us just 2 allocations. AD systems can generate add!! instead of + when accumulating gradient to take advantage of this.

Inplaceable Thunks (InplaceableThunks) avoid allocating values in the first place.

We got down to two allocations from using add!!, but can we do better? We can think of having a tangent type which acts on a partially accumulated result, to mutate it to contain its current value plus the partial derivative being accumulated. Rather than having an actual computed value, we can just have a thing that will act on a value to perform the addition. Let's illustrate it with our example.

b̄ is the partial for X[2] and its value can be computed by:

b̄ = zeros(size(X))
+b̄[2] = ȳ  # the scalar sensitivity of the `sum_first_and_second` output

b̄ is a matrix entirely of zeros, except for at the index 2, where it is set to the output sensitivity ȳ. ā is similar, except with the non-zero at index 1.

What is the action of b̄ upon ā, to get the same result as X̄ = add!!(ā, b̄) (or X̄ = ā + b̄ for that matter)? It is:

function b̄_add!(ā)
+    ā[2] += ȳ
+    return ā
+end

We don't need to worry about all those zeros since x + 0 == x.

InplaceableThunk is the type we have to represent derivatives as gradient accumulating actions. We must note that to do this we do need a value form of ā for b̄ to act upon. For this reason every inplaceable thunk has both a val field holding the value representation, and a add! field holding the action representation. The val field use a plain Thunk to avoid the computation (and thus allocation) if it is unused.

Do we need both representations?

Right now every InplaceableThunk has two fields that need to be specified. The value form (represented as a the Thunk typed field), and the action form (represented as the add! field). It is possible in a future version of ChainRulesCore.jl we will work out a clever way to find the zero tangent for arbitrary primal values. Given that, we could always just determine the value form from inplaceable.add!(zero_tangent(primal)). There are some technical difficulties in finding the zero tangents, but this may be solved at some point.

The + operation on InplaceableThunks is overloaded to unthunk that val field to get the value form. Where as the add!! operation is overloaded to call add! to invoke the action.

With getindex defined to return an InplaceableThunk, we now get to X̄ = add!!(ā, b̄) requires only a single allocation. This allocation occurs when unthunking ā, which is then mutated to become X̄. This is basically as good as we can get: if we want X̄ to be an Array then at some point we need to allocate that array.

Can we do more? Deferred accumulation

We could keep going further to drop allocations if we really wanted. If we didn't care about X̄ being an Array then we could defer its computation too. X̄ = @thunk add!!(ā, b̄). This kind of deferral will work fine and you can keep chaining it. It does start to burn stack space, and might make the compiler's optimization passes cry. But it's valid and should work fine.

Examples of InplaceableThunks

getindex

The aforementioned getindex is really the poster child for this. Consider something like:

function mysum(X::Array{Float64})
+    total = 0.0
+    for i in eachindex(X)
+        total += X[i]
+    end
+    return total
+end

If one only has value representation of derivatives one ends up having to allocate a derivative array for every single element of the original array X. That's terrible. On the other hand, with the action representation that InplaceableThunks provide, there is just a single Array allocated. One can see the getindex rule in ChainRules.jl for the implementation.

matmul etc (*)

Multiplication of scalars/vectors/matrices of compatible dimensions can all also have their derivatives represented as an InplaceableThunk. These tend to pivot around that add! action being defined along the lines of: X̄ -> mul!(X̄, A', Ȳ, true, true). Where 5-arg mul! is the in place multiply-add operation. mul!(X̄, A', Ȳ, true, true) has the same effect as (X̄ .+= A'*Ȳ) but avoids allocating the matrix A'*Ȳ This is one of the fundamental operations provided by BLAS – including the application of the conjugate transpose. e.g. the Matrix-Matrix form is GEMM (GEneralized Matrix-Matrix Multiplication), the Matrix-Vector form is GEMV (GEneralized Matrix-Vector Multiplication) etc. Under the hood doing it out of place is going to call one of these methods anyway, but on a freshly allocated output array. So we are going to hit a very efficient implementation and get the addition for free.

One can see the * rules in ChainRules.jl for the implementations

diff --git a/previews/PR663/rule_author/superpowers/mutation_support.html b/previews/PR663/rule_author/superpowers/mutation_support.html new file mode 100644 index 000000000..d7f9d4e8f --- /dev/null +++ b/previews/PR663/rule_author/superpowers/mutation_support.html @@ -0,0 +1,17 @@ + +Mutation Support (experimental) · ChainRules
GitHub

Mutation Support

ChainRulesCore.jl offers experimental support for mutation, targeting use in forward mode AD. (Mutation support in reverse mode AD is more complicated and will likely require more changes to the interface)

Experimental

This page documents an experimental feature. Expect breaking changes in minor versions while this remains. It is not suitable for general use unless you are prepared to modify how you are using it each minor release. It is thus suggested that if you are using it to use tilde bounds on supported minor versions.

MutableTangent

The MutableTangent type is designed to be a partner to the Tangent type, with specific support for being mutated in place. It is required to be a structural tangent, having one tangent for each field of the primal object.

Technically, not all mutable structs need to use MutableTangent to represent their tangents. Just like not all structs need to use Tangents. Common examples away from this are natural tangent types like for arrays. However, if one is setting up to use a custom tangent type for this it is sufficiently off the beaten path that we can not provide much guidance.

zero_tangent

The zero_tangent function functions to give you a zero (i.e. additive identity) for any primal value. The ZeroTangent type also does this. The difference is that zero_tangent is in general full structural tangent mirroring the structure of the primal. To be technical the promise of zero_tangent is that it will be a value that supports mutation. However, in practice[1] this is achieved through in a structural tangent For mutation support this is important, since it means that there is mutable memory available in the tangent to be mutated when the primal changes. To support this you thus need to make sure your zeros are created in various places with zero_tangent rather than []ZeroTangent](@ref).

It is also useful for reasons of type stability, since it forces a consistent type (generally a structural tangent) for any given primal type. For this reason AD system implementors might chose to use this to create the tangent for all literal values they encounter, mutable or not, and to process the output of frules to convert ZeroTangent into corresponding zero_tangents.

Writing a frule for a mutating function

It is relatively straight forward to write a frule for a mutating function. There are a few key points to follow:

  • There must be a mutable tangent input for every mutated primal input
  • When the primal value is changed, the corresponding change must be made to its tangent partner
  • When a value is returned, return its partnered tangent.
  • If (and only if) primal values alias, then their tangents must also alias.

Example

For example, consider the primal function with:

  1. takes two Refs
  2. doubles the first one in place
  3. overwrites the second one's value with the literal 5.0
  4. returns the first one
function foo!(a::Base.RefValue, b::Base.RefValue)
+    a[] *= 2
+    b[] = 5.0
+    return a
+end

The frule for this would be:

function ChainRulesCore.frule((_, ȧ, ḃ), ::typeof(foo!), a::Base.RefValue, b::Base.RefValue)
+    @assert ȧ isa MutableTangent{typeof(a)}
+    @assert ḃ isa MutableTangent{typeof(b)}
+
+    a[] *= 2
+    ȧ.x *= 2  # `.x` is the field that lives behind RefValues
+
+    b[] = 5.0
+    ḃ.x = zero_tangent(5.0)  # or since we know that the zero for a Float64 is zero could write `ḃ.x = 0.0`
+
+    return a, ȧ
+end

Then assuming the AD system does its part to makes sure you are indeed given mutable values to mutate (i.e. those @assertions are true) then all is well and this rule will make mutation correct.

  • 1Further, it is hard to achieve this promise of allowing mutation to be supported without returning a structural tangent. Except in the special case of where the struct is not mutable and has no nested fields that are mutable.
diff --git a/previews/PR663/rule_author/superpowers/opt_out.html b/previews/PR663/rule_author/superpowers/opt_out.html new file mode 100644 index 000000000..1ba2cdddd --- /dev/null +++ b/previews/PR663/rule_author/superpowers/opt_out.html @@ -0,0 +1,10 @@ + +@opt_out · ChainRules
GitHub

Opting out of rules

It is common to define rules fairly generically. Often matching (or exceeding) how generic the matching original primal method is. Sometimes this is not the correct behaviour. Sometimes the AD can do better than this human defined rule. If this is generally the case, then we should not have the rule defined at all. But if it is only the case for a particular set of types, then we want to opt-out just that one. This is done with the @opt_out macro.

Consider one a rrule for sum (the following simplified from the one in ChainRules.jl itself)

function rrule(::typeof(sum), x::AbstractArray{<:Number})
+    y = sum(x; dims=dims)
+    project = ProjectTo(x)
+    function sum_pullback(ȳ)
+        x̄ = project(fill(ȳ, size(x)))
+        return (NoTangent(), x̄)
+    end
+    return y, sum_pullback
+end

That is a fairly reasonable rrule for the vast majority of cases. You might have a custom array type for which you could write a faster rule. In which case you would do that, by writing a faster, more specific, rrule. But sometimes, it is the case that ADing the (faster, more specific) primal for your custom array type would yeild the faster pullback without you having to write a rrule by hand.

Consider a summing SkewSymmetric (anti-symmetric) matrix. The skew symmetric matrix has structural zeros on the diagonal, and off-diagonals are paired with their negation. Thus the sum is always going to be zero. As such the author of that matrix type would probably have overloaded sum(x::SkewSymmetric{T}) where T = zero(T). ADing this would result in the tangent computed for x as ZeroTangent() and it would be very fast since AD can see that x is never used in the right-hand side. In contrast the generic method for AbstractArray defined above would have to allocate the fill array, and then compute the skew projection. Only to find out the output would be projected to SkewSymmetric(zeros(T)) anyway (slower, and a less useful type).

To opt-out of using the generic rrule and to allow the AD system to do its own thing we use the @opt_out macro, to say to not use it for sum of SkewSymmetric.

@opt_out rrule(::typeof(sum), ::SkewSymmetric)

Perhaps we might not want to ever use rules for SkewSymmetric, because we have determined that it is always better to leave it to the AD, unless a very specific rule has been written[1]. We could then opt-out for all 1 arg functions.

@opt_out rrule(::Any, ::SkewSymmetric)

Though this is likely to cause some method-ambiguities, if we do it for more, but we can resolve those.

Similar can be done @opt_out frule. It can also be done passing in a RuleConfig.

If the general rule uses a config, the opt-out must also

Following the same principles as for rules with config, a rule with a RuleConfig argument will take precedence over one without, including if that one is a opt-out rule. But if the general rule does not use a config, then the opt-out rule can use a config. This allows, for example, you to use opt-out to avoid a particular AD system using a opt-out rule that takes that particular AD's config.

  • 1seems unlikely, but it is possible, there is a lot of structure that can be taken advantage of for some matrix types.
diff --git a/previews/PR663/rule_author/superpowers/projectto.html b/previews/PR663/rule_author/superpowers/projectto.html new file mode 100644 index 000000000..80292d908 --- /dev/null +++ b/previews/PR663/rule_author/superpowers/projectto.html @@ -0,0 +1,18 @@ + +ProjectTo · ChainRules
GitHub

ProjectTo the primal subspace

Rules with abstractly-typed arguments may return incorrect answers when called with certain concrete types. A classic example is the matrix-matrix multiplication rule, a naive definition of which follows:

function rrule(::typeof(*), A::AbstractMatrix, B::AbstractMatrix)
+    function times_pullback(ȳ)
+        dA = ȳ * B'
+        dB = A' * ȳ
+        return NoTangent(), dA, dB
+    end
+    return A * B, times_pullback
+end

When computing *(A, B), where A isa Diagonal and B isa Matrix, the output will be a Matrix. As a result, ȳ in the pullback will be a Matrix, and consequently dA for a A isa Diagonal will be a Matrix, which is wrong. Not only is it the wrong type, but it can contain non-zeros off the diagonal, which is not possible, it is outside of the subspace. While a specialised rules can indeed be written for the Diagonal case, there are many other types and we don't want to be forced to write a rule for each of them. Instead, project_A = ProjectTo(A) can be used (outside the pullback) to extract an object that knows how to project onto the type of A (e.g. also knows the size of the array). This object can be called with a tangent ȳ * B', by doing project_A(ȳ * B'), to project it on the tangent space of A. The correct rule then looks like

function rrule(::typeof(*), A::AbstractMatrix, B::AbstractMatrix)
+    project_A = ProjectTo(A)
+    project_B = ProjectTo(B)
+    function times_pullback(ȳ)
+        dA = ȳ * B'
+        dB = A' * ȳ
+        return NoTangent(), project_A(dA), project_B(dB)
+    end
+    return A * B, times_pullback
+end
It is often good to `@thunk` your projections

The above example is potentially a good place for using a @thunk. This is not required, but can in some cases be more computationally efficient, see Use Thunks appropriately. When combining thunks and projections, @thunk() must be the outermost call.

A more optimized implementation of the matrix-matrix multiplication example would have

times_pullback(ȳ) = NoTangent(), @thunk(project_A(ȳ * B')), @thunk(project_B(A' * ȳ))

within the rrule. This defers both the evaluation of the product rule and the projection until(/if) the tangent gets used.

diff --git a/previews/PR663/rule_author/superpowers/ruleconfig.html b/previews/PR663/rule_author/superpowers/ruleconfig.html new file mode 100644 index 000000000..3732b8d25 --- /dev/null +++ b/previews/PR663/rule_author/superpowers/ruleconfig.html @@ -0,0 +1,18 @@ + +RuleConfig · ChainRules
GitHub

Rule configurations and calling back into AD

RuleConfig is a method for making rules conditionally defined based on the presence of certain features in the AD system. One key such feature is the ability to perform AD either in forwards or reverse mode or both.

This is done with a trait-like system (not Holy Traits), where the RuleConfig has a union of types as its only type-parameter. Where each type represents a particular special feature of this AD. To indicate that the AD system has a special property, its RuleConfig should be defined as:

struct MyADRuleConfig <: RuleConfig{Union{Feature1, Feature2}} end

And rules that should only be defined when an AD has a particular special property write:

rrule(::RuleConfig{>:Feature1}, f, args...) = # rrule that should only be define for ADs with `Feature1`
+
+frule(::RuleConfig{>:Union{Feature1,Feature2}}, f, args...) = # frule that should only be define for ADs with both `Feature1` and `Feature2`
Rules with Config always take precedence over rules without

Even if the other arguments are more specific the rule with the config will always take precedence. For example of there is a rule rrule(::RuleConfig, ::typeof(foo), ::Any) and other rrule(foo, ::Float64), the first will always be selected. This is because the AD will always attempt to provide its config when checking for a rule, and only if that doesn't match, will the config-less rule be tried. In practice this doesn't happen often, but when it does the solution is a little ugly – though very similar to resolving method ambiguities. You need to manually add methods that dispatch from a rule with config to the one without. See for example the rule for sum(abs2, xs) in ChainRules.jl.

A prominent use of this is in declaring that the AD system can, or cannot support being called from within the rule definitions.

Writing rules that call back into AD

To define e.g. rules for higher order functions, it is useful to be able to call back into the AD system to get it to do some work for you.

For example the rule for reverse mode AD for map might like to use forward mode AD if one is available. Particularly for the case where only a single input collection is being mapped over. In that case we know the most efficient way to compute that sub-program is in forwards, as each call with-in the map only takes a single input.

Note: the following is not the most efficient rule for map via forward, but attempts to be clearer for demonstration purposes.

function rrule(config::RuleConfig{>:HasForwardsMode}, ::typeof(map), f::Function, x::Array{<:Real})
+    # real code would support functors/closures, but in interest of keeping example short we exclude it:
+    @assert (fieldcount(typeof(f)) == 0) "Functors/Closures are not supported"
+
+    y_and_ẏ = map(x) do xi
+        frule_via_ad(config, (NoTangent(), one(xi)), f, xi)
+    end
+    y = first.(y_and_ẏ)
+    ẏ = last.(y_and_ẏ)
+
+    pullback_map(ȳ) = NoTangent(), NoTangent(), ȳ .* ẏ
+    return y, pullback_map
+end

Writing rules that depend on other special requirements of the AD.

The >:HasReverseMode and >:HasForwardsMode are two examples of special properties that a RuleConfig could allow. Others could also exist, but right now they are the only two. It is likely that in the future other such will be provided

Note: you can only depend on the presence of a feature, not its absence. This means we may need to define features and their complements, when one is not the obvious default (as in the case of HasReverseMode/NoReverseMode and HasForwardsMode/NoForwardsMode.).

Such special properties generally should only be defined in ChainRulesCore. (Theoretically, they could be defined elsewhere, but the AD and the package containing the rule need to load them, and ChainRulesCore is the place for things like that.)

Writing rules that are only for your own AD

A special case of the above is writing rules that are defined only for your own AD. Rules which otherwise would be type-piracy, and would affect other AD systems. This could be done via making up a special property type and dispatching on it. But there is no need, as we can dispatch on the RuleConfig subtype directly.

For example in order to avoid mutation in nested AD situations, Zygote might want to have a rule for add!! that makes it just do +.

struct ZygoteConfig <: RuleConfig{Union{}} end
+
+rrule(::ZygoteConfig, typeof(ChainRulesCore.add!!), a, b) = a+b, Δ->(NoTangent(), Δ, Δ)

As an alternative to rules only for one AD, would be to add new special property definitions to ChainRulesCore (as described above) which would capture what makes that AD special.

diff --git a/previews/PR663/rule_author/tangents.html b/previews/PR663/rule_author/tangents.html new file mode 100644 index 000000000..e04113efc --- /dev/null +++ b/previews/PR663/rule_author/tangents.html @@ -0,0 +1,2 @@ + +Tangent types · ChainRules
GitHub

Tangent types

The values that come back from pullbacks or pushforwards are not always the same type as the input/outputs of the primal function. They are tangents, which correspond roughly to something able to represent the difference between two values of the primal types. A tangent might be such a regular type, like a Number, or a Matrix, matching to the original type; or it might be one of the AbstractTangent subtypes.

Tangents support a number of operations. Most importantly: + and *, which let them act as mathematical objects.

To be more formal they support operations which let them act as a vector space.

Operations on a tangent type

Any tangent type must support:

  • zero which returns an additive identity for that type (though it can just return ZeroTangent() (see below))
  • + for addition between two tangents of this primal, returning another tangent of this primal. This allows gradient accumulation.
  • * for multiplication (scaling) by a scalar.
  • + between a tangent and its primal type returning another tangent of the primal type – differential geometers sometimes call this exponential map.

Further they often support other linear operators for convenience in writing rules.

The subtypes of AbstractTangent

Not all tangents need to subtype the AbstractTangent type – in fact most don't: most are just numbers or arrays – but ChainRulesCore does provide a number of special tangent types that can be very useful.

  • ZeroTangent: It is a special representation of 0. It does great things around avoiding expanding Thunks in addition.
  • NoTangent: Zero-like, represents that the operation on this input is not differentiable. Its primal type is normally Integer or Bool.
  • Tangent{P}: this is the tangent for tuples and structs. Use it like a Tuple or NamedTuple. The type parameter P is for the primal type.
  • Thunk: this is a deferred computation. A thunk is a word for a zero argument closure. A computation wrapped in a @thunk doesn't get evaluated until unthunk is called on the thunk. unthunk is a no-op on non-thunked inputs.
  • InplaceableThunk: it is like a Thunk but it can do in-place add! which allows for avoiding allocation during gradient accumulation.
diff --git a/previews/PR663/rule_author/testing.html b/previews/PR663/rule_author/testing.html new file mode 100644 index 000000000..c5c65d28e --- /dev/null +++ b/previews/PR663/rule_author/testing.html @@ -0,0 +1,2 @@ + +Testing your rules · ChainRules
GitHub

Testing your rules

ChainRulesTestUtils.jl provides tools for writing tests based on FiniteDifferences.jl. Take a look at the documentation or the existing ChainRules.jl tests to see how to write the tests.

Warning

Don't use analytical derivations for derivatives in the tests. Those are what you use to define the rules, and so cannot be confidently used in the test. If you misread/misunderstood them, then your tests/implementation will have the same mistake. Use finite differencing methods instead, as they are based on the primal computation.

diff --git a/previews/PR663/rule_author/tips_for_packages.html b/previews/PR663/rule_author/tips_for_packages.html new file mode 100644 index 000000000..eec08bc09 --- /dev/null +++ b/previews/PR663/rule_author/tips_for_packages.html @@ -0,0 +1,17 @@ + +Tips for making your package work with AD · ChainRules
GitHub

Tips for making your package work with AD

Ignoring gradients for certain expressions

There exists code that is not meant to be differentiated through, for example logging. In some cases, AD systems might work perfectly well with that code, but in others they might not. A convenience function ignore_derivatives is provided to get around this issue. It captures the functionality of both Zygote.ignore and Zygote.dropgrad.

For example, Zygote does not support mutation, so it will break if you try to store intermediate values as in the following example:

somes = []
+things = []
+
+function loss(x, y)
+    some = f(x, y)
+    thing = g(x)
+    
+    # log
+    push!(somes, some)
+    push!(things, thing)
+
+    return some + thing
+end

It is possible to get around this by using the ignore_derivatives function

ignore_derivatives() do
+    push!(somes, some)
+    push!(things, thing)
+end

or using a macro for one-liners

@ignore_derivatives push!(things, thing)

It is also possible to use this on individual objects, e.g.

ignore_derivatives(a) + b

will ignore the gradients for a only.

Passing in instances of functors (callable structs), ignore_derivatives(functor), will make them behave like normal structs, i.e. propagate without being called and dropping their gradients. If you want to call a functor in the primal computation, wrap it in a closure: ignore_derivatives(() -> functor())

diff --git a/previews/PR663/rule_author/which_functions_need_rules.html b/previews/PR663/rule_author/which_functions_need_rules.html new file mode 100644 index 000000000..df8fd3738 --- /dev/null +++ b/previews/PR663/rule_author/which_functions_need_rules.html @@ -0,0 +1,64 @@ + +Which functions need rules? · ChainRules
GitHub

Which functions need rules?

In principle, a perfect AD system only needs rules for basic operations and can infer the rules for more complicated functions automatically. In practice, performance needs to be considered as well.

Some functions use ccall internally, for example ^. These functions cannot be differentiated through by AD systems, and need custom rules.

Other functions can in principle be differentiated through by an AD system, but there exists a mathematical insight that can dramatically improve the computation of the derivative. An example is numerical integration, where writing a rule implementing the fundamental theorem of calculus removes the need to perform AD through numerical integration.

Furthermore, AD systems make different trade-offs in performance due to their design. This means that a certain rule will help one AD system, but not improve (and also not harm) another. Below, we list some patterns relevant for the Zygote.jl AD system.

Rules for functions which mutate its arguments, e.g. sort!, should not be written at the moment. While technically they are supported, they would break Zygote.jl such that it would sometimes quietly return the wrong answer. This may be resolved in the future by allowing AD systems to opt-in or opt-out of certain types of rules.

Patterns that need rules in Zygote.jl

There are a few classes of functions that Zygote cannot differentiate through. Custom rules will need to be written for these to make AD work.

Other patterns can be AD'ed through, but the backward pass performance can be greatly improved by writing a rule.

Functions which mutate arrays

For example,

function addone(a::AbstractArray)
+    b = similar(a)
+    b .= a .+ 1
+    return sum(b)
+end

complains that

julia> using Zygote
+julia> gradient(addone, a)
+ERROR: Mutating arrays is not supported

However, upon adding the rrule (restart the REPL after calling gradient)

function ChainRules.rrule(::typeof(addone), a)
+    y = addone(a)
+    function addone_pullback(ȳ)
+        return NoTangent(), ones(length(a))
+    end
+    return y, addone_pullback
+end

the gradient can be evaluated:

julia> gradient(addone, a)
+([1.0, 1.0, 1.0],)

Notice that addone(a) mutates another array b internally, but not its input. This is commonly done in less trivial functions, and is often what Zygote's Mutating arrays is not supported error is telling you, even though you did not intend to mutate anything. Functions which mutate their own input are much more problematic. These are the ones named (by convention) with an exclamation mark, such as fill!(a, x) or push!(a, x). It is not possible to write rules which handle all uses of such a function correctly, on current Zygote.

Why restarting REPL after calling `gradient`?

When gradient is called in Zygote for a function with no rrule defined, a backward pass for the function call is generated and cached. When gradient is called for the second time on the same function signature, the backward pass is reused without checking whether an an rrule has been defined between the two calls to gradient.

If an rrule is defined before the first call to gradient it should register the rule and use it, but that prevents comparing what happens before and after the rrule is defined. To compare both versions with and without an rrule in the REPL simultaneously, define a function f(x) = <body> (no rrule), another function f_cr(x) = f(x), and an rrule for f_cr.

Calling Zygote.refresh() will often have the same effect as restarting the REPL.

Exception handling

Zygote does not support differentiating through try/catch statements. For example, differentiating through

function exception(x)
+    try
+        return x^2
+    catch e
+        println("could not square input")
+        throw(e)
+    end
+end

does not work

julia> gradient(exception, 3.0)
+ERROR: Compiling Tuple{typeof(exception),Int64}: try/catch is not supported.

without an rrule defined (restart the REPL after calling gradient)

function ChainRulesCore.rrule(::typeof(exception), x)
+    y = exception(x)
+    function exception_pullback(ȳ)
+        return NoTangent(), 2*x
+    end
+    return y, exception_pullback
+end
julia> gradient(exception, 3.0)
+(6.0,)

Loops

Julia runs loops fast. Unfortunately Zygote differentiates through loops slowly. So, for example, computing the mean squared error by using a loop

function mse(y, ŷ)
+    N = length(y)
+    s = 0.0
+    for i in 1:N
+        s +=  (y[i] - ŷ[i])^2.0
+    end
+    return s/N
+end

takes a lot longer to AD through

julia> y = rand(30)
+julia> ŷ = rand(30)
+julia> @btime gradient(mse, $y, $ŷ)
+  38.180 μs (993 allocations: 65.00 KiB)

than if we supply an rrule, (restart the REPL after calling gradient)

function ChainRules.rrule(::typeof(mse), x, x̂)
+    output = mse(x, x̂)
+    function mse_pullback(ȳ)
+        N = length(x)
+        g = (2 ./ N) .* (x .- x̂) .* ȳ
+        return NoTangent(), g, -g
+    end
+    return output, mse_pullback
+end

which is much faster

julia> @btime gradient(mse, $y, $ŷ)
+  143.697 ns (2 allocations: 672 bytes)

In-place accumulation

In-place accumulation of gradients is slow in Zygote. The issue, demonstrated in the following example, is that the gradient of getindex allocates an array of zeros with a single non-zero element. 

function sum3(array)
+    x = array[1]
+    y = array[2]
+    z = array[3]
+    return x+y+z
+end
julia> @btime gradient(sum3, rand(30))
+  424.510 ns (9 allocations: 2.06 KiB)

Computing the gradient with only a single array allocation using an rrule (restart the REPL after calling gradient)

function ChainRulesCore.rrule(::typeof(sum3), a)
+    y = sum3(a)
+    function sum3_pullback(ȳ)
+        grad = zeros(length(a))
+        grad[1:3] .+= ȳ
+        return NoTangent(), grad
+    end
+    return y, sum3_pullback
+end

turns out to be significantly faster 

julia> @btime gradient(sum3, rand(30))
+  192.818 ns (3 allocations: 784 bytes)
diff --git a/previews/PR663/rule_author/writing_good_rules.html b/previews/PR663/rule_author/writing_good_rules.html new file mode 100644 index 000000000..5b41e62c0 --- /dev/null +++ b/previews/PR663/rule_author/writing_good_rules.html @@ -0,0 +1,112 @@ + +Writing good rules · ChainRules
GitHub

On writing good rrule / frule methods

Code Style

Use named local functions for the pullback in an rrule.

# good:
+function rrule(::typeof(foo), x)
+    Y = foo(x)
+    function foo_pullback(Ȳ)
+        return NoTangent(), bar(Ȳ)
+    end
+    return Y, foo_pullback
+end
+#== output
+julia> rrule(foo, 2)
+(4, var"#foo_pullback#11"())
+==#
+
+# bad:
+function rrule(::typeof(foo), x)
+    return foo(x), x̄ -> (NoTangent(), bar(x̄))
+end
+#== output:
+julia> rrule(foo, 2)
+(4, var"##9#10"())
+==#

While this is more verbose, it ensures that if an error is thrown during the pullback the gensym name of the local function will include the name you gave it. This makes it a lot simpler to debug from the stacktrace.

Use ZeroTangent() as the return value

The ZeroTangent() object exists as an alternative to directly returning 0 or zeros(n). It allows more optimal computation when chaining pullbacks/pushforwards, to avoid work. They should be used where possible.

However, sometimes for performance reasons this is not ideal. Especially, if it is to replace a scalar, and is in a type-unstable way. It causes problems if mapping over such pullbacks/pushforwards. This would be solved once JuliaLang/julia#38241 has been addressed.

Use Thunks appropriately

If work is only required for one of the returned tangents, then it should be wrapped in a @thunk (potentially using a begin-end block).

If there are multiple return values, their computation should almost always be wrapped in a @thunk.

Do not wrap variables in a @thunk; wrap the computations that fill those variables in @thunk:

# good:
+∂A = @thunk(foo(x))
+return ∂A
+
+# bad:
+∂A = foo(x)
+return @thunk(∂A)

In the bad example foo(x) gets computed eagerly, and all that the thunk is doing is wrapping the already calculated result in a function that returns it.

Do not use @thunk if this would be equal or more work than actually evaluating the expression itself. Examples being:

  • The expression being a constant
  • The expression is merely wrapping something in a struct, such as Adjoint(x) or Diagonal(x)
  • The expression being itself a thunk
  • The expression being from another rrule or frule; it would be @thunked if required by the defining rule already.
  • There is only one derivative being returned, so from the fact that the user called frule/rrule they clearly will want to use that one.

Structs: constructors and functors

To define an frule or rrule for a function foo we dispatch on the type of foo, which is typeof(foo). For example, the rrule signature would be like:

function rrule(::typeof(foo), args...; kwargs...)
+    ...
+    return y, foo_pullback
+end

For a struct Bar,

struct Bar
+    a::Float64
+end
+
+(bar::Bar)(x, y) = return bar.a + x + y # functor (i.e. callable object, overloading the call action)

we can define an frule/rrule for the Bar constructor(s), as well as any Bar functors.

Constructors

To define an rrule for a constructor for a type Bar we need to be careful to dispatch only on Type{Bar}. For example, the rrule signature for a Bar constructor would be like:

function ChainRulesCore.rrule(::Type{Bar}, a)
+    Bar_pullback(Δbar) = NoTangent(), Δbar.a
+    return Bar(a), Bar_pullback
+end

Use Type{<:Bar} (with the <:) for non-concrete types, such that the rrule is defined for all subtypes. In particular, be careful not to use typeof(Bar) here. Because typeof(Bar) is DataType, using this to define an rrule/frule will define an rrule/frule for all constructors.

You can check which to use with Core.Typeof:

julia> function foo end
+foo (generic function with 0 methods)
+
+julia> typeof(foo)
+typeof(foo)
+
+julia> Core.Typeof(foob)
+typeof(foo)
+
+julia> typeof(Bar)
+DataType
+
+julia> Core.Typeof(Bar)
+Type{Bar}
+
+julia> abstract type AbstractT end
+
+julia> typeof(AbstractT)
+DataType
+
+julia> Core.Typeof(AbstractT)
+Type{AbstractT}

Functors (callable objects)

In contrast to defining a rule for a constructor, it is possible to define rules for calling an instance of an object. In that case, use bar::Bar, i.e.

function ChainRulesCore.rrule(bar::Bar, x, y)
+    # Notice the first return is not `NoTangent()`
+    Bar_pullback(Δy) = Tangent{Bar}(;a=Δy), Δy, Δy
+    return bar(x, y), Bar_pullback
+end

to define the rules.

Ensure your pullback can accept the right types

As a rule the number of types you need to accept in a pullback is theoretically unlimitted, but practically highly constrained to be in line with the primal return type. The three kinds of inputs you will practically need to accept one or more of: natural tangents, structural tangents, and thunks. You do not in general have to handle AbstractZeros as the AD system will not call the pullback if the input is a zero, since the output will also be. Some more background information on these types can be found in the design notes. In many cases all these tangents can be treated the same: tangent types overload a bunch of linear-operators, and the majority of functions used inside a pullback are linear operators. If you find linear operators from Base/stdlibs that are not supported, consider opening an issue or a PR on the ChainRulesCore.jl repo.

Natural tangents

Natural tangent types are the types you might feel the tangent should be, to represent a small change in the primal value. For example, if the primal is a Float32, the natural tangent is also a Float32. Slightly more complex, for a ComplexF64 the natural tangent is again also a ComplexF64, we almost never want to use the structural tangent Tangent{ComplexF64}(re=..., im=...) which is defined. For other cases, this gets a little more complicated, see below. These are a purely human notion, they are the types the user wants to use because they make the math easy. There is currently no formal definition of what constitutes a natural tangent, but there are a few heuristics. For example, if a primal type P overloads subtraction (-(::P,::P)) then that generally returns a natural tangent type for P; but this is not required to be defined and sometimes it is defined poorly.

Common cases for types that represent a vector-space (e.g. Float64, Array{Float64}) is that the natural tangent type is the same as the primal type. However, this is not always the case. For example for a PDiagMat a natural tangent is Diagonal since there is no requirement that a positive definite diagonal matrix has a positive definite tangent. Another example is for a DateTime, any Period subtype, such as Millisecond or Nanosecond is a natural tangent. There are often many different natural tangent types for a given primal type. However, they are generally closely related and duck-type the same. For example, for most AbstractArray subtypes, most other AbstractArrays (of right size and element type) can be considered as natural tangent types.

Not all types have natural tangent types. For example there is no natural tangent for a Tuple. It is not a Tuple since that doesn't have any method for +. Similar is true for many structs. For those cases there is only a structural tangent.

Structural tangents

Structural tangents are tangent types that shadow the structure of the primal type. They are represented by the Tangent type. They can represent any composite type, such as a tuple, or a structure (or a NamedTuple) etc.

Do I have to support the structural tangents as well?

Technically, you might not actually have to write rules to accept structural tangents; if the AD system never has to decompose down to the level of getfield. This is common for types that don't support user getfield/getproperty access, and that have a lot of rules for the ways they are accessed (such cases include some AbstractArray subtypes). You really should support it just in case; especially if the primal type in question is not restricted to a well-tested concrete type. But if it is causing struggles, then you can leave it off til someone complains.

Thunks

A thunk (either a Thunk, or a InplaceableThunk), represents a delayed computation. They can be thought of as a wrapper of the value the computation returns. In this sense they wrap either a natural or structural tangent.

You should support AbstractThunk inputs even if you don't use thunks

Unfortunately the AD sytems do not know which rules support thunks and which do not. So all rules have to; at least if they want to play nicely with arbitrary AD systems. Luckily it is not hard: much of the time they will duck-type as the object they wrap. If not, then just add a unthunk after the start of your pullback. (Even when they do duck-type, if they are used multiple times then unthunking at the start will prevent them from being recomputed.) If you are using @thunk and the input is only needed for one of them then the unthunk should be in that one. If not, and you have a bunch of pullbacks you might like to write a little helper unthunking(f) = x̄ -> f(unthunk(x̄)) that you can wrap your pullback function in before returning it from the rrule. Yes, this is a bit of boiler-plate, and it is unfortunate. Sadly, it is needed because if the AD wants to benefit it can't get that benifit unless things are not unthunked unnecessarily. Which eventually allows them in some cases to never be unthunked at all. There are two ways common things are never unthunked. One is if the unthunking happens inside a @thunk which is never unthunked itself because it is the tangent for a primal input that never has it's tangent queried. The second is if they are not unthunked because the rule does not need to know what is inside: consider the pullback for identity: x̄ -> (NoTangent(), x̄).

Use @not_implemented appropriately

You can use @not_implemented to mark missing tangents. This is helpful if the function has multiple inputs or outputs, and you have worked out analytically and implemented some but not all tangents.

It is recommended to include a link to a GitHub issue about the missing tangent in the debugging information:

@not_implemented(
+    """
+    derivatives of Bessel functions with respect to the order are not implemented:
+    https://github.com/JuliaMath/SpecialFunctions.jl/issues/160
+    """
+)

Do not use @not_implemented if the tangent does not exist mathematically (use NoTangent() instead).

Note: ChainRulesTestUtils.jl marks @not_implemented tangents as "test broken".

Use rule definition tools

Rule definition tools can help you write more frules and the rrules with less lines of code. See using rule definition tools section for more details.

Be careful about pullback closures calling other methods of themselves

Due to JuliaLang/Julia#40990, a closure calling another (or the same) method of itself often comes out uninferable (and thus effectively type-unstable). This can be avoided by moving the pullback definition outside the function, so that it is no longer a closure. For example:

double_it(x::AbstractArray) = 2 .* x
+
+function ChainRulesCore.rrule(::typeof(double_it), x)
+    double_it_pullback(ȳ::AbstractArray) = (NoTangent(), 2 .* ȳ)
+    double_it_pullback(ȳ::AbstractThunk) = double_it_pullback(unthunk(ȳ))
+    return double_it(x), double_it_pullback
+end

Ends up infering a return type of Any

julia> _, pullback = rrule(double_it, [2.0, 3.0])
+([4.0, 6.0], var"#double_it_pullback#8"(Core.Box(var"#double_it_pullback#8"(#= circular reference @-2 =#))))
+
+julia> @code_warntype pullback(@thunk([10.0, 10.0]))
+Variables
+  #self#::var"#double_it_pullback#8"
+  ȳ::Core.Const(Thunk(var"#9#10"()))
+  double_it_pullback::Union{}
+
+Body::Any
+1 ─ %1 = Core.getfield(#self#, :double_it_pullback)::Core.Box
+│   %2 = Core.isdefined(%1, :contents)::Bool
+└──      goto #3 if not %2
+2 ─      goto #4
+3 ─      Core.NewvarNode(:(double_it_pullback))
+└──      double_it_pullback
+4 ┄ %7 = Core.getfield(%1, :contents)::Any
+│   %8 = Main.unthunk(ȳ)::Vector{Float64}
+│   %9 = (%7)(%8)::Any
+└──      return %9

This can be solved by moving the pullbacks outside the function so they are not closures, and thus to not run into this upstream issue. In this case that is fairly simple, since this example doesn't close over anything (if it did then would need a closure calling an outside function that calls itself. See this example.).

_double_it_pullback(ȳ::AbstractArray) = (NoTangent(), 2 .* ȳ)
+_double_it_pullback(ȳ::AbstractThunk) = _double_it_pullback(unthunk(ȳ))
+
+function ChainRulesCore.rrule(::typeof(double_it), x)
+    return double_it(x), _double_it_pullback
+end

This infers just fine:

julia> _, pullback = rrule(double_it, [2.0, 3.0])
+([4.0, 6.0], _double_it_pullback)
+
+julia> @code_warntype pullback(@thunk([10.0, 10.0]))
+Variables
+  #self#::Core.Const(_double_it_pullback)
+  ȳ::Core.Const(Thunk(var"#7#8"()))
+
+Body::Tuple{NoTangent, Vector{Float64}}
+1 ─ %1 = Main.unthunk(ȳ)::Vector{Float64}
+│   %2 = Main._double_it_pullback(%1)::Core.PartialStruct(Tuple{NoTangent, Vector{Float64}}, Any[Core.Const(NoTangent()), Vector{Float64}])
+└──      return %2

Though in this particular case, it can also be solved by taking advantage of duck-typing and just writing one method. Thus avoiding the call that confuses the compiler. Thunks duck-type as the type they wrap in most cases: including broadcast multiplication.

function ChainRulesCore.rrule(::typeof(double_it), x)
+    double_it_pullback(ȳ) = (NoTangent(), 2 .* ȳ)
+    return double_it(x), double_it_pullback
+end

This infers perfectly.

CAS systems are your friends.

It is very easy to check gradients or derivatives with a computer algebra system (CAS) like WolframAlpha.

diff --git a/previews/PR663/search_index.js b/previews/PR663/search_index.js new file mode 100644 index 000000000..e448c76f1 --- /dev/null +++ b/previews/PR663/search_index.js @@ -0,0 +1,3 @@ +var documenterSearchIndex = {"docs": +[{"location":"videos.html#Videos","page":"Videos","title":"Videos","text":"","category":"section"},{"location":"videos.html","page":"Videos","title":"Videos","text":"For people who learn better by video we have a number of videos of talks we have given about the ChainRules project. Note however, that the videos are frozen in time reflecting the state of the packages at the time they were recorded. This documentation is the continuously updated canonical source. However, we have tried to note below each video notes on its correctness.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"The talks that follow are in reverse chronological order (i.e. most recent video is first).","category":"page"},{"location":"videos.html#EuroAD-2021:-ChainRules.jl:-AD-system-agnostic-rules-for-JuliaLang","page":"Videos","title":"EuroAD 2021: ChainRules.jl: AD system agnostic rules for JuliaLang","text":"","category":"section"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Presented by Frames White. Slides","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"This is the talk to watch if you want to understand why the ChainRules project exists, what its challenges are, and how those have been overcome. It is intended less for users of the package, and more for people working in the field of AD more generally. It does also serve as a nice motivation for those first coming across the package as well though.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"
\n\n
","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Abstract:","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"The ChainRules project is a suite of JuliaLang packages that define custom primitives (i.e. rules) for doing AD in JuliaLang. Importantly it is AD system agnostic. It has proved successful in this goal. At present it works with about half a dozen different JuliaLang AD systems. It has been a long journey, but as of August 2021, the core packages have now hit version 1.0.This talk will go through why this is useful, the particular objectives the project had, and the challenges that had to be solved. This talk is not intended as an educational guide for users (For that see our 2021 JuliaCon talk: > Everything you need to know about ChainRules 1.0 (https://live.juliacon.org/talk/LWVB39)). Rather this talk is to share the insights we have had, and likely (inadvertently) the mistakes we have made, with the wider autodiff community. We believe these insights can be informative and useful to efforts in other languages and ecosystems.","category":"page"},{"location":"videos.html#JuliaCon-2021:-Everything-you-need-to-know-about-ChainRules-1.0","page":"Videos","title":"JuliaCon 2021: Everything you need to know about ChainRules 1.0","text":"","category":"section"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Presented by Miha Zgubič. Slides","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"If you are just wanting to watch a video to learn all about ChainRules and how to use it, watch this one.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"note: Slide on opting out is incorrect\nSlide 42 is incorrect (@no_rrule sum_array(A::Diagonal)), in the ChainRulesCore 1.0 release the following syntax is used: @opt_out rrule(::typeof(sum_array), A::Diagonal). This syntax allows us to include rule config information.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"
\n\n
","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Abstract:","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"ChainRules is an automatic differentiation (AD)-independent ecosystem for forward-, reverse-, and mixed-mode primitives. It comprises ChainRules.jl, a collection of primitives for Julia Base, ChainRulesCore.jl, the utilities for defining custom primitives, and ChainRulesTestUtils.jl, the utilities to test primitives using finite differences. This talk provides brief updates on the ecosystem since last year and focuses on when and how to write and test custom primitives.","category":"page"},{"location":"videos.html#JuliaCon-2020:-ChainRules.jl","page":"Videos","title":"JuliaCon 2020: ChainRules.jl","text":"","category":"section"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Presented by Frames White. Slides","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"This talk is primarily of historical interest. This was the first public presentation of ChainRules. Though the project was a few years old by this stage. A lot of things are still the same; conceptually, but a lot has changed. Most people shouldn't watch this talk now.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"warning: Outdated Terminology\nA lot of terminology has changed since this presentation.DoesNotExist → NoTangent\nZero → ZeroTangent\nComposite{P} → Tangent{T}The talk also says differential in a lot of places where we now would say tangent.","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"
\n\n
","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"Abstract:","category":"page"},{"location":"videos.html","page":"Videos","title":"Videos","text":"The ChainRules project allows package authors to write rules for custom sensitivities (sometimes called custom adjoints) in a way that is not dependent on any particular autodiff (AD) package. It allows authors of AD packages to access a wealth of prewritten custom sensitivities, saving them the effort of writing them all out themselves. ChainRules is the successor to DiffRules.jl and is the native rule system currently used by ForwardDiff2, Zygote and soon ReverseDiff","category":"page"},{"location":"rule_author/debug_mode.html#Debug-Mode","page":"Debug mode","title":"Debug Mode","text":"","category":"section"},{"location":"rule_author/debug_mode.html","page":"Debug mode","title":"Debug mode","text":"ChainRulesCore supports a debug mode which you can use while writing new rules. It provides better error messages. If you are developing some new rules, and you get a weird error message, it is worth enabling debug mode.","category":"page"},{"location":"rule_author/debug_mode.html","page":"Debug mode","title":"Debug mode","text":"There is some overhead to having it enabled, so it is disabled by default.","category":"page"},{"location":"rule_author/debug_mode.html","page":"Debug mode","title":"Debug mode","text":"To enable, redefine the ChainRulesCore.debug_mode function to return true.","category":"page"},{"location":"rule_author/debug_mode.html","page":"Debug mode","title":"Debug mode","text":"ChainRulesCore.debug_mode() = true","category":"page"},{"location":"rule_author/debug_mode.html#Features-of-Debug-Mode:","page":"Debug mode","title":"Features of Debug Mode:","text":"","category":"section"},{"location":"rule_author/debug_mode.html","page":"Debug mode","title":"Debug mode","text":"If you add a Tangent to a primal value, and it was unable to construct a new primal values, then a better error message will be displayed detailing what overloads need to be written to fix this.\nduring add!!, if an InplaceThunk is used, and it runs the code that is supposed to run in place, but the return result is not the input (with updated values), then an error is thrown. Rather than silently using what ever values were returned.","category":"page"},{"location":"maths/nondiff_points.html#What-to-return-for-non-differentiable-points","page":"Non-differentiable Points","title":"What to return for non-differentiable points","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"info: What is the short version?\nIf the function is not-differentiable choose to return something useful rather than erroring. For a branch a function is not differentiable due to e.g. a branch, like abs, your rule can reasonably claim the derivative at that point is the value from either branch, or any value in-between. In particular for local optima (like in the case of abs) claiming the derivative is 0 is a good idea. Similarly, if derivative is from one side is not defined, or is not finite, return the derivative from the other side. Throwing an error, or returning NaN is generally the least useful option.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"However, contrary to what calculus says most autodiff systems will return an answer for such functions. For example for: abs_left(x) = (x <= 0) ? -x : x, AD will say the derivative at x=0 is -1. Alternatively for: abs_right(x) = (x < 0) ? -x : x, AD will say the derivative at x=0 is 1. Those two examples are weird since they are equal at all points, but AD claims different derivatives at x=0. The way to fix autodiff systems being weird is to write custom rules. So what rule should we write for this case?","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"The obvious answer, would be to write a rule that throws an error if input at a point where calculus says the derivative is not defined. Another option is to return some error signally value like NaN. Which you can do. However, there is no where to go with an error, the user still wants a derivative; so this is not useful.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Let us explore what is useful:","category":"page"},{"location":"maths/nondiff_points.html#Case-Studies","page":"Non-differentiable Points","title":"Case Studies","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"using Plots\ngr(framestyle=:origin, legend=false)","category":"page"},{"location":"maths/nondiff_points.html#Derivative-is-defined-in-usual-sense","page":"Non-differentiable Points","title":"Derivative is defined in usual sense","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(x->x^3)","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"This is the standard case, one can return the derivative that is defined according to school room calculus. Here we would reasonably say that at x=0 the derivative is 3*0^2=0. ","category":"page"},{"location":"maths/nondiff_points.html#Local-Minima-/-Maxima","page":"Non-differentiable Points","title":"Local Minima / Maxima","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(abs)","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"abs is the classic example of a function where the derivative is not defined, as the limit from above is not equal to the limit from below.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"operatornameabs(0) = lim_h to 0^- dfracoperatornameabs(0)-operatornameabs(0-h)0-h = -1","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"operatornameabs(0) = lim_h to 0^+ dfracoperatornameabs(0)-operatornameabs(0-h)0-h = 1","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Now, as discussed in the introduction, the AD system would on it's own choose either 1 or -1, depending on implementation.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"We however have a potentially much nicer answer available to use: 0.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"This has a number of advantages.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"It follows the rule that derivatives are zero at local minima (and maxima).\nIf you leave a gradient descent optimizer running it will eventually actually converge absolutely to the point – where as with it being 1 or -1 it would never outright converge it would always flee.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Further:","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"It is a perfectly nice member of the subderivative.\nIt is the mean of the derivative on each side; which means that it will agree with central finite differencing at the point.","category":"page"},{"location":"maths/nondiff_points.html#Piecewise-slope-change","page":"Non-differentiable Points","title":"Piecewise slope change","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(x-> x < 0 ? x : 5x)","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Here we have 3 main options, all are good.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"We could say the derivative at 0 is:","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"1: which agrees with backwards finite differencing\n5: which agrees with forwards finite differencing\n3: which is the mean of [1, 5], and agrees with central finite differencing","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"All of these options are perfectly nice members of the subderivative. 3 is the arguably the nicest, but it is also the most expensive to compute. In general all are acceptable.","category":"page"},{"location":"maths/nondiff_points.html#Derivative-zero-almost-everywhere","page":"Non-differentiable Points","title":"Derivative zero almost everywhere","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(ceil)","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Here it is most useful to say the derivative is zero everywhere. The limits are zero from both sides.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"The other option for x->ceil(x) would be to relax the problem into x->x, and thus say it is 1 everywhere. But that it too weird, if the user wanted a relaxation of the problem then they would provide one. We can not be imposing that relaxation on to ceil, as it is not reasonable for everyone.","category":"page"},{"location":"maths/nondiff_points.html#Not-defined-on-one-side","page":"Non-differentiable Points","title":"Not defined on one-side","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(x->exp(2log(x)))\nplot!(; xlims=(-10,10), ylims=(-10,10)) #hide","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"We do not have to worry about what to return for the side where it is not defined. As we will never be asked for the derivative at e.g. x=-2.5 since the primal function errors. But we do need to worry about at the boundary – if that boundary point doesn't error.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Since we will never be asked about the left-hand side (as the primal errors), we can use just the right-hand side derivative. In this case giving 0.0.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Also nice in this case is that it agrees with the symbolic simplification of x->exp(2log(x)) into x->x^2.","category":"page"},{"location":"maths/nondiff_points.html#Derivative-nonfinite-and-same-on-both-sides","page":"Non-differentiable Points","title":"Derivative nonfinite and same on both sides","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(cbrt)","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Here we have no real choice but to say the derivative at 0 is Inf. We could consider as an alternative saying some large but finite value. However, if too large it will just overflow rapidly anyway; and if too small it will not dominate over finite terms. It is not possible to find a given value that is always large enough. Our alternatives would be to consider the derivative at nextfloat(0.0) or prevfloat(0.0). But this is more or less the same as choosing some large value – in this case an extremely large value that will rapidly overflow.","category":"page"},{"location":"maths/nondiff_points.html#Derivative-on-finite-and-different-on-both-sides","page":"Non-differentiable Points","title":"Derivative on-finite and different on both sides","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"plot(x-> sign(x) * cbrt(x))","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"In this example, the primal is defined and finite, so we would like a derivative to be defined. We are back in the case of a local minimum like we were for abs. We can make most of the same arguments as we made there to justify saying the derivative is zero.","category":"page"},{"location":"maths/nondiff_points.html#Conclusion","page":"Non-differentiable Points","title":"Conclusion","text":"","category":"section"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"From the case studies a few general rules can be seen for how to choose a value that is useful. These rough rules are:","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Say the derivative is 0 at local optima.\nIf the derivative from one side is defined and the other isn't, say it is the derivative taken from the defined side.\nIf the derivative from one side is finite and the other isn't, say it is the derivative taken from the finite side.\nWhen derivative from each side is not equal, strongly consider reporting the average.","category":"page"},{"location":"maths/nondiff_points.html","page":"Non-differentiable Points","title":"Non-differentiable Points","text":"Our goal as always, is to get a pragmatically useful result for everyone, which must by necessity also avoid a pathological result for anyone.","category":"page"},{"location":"rule_author/superpowers/projectto.html#projectto","page":"ProjectTo","title":"ProjectTo the primal subspace","text":"","category":"section"},{"location":"rule_author/superpowers/projectto.html","page":"ProjectTo","title":"ProjectTo","text":"Rules with abstractly-typed arguments may return incorrect answers when called with certain concrete types. A classic example is the matrix-matrix multiplication rule, a naive definition of which follows:","category":"page"},{"location":"rule_author/superpowers/projectto.html","page":"ProjectTo","title":"ProjectTo","text":"function rrule(::typeof(*), A::AbstractMatrix, B::AbstractMatrix)\n function times_pullback(ȳ)\n dA = ȳ * B'\n dB = A' * ȳ\n return NoTangent(), dA, dB\n end\n return A * B, times_pullback\nend","category":"page"},{"location":"rule_author/superpowers/projectto.html","page":"ProjectTo","title":"ProjectTo","text":"When computing *(A, B), where A isa Diagonal and B isa Matrix, the output will be a Matrix. As a result, ȳ in the pullback will be a Matrix, and consequently dA for a A isa Diagonal will be a Matrix, which is wrong. Not only is it the wrong type, but it can contain non-zeros off the diagonal, which is not possible, it is outside of the subspace. While a specialised rules can indeed be written for the Diagonal case, there are many other types and we don't want to be forced to write a rule for each of them. Instead, project_A = ProjectTo(A) can be used (outside the pullback) to extract an object that knows how to project onto the type of A (e.g. also knows the size of the array). This object can be called with a tangent ȳ * B', by doing project_A(ȳ * B'), to project it on the tangent space of A. The correct rule then looks like","category":"page"},{"location":"rule_author/superpowers/projectto.html","page":"ProjectTo","title":"ProjectTo","text":"function rrule(::typeof(*), A::AbstractMatrix, B::AbstractMatrix)\n project_A = ProjectTo(A)\n project_B = ProjectTo(B)\n function times_pullback(ȳ)\n dA = ȳ * B'\n dB = A' * ȳ\n return NoTangent(), project_A(dA), project_B(dB)\n end\n return A * B, times_pullback\nend","category":"page"},{"location":"rule_author/superpowers/projectto.html","page":"ProjectTo","title":"ProjectTo","text":"note: It is often good to `@thunk` your projections\nThe above example is potentially a good place for using a @thunk. This is not required, but can in some cases be more computationally efficient, see Use Thunks appropriately. When combining thunks and projections, @thunk() must be the outermost call.A more optimized implementation of the matrix-matrix multiplication example would havetimes_pullback(ȳ) = NoTangent(), @thunk(project_A(ȳ * B')), @thunk(project_B(A' * ȳ))within the rrule. This defers both the evaluation of the product rule and the projection until(/if) the tangent gets used.","category":"page"},{"location":"maths/propagators.html#The-propagators:-pushforward-and-pullback","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"terminology: pushforward and pullback\nPushforward and pullback are fancy words that the autodiff community recently adopted from Differential Geometry. They are broadly in agreement with the use of pullback and pushforward in differential geometry. But any geometer will tell you these are the super-boring flat cases. Some will also frown at you. They are also sometimes described in terms of the jacobian: The pushforward is jacobian vector product (jvp), and pullback is jacobian transpose vector product (j'vp). Other terms that may be used include for pullback the backpropagator, and by analogy for pushforward the forwardpropagator, thus these are the propagators. These are also good names because effectively they propagate wiggles and wobbles through them, via the chain rule. (the term backpropagator may originate with \"Lambda The Ultimate Backpropagator\" by Pearlmutter and Siskind, 2008)","category":"page"},{"location":"maths/propagators.html#Core-Idea","page":"The propagators: pushforward and pullback","title":"Core Idea","text":"","category":"section"},{"location":"maths/propagators.html#Less-formally","page":"The propagators: pushforward and pullback","title":"Less formally","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pushforward takes a wiggle in the input space, and tells what wobble you would create in the output space, by passing it through the function.\nThe pullback takes wobbliness information with respect to the function's output, and tells the equivalent wobbliness with respect to the functions input.","category":"page"},{"location":"maths/propagators.html#More-formally","page":"The propagators: pushforward and pullback","title":"More formally","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pushforward of f takes the sensitivity of the input of f to a quantity, and gives the sensitivity of the output of f to that quantity.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pullback of f takes the sensitivity of a quantity to the output of f, and gives the sensitivity of that quantity to the input of f.","category":"page"},{"location":"maths/propagators.html#Math","page":"The propagators: pushforward and pullback","title":"Math","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"This is all a bit simplified by talking in 1D.","category":"page"},{"location":"maths/propagators.html#Lighter-Math","page":"The propagators: pushforward and pullback","title":"Lighter Math","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"For a chain of expressions:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"a = f(x)\nb = g(a)\nc = h(b)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pullback of g, which incorporates the knowledge of ∂b/∂a, applies the chain rule to go from ∂c/∂b to ∂c/∂a.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pushforward of g, which also incorporates the knowledge of ∂b/∂a, applies the chain rule to go from ∂a/∂x to ∂b/∂x.","category":"page"},{"location":"maths/propagators.html#Geometric-interpretation-of-reverse-and-forwards-mode-AD","page":"The propagators: pushforward and pullback","title":"Geometric interpretation of reverse and forwards mode AD","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Let us think of our types geometrically. In other words, elements of a type form a manifold. This document will explain this point of view in some detail.","category":"page"},{"location":"maths/propagators.html#Some-terminology/conventions","page":"The propagators: pushforward and pullback","title":"Some terminology/conventions","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Let p be an element of type M, which is defined by some assignment of numbers x_1 dots x_m, say (x_1 dots x_m) = (a_1 dots a_m)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"A function fM to K on M is (for simplicity) a polynomial Kx_1 dots x_m","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The tangent space T_pM of M at point p is the K-vector space spanned by derivations ddx. The tangent space acts linearly on the space of functions. They act as usual on functions. Our starting point is that we know how to write down ddx(f) = dfdx.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The collection of tangent spaces T_pM for pin M is called the tangent bundle of M.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Let df denote the first order information of f at each point. This is called the differential of f. If the derivatives of f and g agree at p, we say that df and dg represent the same cotangent at p. The covectors dx_1 dots dx_m form the basis of the cotangent space T^*_pM at p. Notice that this vector space is dual to T_pM.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The collection of cotangent spaces T^*_pM for pin M is called the cotangent bundle of M.","category":"page"},{"location":"maths/propagators.html#Push-forwards-and-pullbacks","page":"The propagators: pushforward and pullback","title":"Push-forwards and pullbacks","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Let N be another type, defined by numbers y_1y_n, and let gM to N be a map, that is, an n-dimensional vector (g_1 g_m) of functions on M.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"We define the push-forward g_*TM to TN between tangent bundles by g_*(X)(h) = X(gcirc h) for any tangent vector X and function f. We have g_*(ddx_i)(y_j) = dg_jdx_i, so the push-forward corresponds to the Jacobian, given a chosen basis.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Similarly, the pullback of the differential df is defined by g^*(df) = d(fcirc g). So for a coordinate differential dy_j, we have g^*(dy_j) = d(g_j). Notice that this is a covector, and we could have defined the pullback by its action on vectors by g^*(dh)(X) = g_*(X)(dh) = X(gcirc h) for any function f on N and Xin TM. In particular, g^*(dy_j)(ddx_i) = d(g_j)dx_i. If you work out the action in a basis of the cotangent space, you see that it acts by the adjoint of the Jacobian.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Notice that the pullback of a differential and the pushforward of a vector have a very different meaning, and this should be reflected on how they are used in code.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The information contained in the push-forward map is exactly what does my function do to tangent vectors. Pullbacks, acting on differentials of functions, act by taking the total derivative of a function. This works in a coordinate invariant way, and works without the notion of a metric. Gradients recall are vectors, yet they should contain the same information of the differential df. Assuming we use the standard euclidean metric, we can identify df and nabla f as vectors. But pulling back gradients still should not be a thing.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"If the goal is to evaluate the gradient of a function f=gcirc hM to N to K, where g is a map and h is a function, we have two obvious options: First, we may push-forward a basis of M to TK which we identify with K itself. This results in m scalars, representing components of the gradient. Step-by-step in coordinates:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Compute the push-forward of the basis of T_pM, i.e. just the columns of the Jacobian dg_idx_j.\nCompute the push-forward of the function h (consider it as a map, K is also a manifold!) to get h_*(g_*T_pM) = sum_j dhdy_i (dg_idx_j)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Second, we pull back the differential dh:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"compute dh = dhdy_1dhdy_n in coordinates.\npull back by (in coordinates) multiplying with the adjoint of the Jacobian, resulting in g_*(dh) = sum_i(dg_idx_j)(dhdy_i).","category":"page"},{"location":"maths/propagators.html#The-anatomy-of-pullback-and-pushforward","page":"The propagators: pushforward and pullback","title":"The anatomy of pullback and pushforward","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"For our function foo(args...; kwargs...) = y:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"function pullback(Δy)\n ...\n return ∂self, ∂args...\nend","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The input to the pullback is often called the seed. If the function is y = f(x) often the pullback will be written s̄elf, x̄ = pullback(ȳ).","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"note: Note\nThe pullback returns one ∂arg per arg to the original function, plus one ∂self for the fields of the function itself (explained below).","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"terminology: perturbation, seed, sensitivity\nSometimes perturbation, seed, and even sensitivity will be used interchangeably. They are not generally synonymous, and ChainRules shouldn't mix them up. One must be careful when reading literature. At the end of the day, they are all wiggles or wobbles.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The pushforward is a part of the frule function. Considered alone it would look like:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"function pushforward(Δself, Δargs...)\n ...\n return ∂y\nend","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"But because it is fused into frule we see it as part of:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"function frule((Δself, Δargs...), ::typeof(foo), args...; kwargs...)\n ...\n return y, ∂y\nend","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The input to the pushforward is often called the perturbation. If the function is y = f(x) often the pushforward will be written ẏ = last(frule((ṡelf, ẋ), f, x)). ẏ is commonly used to represent the perturbation for y.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"note: Note\nIn the frule/pushforward, there is one Δarg per arg to the original function. The Δargs are similar in type/structure to the corresponding inputs args (Δself is explained below). The ∂y are similar in type/structure to the original function's output Y. In particular if that function returned a tuple then ∂y will be a tuple of the same size.","category":"page"},{"location":"maths/propagators.html#self_derivative","page":"The propagators: pushforward and pullback","title":"Self derivative Δself, ∂self, s̄elf, ṡelf etc","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"terminology: Δself, ∂self, s̄elf, ṡelf\nIt is the derivatives with respect to the internal fields of the function. To the best of our knowledge there is no standard terminology for this. Other good names might be Δinternal/∂internal.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"From the mathematical perspective, one may have been wondering what all this Δself, ∂self is. Given that a function with two inputs, say f(a, b), only has two partial derivatives: dfracfa, dfracfb. Why then does a pushforward take in this extra Δself, and why does a pullback return this extra ∂self?","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The reason is that in Julia the function f may itself have internal fields. For example a closure has the fields it closes over; a callable object (i.e. a functor) like a Flux.Dense has the fields of that object.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Thus every function is treated as having the extra implicit argument self, which captures those fields. So every pushforward takes in an extra argument, which is ignored unless the original function has fields. It is common to write function foo_pushforward(_, Δargs...) in the case when foo does not have fields. Similarly every pullback returns an extra ∂self, which for things without fields is NoTangent(), indicating there are no fields within the function itself.","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Here's an example showing how to define ∂self in an rrule when the primal function has internal fields (implicit arguments):","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"struct Multiplier{T}\n x::T\nend\n(m::Multiplier)(y) = m.x * y\n\nfunction ChainRulesCore.rrule(m::Multiplier, y)\n product = m(y)\n function pullback(Δproduct) \n ∂self = Tangent{Multiplier}(; x = Δproduct * y')\n ∂y = m.x' * Δproduct\n return ∂self, ∂y\n end \n return product, pullback\nend","category":"page"},{"location":"maths/propagators.html#Pushforward-/-Pullback-summary","page":"The propagators: pushforward and pullback","title":"Pushforward / Pullback summary","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Pullback\nreturned by rrule\ntakes output space wobbles, gives input space wiggles\nArgument structure matches structure of primal function output\nIf primal function returns a tuple, then pullback takes in a tuple of differentials.\n1 return per original function argument + 1 for the function itself\nPushforward:\npart of frule\ntakes input space wiggles, gives output space wobbles\nArgument structure matches primal function argument structure, but passed as a tuple at start of frule\n1 argument per original function argument + 1 for the function itself\n1 return per original function return","category":"page"},{"location":"maths/propagators.html#Pullback/Pushforward-and-Directional-Derivative/Gradient","page":"The propagators: pushforward and pullback","title":"Pullback/Pushforward and Directional Derivative/Gradient","text":"","category":"section"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"The most trivial use of the pushforward from within frule is to calculate the directional derivative:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"If we would like to know the directional derivative of f for an input change of (1.5, 0.4, -1)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"direction = (1.5, 0.4, -1) # (ȧ, ḃ, ċ)\ny, ẏ = frule((ZeroTangent(), direction...), f, a, b, c)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"On the basis directions one gets the partial derivatives of y:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"y, ∂y_∂a = frule((ZeroTangent(), 1, 0, 0), f, a, b, c)\ny, ∂y_∂b = frule((ZeroTangent(), 0, 1, 0), f, a, b, c)\ny, ∂y_∂c = frule((ZeroTangent(), 0, 0, 1), f, a, b, c)","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Similarly, the most trivial use of rrule and returned pullback is to calculate the gradient:","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"y, f_pullback = rrule(f, a, b, c)\n∇f = f_pullback(1) # for appropriate `1`-like seed.\ns̄elf, ā, b̄, c̄ = ∇f","category":"page"},{"location":"maths/propagators.html","page":"The propagators: pushforward and pullback","title":"The propagators: pushforward and pullback","text":"Then we have that ∇f is the gradient of f at (a, b, c). And we thus have the partial derivatives overlinemathrmself = dfracfmathrmself, overlinea = dfracfa, overlineb = dfracfb, overlinec = dfracfc, including the self-partial derivative, overlinemathrmself.","category":"page"},{"location":"rule_author/intro.html#Introduction","page":"Introduction","title":"Introduction","text":"","category":"section"},{"location":"rule_author/intro.html","page":"Introduction","title":"Introduction","text":"This section of the docs will tell you everything you need to know about writing rules for your package.","category":"page"},{"location":"rule_author/intro.html","page":"Introduction","title":"Introduction","text":"It will help with understanding tangent types, the anatomy of the frule and the rrule, and provide tips on writing good rules, as well as how to test them easily using finite differences.","category":"page"},{"location":"rule_author/intro.html","page":"Introduction","title":"Introduction","text":"This section also outlines some ChainRules superpowers that can be considered advanced usage. Most users can ignore these. However:","category":"page"},{"location":"rule_author/intro.html","page":"Introduction","title":"Introduction","text":"If you are writing rules with abstractly typed arguments, read about ProjectTo.\nIf you want to opt out of using the abstractly typed rule for certain argument types, read about @opt_out.\nIf you are writing rules for higher order functions, read about calling back into AD.\nIf you want to accumulate gradients in-place to avoid extra allocations, read about gradient accumulation.","category":"page"},{"location":"rule_author/rule_definition_tools.html#ruletools","page":"Rule definition tools","title":"Using rule definition tools","text":"","category":"section"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"Rule definition tools can help you write more frules and the rrules with less lines of code.","category":"page"},{"location":"rule_author/rule_definition_tools.html#[@non_differentiable](@ref)","page":"Rule definition tools","title":"@non_differentiable","text":"","category":"section"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"For non-differentiable functions the @non_differentiable macro can be used. For example, instead of manually defining the frule and the rrule for string concatenation *(String..), the macro call","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"@non_differentiable *(String...)","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"defines the following frule and rrule automatically","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"function ChainRulesCore.frule(var\"##_#1600\", ::Core.Typeof(*), String::Any...; kwargs...)\n return (*(String...; kwargs...), NoTangent())\nend\nfunction ChainRulesCore.rrule(::Core.Typeof(*), String::Any...; kwargs...)\n return (*(String...; kwargs...), function var\"*_pullback\"(_)\n (ZeroTangent(), ntuple((_->NoTangent()), 0 + length(String))...)\n end)\nend","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"Note that the types of arguments are propagated to the frule and rrule definitions. This is needed in case the function differentiable for some but not for other types of arguments. For example *(1, 2, 3) is differentiable, and is not defined with the macro call above.","category":"page"},{"location":"rule_author/rule_definition_tools.html#[@scalar_rule](@ref)","page":"Rule definition tools","title":"@scalar_rule","text":"","category":"section"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"For functions involving only scalars, i.e. subtypes of Number (no structs, Strings...), both the frule and the rrule can be defined using a single @scalar_rule macro call.","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"Note that the function does not have to be mathbbR rightarrow mathbbR. In fact, any number of scalar arguments is supported, as is returning a tuple of scalars.","category":"page"},{"location":"rule_author/rule_definition_tools.html","page":"Rule definition tools","title":"Rule definition tools","text":"See docstrings for the comprehensive usage instructions.","category":"page"},{"location":"ad_author/call_back_into_ad.html#Declaring-support-for-calling-back-into-ADs","page":"Support calling back into ADs","title":"Declaring support for calling back into ADs","text":"","category":"section"},{"location":"ad_author/call_back_into_ad.html","page":"Support calling back into ADs","title":"Support calling back into ADs","text":"To declare support or lack of support for forward and reverse-mode, use the two pairs of complementary types. For reverse mode: HasReverseMode, NoReverseMode. For forwards mode: HasForwardsMode, NoForwardsMode. AD systems that support any calling back into AD should have one from each set.","category":"page"},{"location":"ad_author/call_back_into_ad.html","page":"Support calling back into ADs","title":"Support calling back into ADs","text":"If an AD HasReverseMode, then it must define rrule_via_ad for that RuleConfig subtype. Similarly, if an AD HasForwardsMode then it must define frule_via_ad for that RuleConfig subtype.","category":"page"},{"location":"ad_author/call_back_into_ad.html","page":"Support calling back into ADs","title":"Support calling back into ADs","text":"For example:","category":"page"},{"location":"ad_author/call_back_into_ad.html","page":"Support calling back into ADs","title":"Support calling back into ADs","text":"struct MyReverseOnlyADRuleConfig <: RuleConfig{Union{HasReverseMode, NoForwardsMode}} end\n\nfunction ChainRulesCore.rrule_via_ad(::MyReverseOnlyADRuleConfig, f, args...)\n ...\n return y, pullback\nend","category":"page"},{"location":"ad_author/call_back_into_ad.html","page":"Support calling back into ADs","title":"Support calling back into ADs","text":"Note that it is not actually required that the same AD is used for forward and reverse. For example Nabla.jl is a reverse mode AD. It might declare that it HasForwardsMode, and then define a wrapper around ForwardDiff.jl in order to provide that capacity.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Converting-ZygoteRules.@adjoint-to-rrules","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"ZygoteRules.jl is a legacy package similar to ChainRulesCore but supporting Zygote.jl only.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"If you have some rules written with ZygoteRules it is a good idea to upgrade them to use ChainRules instead. Zygote will still be able to use them, but so will other AD systems, and you will get access to some more advanced features. Some of these features are currently ignored by Zygote, but could be supported in the future.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Example","page":"Converting ZygoteRules.@adjoint to rrules","title":"Example","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Consider the function","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"struct Foo\n a::Float64,\n b::Float64\nend\n\nf(x, y::Foo, z) = 2*x + y.a","category":"page"},{"location":"rule_author/converting_zygoterules.html#ZygoteRules","page":"Converting ZygoteRules.@adjoint to rrules","title":"ZygoteRules","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"@adjoint function f(x, y::Foo, z)\n f_pullback(Ω̄) = (2Ω̄, NamedTuple(;a=Ω̄, b=nothing), nothing)\n return f(x, y, z), f_pullback\nend","category":"page"},{"location":"rule_author/converting_zygoterules.html#ChainRules","page":"Converting ZygoteRules.@adjoint to rrules","title":"ChainRules","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"function rrule(::typeof(f), x, y::Foo, z)\n f_pullback(Ω̄) = (NoTangent(), 2Ω̄, Tangent{Foo}(;a=Ω̄), ZeroTangent())\n return f(x, y, z), f_pullback\nend","category":"page"},{"location":"rule_author/converting_zygoterules.html#Write-as-a-rrule(::typeof(f),-...)","page":"Converting ZygoteRules.@adjoint to rrules","title":"Write as a rrule(::typeof(f), ...)","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"No magic macro here, rrule is the function that it is. The function it is the rule for is the first argument, or second argument if you need to take a RuleConfig.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Note that when writing the rule for constructor you will need to use ::Type{Foo}, not typeof(Foo). See docs on Constructors.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Include-the-derivative-with-respect-to-the-function-object-itself","page":"Converting ZygoteRules.@adjoint to rrules","title":"Include the derivative with respect to the function object itself","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"The ZygoteRules.@adjoint macro automagically[1] inserts an extra nothing in the return for the function it generates to represent the derivative of output with respect to the function object. ChainRules as a philosophy avoids magic as much as possible, and thus require you to return it explicitly. If it is a plain function (like typeof(sin)), then the tangent will be NoTangent.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"[1]: unless you write it in functor form (i.e. @adjoint (f::MyType)(args...)=...), in that case like for rrule you need to include it explicitly.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Tangent-Type-changes","page":"Converting ZygoteRules.@adjoint to rrules","title":"Tangent Type changes","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"ChainRules uses tangent types that must represent vector spaces (i.e. tangent spaces). They need to have things like + defined on them. ZygoteRules takes a more adhoc approach to this.","category":"page"},{"location":"rule_author/converting_zygoterules.html#nothing-becomes-an-AbstractZero","page":"Converting ZygoteRules.@adjoint to rrules","title":"nothing becomes an AbstractZero","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"ZygoteRules uses nothing to represent some sense of zero, in a primal type agnostic way. There are many senses of zero. ChainRules represents two of them, as subtypes of AbstractZero.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"ZeroTangent for the case that there is no relationship between the primal output and the primal input. NoTangent for the case where conceptually the tangent space doesn't exist. e.g. what is the Tangent to a String or an index: those can't be perturbed.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"See FAQ on the difference between ZeroTangent and NoTangent. At the end of the day it doesn't matter too much if you get them wrong. NoTangent and ZeroTangent more or less act identically.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Tuples-and-NamedTuples-become-Tangent{T}s","page":"Converting ZygoteRules.@adjoint to rrules","title":"Tuples and NamedTuples become Tangent{T}s","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Zygote uses Tuples and NamedTuples to represent the structural tangents for Tuples and structs respectively. ChainRules core provides a generic Tangent{T} to represent the structural tangent of a primal type T. It takes positional arguments if representing tangent for a Tuple. Or keyword argument to represent the tangent for a struct or a NamedTuple. When representing a struct you only need to list the nonzero fields – any not given are implicit considered to be ZeroTangent.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"When we say structural tangent we mean tangent types that are based only on the structure of the primal. This is in contrast to a natural tangent which captures some knowledge based on what the primal type represents. (E.g. for arrays a natural tangent is often the same kind of array). For more details see the the design docs on the many tangent types","category":"page"},{"location":"rule_author/converting_zygoterules.html#Calling-back-into-AD-(ZygoteRules.pullback)","page":"Converting ZygoteRules.@adjoint to rrules","title":"Calling back into AD (ZygoteRules.pullback)","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Rules that need to call back into the AD system, e.g, for higher order functions like map(f, xs), need to be changed. In ZygoteRules you can use ZygoteRules.pullback or ZygoteRules._pullback, which will always result in calling into Zygote. Since ChainRules is AD agnostic, you can't do that. Instead you use a RuleConfig to specify requirements of an AD system e.g ::RuleConfig{>:HasReverseMode} work for Zygote, and then use rrule_via_ad.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"See the docs on calling back into AD for more details.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Consider-adding-some-thunks","page":"Converting ZygoteRules.@adjoint to rrules","title":"Consider adding some thunks","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"A feature ChainRulesCore offers that ZygoteRules doesn't is support for thunks. Thunks delay work until it is needed, and avoid it if it never is. See docs on @thunk, Thunk, InplaceableThunk.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"You don't have to use thunks, though. It is easy to go overboard with using thunks.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Testing-Changes","page":"Converting ZygoteRules.@adjoint to rrules","title":"Testing Changes","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"One of the advantages of using ChainRules is that you can easily and robustly test your rules with ChainRulesTestUtils.jl. This uses finite differencing to test the accuracy of derivative, as well as checks the correctness of the API. It should catch anything you might have gotten wrong referred to in this page.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"The test for the above example is test_rrule(f, 2.5, Foo(9.9, 7.2), 31.0). You can see it looks a lot like an example call to rrule, just with the prefix test_ added to the start.","category":"page"},{"location":"rule_author/converting_zygoterules.html#@nograd-becomes-@non_differentiable","page":"Converting ZygoteRules.@adjoint to rrules","title":"@nograd becomes @non_differentiable","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Probably more or less with no changes. @non_differentiable also lets you specify a signature in case you want to restrict non-differentiability to a certain subset of argument types.","category":"page"},{"location":"rule_author/converting_zygoterules.html#No-such-thing-as-literal_getproperty","page":"Converting ZygoteRules.@adjoint to rrules","title":"No such thing as literal_getproperty","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"That is just getproperty, it takes Symbol. It should constant-fold. It likely doesn't though as Zygote doesn't play nice with the optimizer.","category":"page"},{"location":"rule_author/converting_zygoterules.html#Take-embedded-spaces-and-types-seriously","page":"Converting ZygoteRules.@adjoint to rrules","title":"Take embedded spaces and types seriously","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Traditionally Zygote has taken a very laissez-faire attitude towards types and mathematical spaces. Sometimes treating Reals as embedded in the Complex plane; sometimes not. Sometimes treating sparse and structuredly-sparse matrix as embedded in the space of dense matrices. Writing rules that apply to any Array{T} which perhaps are only applicable for Array{<:Real} and not so much for Array{Quaternion}. Traditionally ChainRules takes a much more considered approach.","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"See for example our docs on how to handle complex numbers correctly. (The outcome of several long long long discussions with a number of experts in our community)","category":"page"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"Now, I am not here to tell you what to do in your package, but this is a good time to reconsider how seriously you take these things in the rules you are converting.","category":"page"},{"location":"rule_author/converting_zygoterules.html#What-if-I-miss-something","page":"Converting ZygoteRules.@adjoint to rrules","title":"What if I miss something","text":"","category":"section"},{"location":"rule_author/converting_zygoterules.html","page":"Converting ZygoteRules.@adjoint to rrules","title":"Converting ZygoteRules.@adjoint to rrules","text":"It is not great, but it probably OK. Zygote's ChainRules interface is fairly forgiving. Other AD systems may not be. If you test with ChainRulesTestUtils.jl then you can be confident that you didn't miss anything.","category":"page"},{"location":"rule_author/example.html#Pedagogical-Example","page":"Pedagogical example","title":"Pedagogical Example","text":"","category":"section"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"This pedagogical example will show you how to write an rrule. See On writing good rrule / frule methods section for more tips and gotchas. If you want to learn about frules, you should still read and understand this example as many concepts are shared, and then look for real world frule examples in ChainRules.jl.","category":"page"},{"location":"rule_author/example.html#The-primal","page":"Pedagogical example","title":"The primal","text":"","category":"section"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"We define a struct Foo","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"struct Foo{T}\n A::Matrix{T}\n c::Float64\nend","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"and a function that multiplies Foo with an AbstractArray:","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"function foo_mul(foo::Foo, b::AbstractArray)\n return foo.A * b\nend","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"Note that field c is ignored in the calculation.","category":"page"},{"location":"rule_author/example.html#The-rrule","page":"Pedagogical example","title":"The rrule","text":"","category":"section"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The rrule method for our primal computation should extend the ChainRulesCore.rrule function.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"function ChainRulesCore.rrule(::typeof(foo_mul), foo::Foo{T}, b::AbstractArray) where T\n y = foo_mul(foo, b)\n function foo_mul_pullback(ȳ)\n f̄ = NoTangent()\n f̄oo = Tangent{Foo{T}}(; A=ȳ * b', c=ZeroTangent())\n b̄ = @thunk(foo.A' * ȳ)\n return f̄, f̄oo, b̄\n end\n return y, foo_mul_pullback\nend","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"We can check this rule against a finite-differences approach using ChainRulesTestUtils:","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"julia> using ChainRulesTestUtils\njulia> test_rrule(foo_mul, Foo(rand(3, 3), 3.0), rand(3, 3))\nTest Summary: | Pass Total\ntest_rrule: foo_mul on Foo{Float64},Matrix{Float64} | 10 10\nTest.DefaultTestSet(\"test_rrule: foo_mul on Foo{Float64},Matrix{Float64}\", Any[], 10, false, false)","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"Now let's examine the rule in more detail:","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"function ChainRulesCore.rrule(::typeof(foo_mul), foo::Foo, b::AbstractArray)\n ...\n return y, foo_mul_pullback\nend","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The rrule dispatches on the typeof of the function we are writing the rrule for, as well as the types of its arguments. Read more about writing rules for constructors and callable objects here. The rrule returns the primal result y, and the pullback function. It is a very good idea to name your pullback function, so that they are helpful when appearing in the stacktrace.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"y = foo_mul(foo, b)","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"Computes the primal result. It is possible to change the primal computation so that work can be shared between the primal and the pullback. See e.g. the rule for sort, where the sorting is done only once.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"function foo_mul_pullback(ȳ)\n ...\n return f̄, f̄oo, b̄\nend","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The pullback function takes in the tangent of the primal output (ȳ) and returns the tangents of the primal inputs. Note that it returns a tangent for the primal function in addition to the tangents of primal arguments.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"Finally, computing the tangents of primal inputs:","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"f̄ = NoTangent()","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The function foo_mul has no fields (i.e. it is not a closure) and can not be perturbed. Therefore its tangent (f̄) is a NoTangent.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"f̄oo = Tangent{Foo}(; A=ȳ * b', c=ZeroTangent())","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The struct foo::Foo gets a Tangent{Foo} structural tangent, which stores the tangents of fields of foo.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The tangent of the field A is ȳ * b',","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The tangent of the field c is ZeroTangent(), because c can be perturbed but has no effect on the primal output.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"b̄ = @thunk(foo.A' * ȳ)","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"The tangent of b is foo.A' * ȳ, but we have wrapped it into a Thunk, a tangent type that represents delayed computation. The idea is that in case the tangent is not used anywhere, the computation never happens. Use InplaceableThunk if you are interested in accumulating gradients in-place. Note that in practice one would also @thunk the f̄oo.A tangent, but it was omitted in this example for clarity.","category":"page"},{"location":"rule_author/example.html","page":"Pedagogical example","title":"Pedagogical example","text":"As a final note, since b is an AbstractArray, its tangent b̄ should be projected to the right subspace. See the ProjectTo the primal subspace section for more information and an example that motivates the projection operation.","category":"page"},{"location":"ad_author/use_in_ad_system.html#Using-ChainRules-in-your-AD-system","page":"Usage in AD","title":"Using ChainRules in your AD system","text":"","category":"section"},{"location":"ad_author/use_in_ad_system.html","page":"Usage in AD","title":"Usage in AD","text":"This section is for authors of AD systems. It assumes a pretty solid understanding of both Julia and automatic differentiation. It explains how to make use of ChainRule's \"rulesets\" (frules, rrules,) to avoid having to code all your own AD primitives / custom sensitives.","category":"page"},{"location":"ad_author/use_in_ad_system.html","page":"Usage in AD","title":"Usage in AD","text":"There are 3 main ways to access ChainRules rule sets in your AutoDiff system.","category":"page"},{"location":"ad_author/use_in_ad_system.html","page":"Usage in AD","title":"Usage in AD","text":"Operator Overloading Generation\nUse ChainRulesOverloadGeneration.jl.\nThis is primarily intended for operator overloading based AD systems which will generate overloads for primal functions based for their overloaded types based on the existence of an rrule/frule.\nA source code generation based AD can also use this by overloading their transform generating function directly so as not to recursively generate a transform but to just return the rule.\nThis does not play nice with Revise.jl, adding or modifying rules in loaded files will not be reflected until a manual refresh, and deleting rules will not be reflected at all.\nSource code transform based on inserting branches that check of rrule/frule return nothing\nIf the rrule/frule returns a rule result then use it, if it returns nothing then do normal AD path.\nIn theory type inference optimizes these branches out; in practice it may not.\nThis is a fairly simple Cassette overdub (or similar) of all calls, and is suitable for overloading based AD or source code transformation.\nSource code transform based on rrule/frule method-table\nIf an applicable rrule/frule exists in the method table then use it, else generate normal AD path.\nThis avoids having branches in your generated code.\nThis requires maintaining your own back-edges.\nThis is pretty hardcore even by the standard of source code transformations.","category":"page"},{"location":"maths/complex.html#complexfunctions","page":"Complex numbers","title":"How do chain rules work for complex functions?","text":"","category":"section"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"ChainRules follows the convention that frule applied to a function f(x + i y) = u(xy) + i v(xy) with perturbation Delta x + i Delta y returns the value and","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"tfracpartial upartial x Delta x + tfracpartial upartial y Delta y + i Bigl( tfracpartial vpartial x Delta x + tfracpartial vpartial y Delta y Bigr)\n","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"Similarly, rrule applied to the same function returns the value and a pullback function which, when applied to the adjoint Delta u + i Delta v, returns","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"Delta u tfracpartial upartial x + Delta v tfracpartial vpartial x + i Bigl(Delta u tfracpartial u partial y + Delta v tfracpartial vpartial y Bigr)\n","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"If we interpret complex numbers as vectors in mathbbR^2, then frule (rrule) corresponds to multiplication with the (transposed) Jacobian of f(z), i.e. frule corresponds to","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"beginpmatrix\ntfracpartial upartial x Delta x + tfracpartial upartial y Delta y\n\ntfracpartial vpartial x Delta x + tfracpartial vpartial y Delta y\nendpmatrix\n=\nbeginpmatrix\ntfracpartial upartial x tfracpartial upartial y \ntfracpartial vpartial x tfracpartial vpartial y \nendpmatrix\nbeginpmatrix\nDelta x Delta y\nendpmatrix\n","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"and rrule corresponds to","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"beginpmatrix\ntfracpartial upartial x Delta u + tfracpartial vpartial x Delta v\n\ntfracpartial upartial y Delta u + tfracpartial vpartial y Delta v\nendpmatrix\n=\nbeginpmatrix\ntfracpartial upartial x tfracpartial upartial y \ntfracpartial vpartial x tfracpartial vpartial y \nendpmatrix^mathsfT\nbeginpmatrix\nDelta u Delta v\nendpmatrix\n","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"The Jacobian of fmathbbC to mathbbC interpreted as a function mathbbR^2 to mathbbR^2 can hence be evaluated using either of the following functions.","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"function jacobian_via_frule(f,z)\n du_dx, dv_dx = reim(frule((ZeroTangent(), 1),f,z)[2])\n du_dy, dv_dy = reim(frule((ZeroTangent(),im),f,z)[2])\n return [\n du_dx du_dy\n dv_dx dv_dy\n ]\nend","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"function jacobian_via_rrule(f,z)\n _, pullback = rrule(f,z)\n du_dx, du_dy = reim(pullback( 1)[2])\n dv_dx, dv_dy = reim(pullback(im)[2])\n return [\n du_dx du_dy\n dv_dx dv_dy\n ]\nend","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"If f(z) is holomorphic, then the derivative part of frule can be implemented as f(z) Delta z and the derivative part of rrule can be implemented as bigl(f(z)bigr)^* Delta f, where cdot^* is the complex conjugate. Consequently, holomorphic derivatives can be evaluated using either of the following functions.","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"function holomorphic_derivative_via_frule(f,z)\n fz,df_dz = frule((ZeroTangent(),1),f,z)\n return df_dz\nend","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"function holomorphic_derivative_via_rrule(f,z)\n fz, pullback = rrule(f,z)\n dself, conj_df_dz = pullback(1)\n return conj(conj_df_dz)\nend","category":"page"},{"location":"maths/complex.html","page":"Complex numbers","title":"Complex numbers","text":"note: Note\nThere are various notions of complex derivatives (holomorphic and Wirtinger derivatives, Jacobians, gradients, etc.) which differ in subtle but important ways. The goal of ChainRules is to provide the basic differentiation rules upon which these derivatives can be implemented, but it does not implement these derivatives itself. It is recommended that you carefully check how the above definitions of frule and rrule translate into your specific notion of complex derivative, since getting this wrong will quietly give you wrong results.","category":"page"},{"location":"rule_author/tangents.html#tangents","page":"Tangent types","title":"Tangent types","text":"","category":"section"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"The values that come back from pullbacks or pushforwards are not always the same type as the input/outputs of the primal function. They are tangents, which correspond roughly to something able to represent the difference between two values of the primal types. A tangent might be such a regular type, like a Number, or a Matrix, matching to the original type; or it might be one of the AbstractTangent subtypes.","category":"page"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"Tangents support a number of operations. Most importantly: + and *, which let them act as mathematical objects.","category":"page"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"To be more formal they support operations which let them act as a vector space.","category":"page"},{"location":"rule_author/tangents.html#Operations-on-a-tangent-type","page":"Tangent types","title":"Operations on a tangent type","text":"","category":"section"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"Any tangent type must support:","category":"page"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"zero which returns an additive identity for that type (though it can just return ZeroTangent() (see below))\n+ for addition between two tangents of this primal, returning another tangent of this primal. This allows gradient accumulation.\n* for multiplication (scaling) by a scalar.\n+ between a tangent and its primal type returning another tangent of the primal type – differential geometers sometimes call this exponential map.","category":"page"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"Further they often support other linear operators for convenience in writing rules. ","category":"page"},{"location":"rule_author/tangents.html#The-subtypes-of-AbstractTangent","page":"Tangent types","title":"The subtypes of AbstractTangent","text":"","category":"section"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"Not all tangents need to subtype the AbstractTangent type – in fact most don't: most are just numbers or arrays – but ChainRulesCore does provide a number of special tangent types that can be very useful. ","category":"page"},{"location":"rule_author/tangents.html","page":"Tangent types","title":"Tangent types","text":"ZeroTangent: It is a special representation of 0. It does great things around avoiding expanding Thunks in addition.\nNoTangent: Zero-like, represents that the operation on this input is not differentiable. Its primal type is normally Integer or Bool.\nTangent{P}: this is the tangent for tuples and structs. Use it like a Tuple or NamedTuple. The type parameter P is for the primal type.\nThunk: this is a deferred computation. A thunk is a word for a zero argument closure. A computation wrapped in a @thunk doesn't get evaluated until unthunk is called on the thunk. unthunk is a no-op on non-thunked inputs.\nInplaceableThunk: it is like a Thunk but it can do in-place add! which allows for avoiding allocation during gradient accumulation.","category":"page"},{"location":"maths/arrays.html#Deriving-Array-Rules","page":"Deriving array rules","title":"Deriving Array Rules","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"One of the goals of the ChainRules interface is to make it easy to define your own rules for a function. This tutorial attempts to demystify deriving and implementing custom rules for arrays with real and complex entries, with examples. The approach we use is similar to the one succinctly explained and demonstrated in [Giles2008] and its extended work [Giles2008ext], but we generalize it to support functions of multidimensional arrays with both real and complex entries.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Throughout this tutorial, we will use the following type alias:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"const RealOrComplex = Union{Real,Complex}","category":"page"},{"location":"maths/arrays.html#Forward-mode-rules","page":"Deriving array rules","title":"Forward-mode rules","text":"","category":"section"},{"location":"maths/arrays.html#Approach","page":"Deriving array rules","title":"Approach","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Consider a function","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = f(X::Array{<:RealOrComplex}...)::Array{<:RealOrComplex}","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"or in math notation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"f (ldots X_m ldots) mapsto Omega","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where the components of X_m are written as (X_m)_ildotsj. The variables X_m and Omega are intermediates in a larger program (function) that, by considering only a single real input t and real output s can always be written as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"t mapsto (ldots X_m ldots) mapsto Omega mapsto s","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where t and s are real numbers. If we know the partial derivatives of X_m with respect to t, fracdX_mdt = dotX_m, the chain rule gives the pushforward of f as:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation labelpf\ndotOmega\n = f_*(ldots dotX_m ldots)\n = sum_m sum_i ldots j\n fracpartial Omega partial (X_m)_ildotsj (dotX_m)_ildotsj\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"That's ugly, but in practice we can often write it more simply by using forward mode rules for simpler functions, as we'll see below. The forward-mode rules for arrays follow directly from the usual scalar chain rules.","category":"page"},{"location":"maths/arrays.html#Array-addition","page":"Deriving array rules","title":"Array addition","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = A + B","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"This one is easy:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Omega = A + B","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotOmega = dotA + dotB","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We can implement the frule in ChainRules's notation:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule(\n (_, ΔA, ΔB),\n ::typeof(+),\n A::Array{<:RealOrComplex},\n B::Array{<:RealOrComplex},\n)\n Ω = A + B\n ∂Ω = ΔA + ΔB\n return (Ω, ∂Ω)\nend","category":"page"},{"location":"maths/arrays.html#Matrix-multiplication","page":"Deriving array rules","title":"Matrix multiplication","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = A * B","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Omega = A B","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"First we write in component form:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Omega_ij = sum_k A_ik B_kj","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Then we use the product rule to get the pushforward for each scalar entry:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\ndotOmega_ij\n = sum_k left( dotA_ik B_kj + A_ik dotB_kj right)\n textapply scalar product rule \n fracddt(x y) = fracdxdt y + x fracdydt \n = sum_k dotA_ik B_kj + sum_k A_ik dotB_kj\n textsplit sum\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"But the last expression is just the component form of a sum of matrix products:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequationlabeldiffprod\ndotOmega = dotA B + A dotB\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"This is the matrix product rule, and we write its frule as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule(\n (_, ΔA, ΔB),\n ::typeof(*),\n A::Matrix{<:RealOrComplex},\n B::Matrix{<:RealOrComplex},\n)\n Ω = A * B\n ∂Ω = ΔA * B + A * ΔB\n return (Ω, ∂Ω)\nend","category":"page"},{"location":"maths/arrays.html#Matrix-inversion","page":"Deriving array rules","title":"Matrix inversion","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = inv(A)","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Omega = A^-1","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"It's easiest to derive this rule from either of the two constraints:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nOmega A = A^-1 A = I\nA Omega = A A^-1 = I\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where I is the identity matrix.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We use the matrix product rule to differentiate the first constraint:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotOmega A + Omega dotA = 0","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Then, right-multiply both sides by A^-1 to isolate dotOmega:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign\n0 = dotOmega A A^-1 + Omega dotA A^-1 nonumber\n = dotOmega I + Omega dotA A^-1\n textuse A A^-1 = I nonumber\n = dotOmega + Omega dotA Omega\n textsubstitute A^-1 = Omega nonumber\ndotOmega\n = -Omega dotA Omega\n textsolve for dotOmega labelinvdiff\nendalign","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We write the frule as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule((_, ΔA), ::typeof(inv), A::Matrix{<:RealOrComplex})\n Ω = inv(A)\n ∂Ω = -Ω * ΔA * Ω\n return (Ω, ∂Ω)\nend","category":"page"},{"location":"maths/arrays.html#Other-useful-identities","page":"Deriving array rules","title":"Other useful identities","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"These identities are particularly useful:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nfracddt left( Re(A) right) = Re(dotA)\nfracddt left( A^* right) = dotA^*\nfracddt left( A^mathsfT right) = dotA^mathsfT\nfracddt left( A^mathsfH right) = dotA^mathsfH\nfracddt left( sum_j A_i ldots j ldots k right) =\n sum_j dotA_i ldots j ldots k\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where cdot^* is the complex conjugate (conj), and cdot^mathsfH = left(cdot^mathsfTright)^* is the conjugate transpose (the adjoint function).","category":"page"},{"location":"maths/arrays.html#Reverse-mode-rules","page":"Deriving array rules","title":"Reverse-mode rules","text":"","category":"section"},{"location":"maths/arrays.html#Approach-2","page":"Deriving array rules","title":"Approach","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Reverse-mode rules are a little less intuitive, but we can re-use our pushforwards to simplify their derivation. Recall our program:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"t mapsto (ldots X_m ldots) mapsto Omega mapsto s","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"At any step in the program, if we have intermediates X_m, we can write down the derivative fracdsdt in terms of the tangents dotX_m = fracdX_mdt and adjoints overlineX_m = fracpartial spartial X_m","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nfracdsdt\n = sum_m Releft( sum_ildotsj\n left( fracpartial spartial (X_m)_ildotsj right)^*\n fracd (X_m)_ildotsjdt\n right)\n = sum_m Releft( sum_ildotsj\n (overlineX_m)_ildotsj^*\n (dotX_m)_ildotsj\n right)\n = sum_m Reip overlineX_m dotX_m \nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where Re(cdot) is the real part of a number (real), and ipcdotcdot is the Frobenius inner product (LinearAlgebra.dot). Because this equation follows at any step of the program, we can equivalently write ","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"fracdsdt = Reip overlineOmega dotOmega ","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"which gives the identity","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation labelpbident\nReip overlineOmega dotOmega = sum_m Reip overlineX_m dotX_m \nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"For matrices and vectors, ipAB = tr(A^mathsfH B), and the identity simplifies to:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation labelpbidentmat\nReleft( trleft(\n overlineOmega^mathsfH dotOmega\nright) right) =\nsum_m Re left( tr left(\n overlineX_m^mathsfH dotX_m\nright) right)\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where tr(cdot) is the matrix trace (LinearAlgebra.tr) function. However, it is often cleaner and more general to work with the inner product.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Our approach for deriving the adjoints overlineX_m is then:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Derive the pushforward (dotOmega in terms of dotX_m) using \\eqref{pf}.\nSubstitute this expression for dotOmega into the left-hand side of \\eqref{pbident}.\nManipulate until it looks like the right-hand side of \\eqref{pbident}.\nSolve for each overlineX_m.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Note that the final expressions for the adjoints will not contain any dotX_m terms.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"note: Note\nWhy do we conjugate, and why do we only use the real part of the dot product in \\eqref{pbident}? Recall from Complex Numbers that we treat a complex number as a pair of real numbers. These identities are a direct consequence of this convention. Consider fracdsdt for a scalar function f (x + i y) mapsto (u + i v):beginalign*\nfracdsdt\n = Reip overlinex + i overliney dotx + i doty \n = Releft(\n left( overlinex + i overliney right)^*\n left( dotx + i doty right)\n right) \n = Releft(\n left( overlinex - i overliney right)\n left( dotx + i doty right)\n right) \n = Releft(\n left( overlinex dotx + overliney doty right) +\n i left( overlinex doty - overliney dotx right)\n right)\n = overlinex dotx + overliney doty\nendalign*which is exactly what the identity would produce if we had written the function as f (x y) mapsto (u v).","category":"page"},{"location":"maths/arrays.html#Useful-properties-of-the-inner-product","page":"Deriving array rules","title":"Useful properties of the inner product","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Several properties of the Frobenius inner product come in handy. First, it is linear in its second argument and conjugate linear in its first. That is, for arrays A B C D and scalars a and b,","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign\nipA+BC+D = ipAC + ipBC + ipAD + ipBD labeliplinear\nipaAbB = a^* b ipAB nonumber\nendalign","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Second, swapping arguments is equivalent to conjugating the inner product:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation\nipAB = ipBA^* labelipconj\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Third, for matrices and vectors A, B, and C, we can move arguments from the left or right of one side to the other using the matrix adjoint:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation\nipABCD = ipB^mathsfH ACD = ipB^mathsfH A D^mathsfHC labelipperm\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Fourth, the inner product of two arrays A and B is equivalent to the sum of the elementwise inner products of the two arrays:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginequation\nipAB = sum_ildotsk ipA_ildotskB_ildotsk = sum_ildotsk A_ildotsk^* B_ildotsk\nendequation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"As a result, only elements that are nonzero on both sides contribute to the inner product. This property is especially useful when deriving rules involving structurally sparse arrays.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Now let's derive a few pullbacks using this approach.","category":"page"},{"location":"maths/arrays.html#Matrix-multiplication-2","page":"Deriving array rules","title":"Matrix multiplication","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = A * B","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We above derived in \\eqref{diffprod} the pushforward","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotOmega = dotA B + A dotB","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Using \\eqref{pbidentmat}, we now multiply by overlineOmega^mathsfH and take the real trace:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nReipoverlineOmegadotOmega\n = Re ipoverline OmegadotA B + A dotB\n textsubstitute dotOmega text from eqrefdiffprod\n = Re ipoverline OmegadotA B + Re ipoverline OmegaA dotB\n textexpand using eqrefiplinear \n = Re ipoverline Omega B^mathsfHdotA + Re ipA^mathsfH overline OmegadotB\n textrearrange the left term using eqrefipperm\n = Re ipoverline AdotA + Re ipoverline BdotB\n textright-hand side of eqrefpbidentmat\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"That's it! The expression is in the desired form to solve for the adjoints by comparing the last two lines:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"overline A = overline Omega B^mathsfH qquad overline B = A^mathsfH overline Omega","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Using ChainRules's notation, we would implement the rrule as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function rrule(::typeof(*), A::Matrix{<:RealOrComplex}, B::Matrix{<:RealOrComplex})\n function times_pullback(ΔΩ)\n ∂A = @thunk(ΔΩ * B')\n ∂B = @thunk(A' * ΔΩ)\n return (NoTangent(), ∂A, ∂B)\n end\n return A * B, times_pullback\nend","category":"page"},{"location":"maths/arrays.html#Matrix-inversion-2","page":"Deriving array rules","title":"Matrix inversion","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = inv(A)","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"In \\eqref{invdiff}, we derived the pushforward as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotOmega = -Omega dotA Omega","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Using \\eqref{pbidentmat},","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nReipoverlineOmegadotOmega\n = ReipoverlineOmega-Omega dotA Omega\n textsubstitute eqrefinvdiff\n = Reip-Omega^mathsfH overlineOmega Omega^mathsfHdotA\n textrearrange using eqrefipperm\n = ReipoverlineAdotA\n textright-hand side of eqrefpbidentmat\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"we can now solve for overlineA:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"overlineA = -Omega^mathsfH overlineOmega Omega^mathsfH","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We can implement the resulting rrule as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function rrule(::typeof(inv), A::Matrix{<:RealOrComplex})\n Ω = inv(A)\n function inv_pullback(ΔΩ)\n ∂A = -Ω' * ΔΩ * Ω'\n return (NoTangent(), ∂A)\n end\n return Ω, inv_pullback\nend","category":"page"},{"location":"maths/arrays.html#A-multidimensional-array-example","page":"Deriving array rules","title":"A multidimensional array example","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We presented the approach for deriving pushforwards and pullbacks for arrays of arbitrary dimensions, so let's cover an example. For multidimensional arrays, it's often easier to work in component form. Consider the following function:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Ω = sum(abs2, X::Array{<:RealOrComplex,3}; dims=2)::Array{<:Real,3}","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"which we write as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Omega_i1k = sum_j X_ijk^2\n = sum_j Re ipX_ijkX_ijk","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"The pushforward from \\eqref{pf} is","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign\ndotOmega_i1k\n = sum_j ReipdotX_ijkX_ijk + ipX_ijkdotX_ijk nonumber\n = sum_j ReipX_ijkdotX_ijk^* + ipX_ijkdotX_ijk nonumber\n = sum_j 2 ReipX_ijkdotX_ijk labelsumabspf\nendalign","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where in the last step we have used the fact that for all real a and b,","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"(a + i b) + (a + i b)^*\n = (a + i b) + (a - i b)\n = 2 a\n = 2 Re (a + i b)","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Because none of this derivation depended on the index (or indices), we implement frule generically as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule(\n (_, _, ΔX),\n ::typeof(sum),\n ::typeof(abs2),\n X::Array{<:RealOrComplex};\n dims = :,\n)\n Ω = sum(abs2, X; dims = dims)\n ∂Ω = sum(2 .* real.(conj.(X) .* ΔX); dims = dims)\n return (Ω, ∂Ω)\nend","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We can now derive the reverse-mode rule. The elementwise form of \\eqref{pbident} is","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nReip overlineOmega dotOmega \n = Re left( sum_ik overlineOmega_i1k^*\n dotOmega_i1k right)\n textexpand left-hand side of eqrefpbident\n = Re left(sum_ijk overlineOmega_i1k^*\n 2 Releft( X_ijk^* dotX_ijk right)\n right)\n textsubstitute eqrefsumabspf\n = Re left( sum_ijk\n left(\n 2 Re left( overlineOmega_i1k right)\n X_ijk^*\n right) dotX_ijk\n right)\n textbring dotX_ijk text outside of Re\n = sum_ijk Reip2 Re left( overlineOmega_i1k right) X_ijkdotX_ijk\n textrewrite as an inner product\n = sum_ijk ReipoverlineX_ijkdotX_i1k\n textright-hand side of eqrefpbident\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We now solve for overlineX:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"overlineX_ijk = 2Re left( overlineOmega_i1k right) X_ijk","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Like the frule, this rrule can be implemented generically:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function rrule(::typeof(sum), ::typeof(abs2), X::Array{<:RealOrComplex}; dims = :)\n function sum_abs2_pullback(ΔΩ)\n ∂abs2 = NoTangent()\n ∂X = @thunk(2 .* real.(ΔΩ) .* X)\n return (NoTangent(), ∂abs2, ∂X)\n end\n return sum(abs2, X; dims = dims), sum_abs2_pullback\nend","category":"page"},{"location":"maths/arrays.html#Functions-that-return-a-tuple","page":"Deriving array rules","title":"Functions that return a tuple","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Every Julia function returns a single output. For example, let's look at LinearAlgebra.logabsdet, the logarithm of the absolute value of the determinant of a matrix, which returns log det(A) and operatornamesign(det A) = fracdet A det A :","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"(l, s) = logabsdet(A)","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"The return type is actually a single output, a tuple of scalars, but when deriving, we treat them as multiple outputs. The left-hand side of \\eqref{pbident} then becomes a sum over terms, just like the right-hand side.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Let's derive the forward- and reverse-mode rules for logabsdet.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nl = log det(A)\ns = operatornamesign(det(A))\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"where operatornamesign(x) = fracxx.","category":"page"},{"location":"maths/arrays.html#Forward-mode-rule","page":"Deriving array rules","title":"Forward-mode rule","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"To make this easier, let's break the computation into more manageable steps:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nd = det(A)\na = d = sqrtRe left( d^* d right)\nl = log a\ns = fracda\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We'll make frequent use of the identities:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"d = a s","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"s^* s = fracd^* da^2 = fraca^2a^2 = 1","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"It will also be useful to define b = trleft( A^-1 dotA right).","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"For dotd, we use the pushforward for the determinant given in section 2.2.4 of [Giles2008ext]:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotd = d b","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Now we'll compute the pushforwards for the remaining steps.","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\ndota = frac12 a fracddt\n Releft( d^* d right)\n = frac22 a Re left( d^* dotd right)\n = Re left( s^* dotd right)\n textuse d = a s \n = Re left( s^* d b right)\n textsubstitute dotd \ndotl = a^-1 dota\n = a^-1 Re left( s^* d b right)\n textsubstitute dota\n = Re left( s^* s b right)\n textuse d = a s \n = Re left(b right)\n textuse s^* s = 1\ndots = a^-1 dotd - a^-2 d dota\n = a^-1 left( dotd - dota s right)\n textuse d = a s \n = a^-1 left(\n dotd - Re left( s^* dotd right) s\n right)\n textsubstitute dota\n = a^-1 left(\n dotd - left(\n s^* dotd -\n i Im left( s^* dotd right)\n right) s\n right)\n textuse Re(x) = x - i Im(x)\n = a^-1 left(\n dotd - left( s^* s right) dotd +\n i Im left( s^* dotd right) s \n right)\n = i a^-1 Im left( s^* dotd right) s\n textuse s^* s = 1\n = i a^-1 Im left( s^* d b right) s\n textsubstitute dotd\n = i Im left( s^* s b right) s\n textuse d = a s \n = i Im(b) s\n textuse s^* s = 1\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Note that the term b is reused. In summary, after all of that work, the final pushforward is quite simple:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign\nb = tr left( A^-1 dotA right) labellogabsdet_b \ndotl = Re(b) labellogabsdet_ldot\ndots = i Im(b) s labellogabsdet_sdot\nendalign","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We can define the frule as:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule((_, ΔA), ::typeof(logabsdet), A::Matrix{<:RealOrComplex})\n # The primal function uses the lu decomposition to compute logabsdet\n # we reuse this decomposition to compute inv(A) * ΔA\n F = lu(A, check = false)\n Ω = logabsdet(F) # == logabsdet(A)\n b = tr(F \\ ΔA) # == tr(inv(A) * ΔA)\n s = last(Ω)\n ∂l = real(b)\n # for real A, ∂s will always be zero (because imag(b) = 0)\n # this is type-stable because the eltype is known\n ∂s = eltype(A) <: Real ? ZeroTangent() : im * imag(b) * s\n # tangents of tuples are of type Tangent{<:Tuple}\n ∂Ω = Tangent{typeof(Ω)}(∂l, ∂s)\n return (Ω, ∂Ω)\nend","category":"page"},{"location":"maths/arrays.html#Reverse-mode-rule","page":"Deriving array rules","title":"Reverse-mode rule","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\nReipoverlineldotl + Reipoverlinesdots\n textleft-hand side of eqrefpbidentmat\n= Releft( overlinel^* dotl + overlines^* dots right) \n= Releft( \n overlinel^* Re(b) + i overlines^* s Im(b)\n right)\n textsubstitute eqreflogabsdet_ldot text and eqreflogabsdet_sdot \n= Releft( \n Releft( overlinel right) Re(b) -\n Im left( overlines^* s right) Im(b)\n right)\n textdiscard imaginary parts \n= Releft(\n left(\n Re left( overlinel right) +\n i Im left( overlines^* s right)\n right) b\n right)\n textgather parts of b \n= Releft(\n left(\n Re left( overlinel right) +\n i Im left( overlines^* s right)\n right)\n tr(A^-1 dotA)\n right)\n textsubstitute b text from eqreflogabsdet_b \n= Releft( tr left(\n left(\n Re left( overlinel right) +\n i Im left( overlines^* s right)\n right)\n A^-1 dotA\n right) right)\n textbring scalar within tr \n= Reip\n left(\n Re left( overlinel right) + i Im left( s^* overlines right)\n right) A^-mathsfH\n dotA textrewrite as inner product\n= ReipoverlineAdotA textright-hand side of eqrefpbidentmat\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Now we solve for overlineA:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\noverlineA = left(\n Re left( overlinel right) +\n i Im left( s^* overlines right)\nright) A^-mathsfH\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"The rrule can be implemented as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function rrule(::typeof(logabsdet), A::Matrix{<:RealOrComplex})\n # The primal function uses the lu decomposition to compute logabsdet\n # we reuse this decomposition to compute inv(A)\n F = lu(A, check = false)\n Ω = logabsdet(F) # == logabsdet(A)\n s = last(Ω)\n function logabsdet_pullback(ΔΩ)\n (Δl, Δs) = ΔΩ\n f = conj(s) * Δs\n imagf = f - real(f) # 0 for real A and Δs, im * imag(f) for complex A and/or Δs\n g = real(Δl) + imagf\n ∂A = g * inv(F)' # == g * inv(A)'\n return (NoTangent(), ∂A)\n end\n return (Ω, logabsdet_pullback)\nend","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"note: Note\nIt's a good idea when deriving pushforwards and pullbacks to verify that they make sense. For the pushforward, since l is real, it follows that dotl is too.What about dots? Well, s = fracdd is point on the unit circle in the complex plane. Multiplying a complex number by i rotates it counter-clockwise by 90°. So the expression for dots takes a real number, Im(b), multiplies by s to make it parallel to s, then multiplies by i to make it perpendicular to s, that is, perfectly tangent to the unit complex circle at s.For the pullback, it again follows that only the real part of overlinel is pulled back.s^* rotates a number parallel to s to the real line. So s^* overlines rotates overlines so that its imaginary part is the part that was tangent to the complex circle at s, while the real part is the part that was not tangent. Then the pullback isolates the imaginary part, which effectively is a projection. That is, any part of the adjoint overlines that is not tangent to the complex circle at s will not contribute to overlineA.","category":"page"},{"location":"maths/arrays.html#Implicit-functions","page":"Deriving array rules","title":"Implicit functions","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Sometimes a function is only defined implicitly, and internally some solver or iterative algorithm is used to compute the result. We can still in some cases derive rules by considering only the implicit functions and not the internals. One example is the solution X to the Sylvester equation","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"A X + X B = -C","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"for inputs A, B, and C. We can also write this solution as X = operatornamesylvester(A B C), which in Julia is computed using LinearAlgebra.sylvester(A, B, C).","category":"page"},{"location":"maths/arrays.html#Forward-mode-Rule","page":"Deriving array rules","title":"Forward-mode Rule","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"We start by differentiating the implicit function:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotA X + A dotX + dotX B + X dotB = -dotC","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Then we isolate the terms with dotX on one side:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign\nA dotX + dotX B\n = -dotC - dotA X - X dotB labelsylpfimplicit\n = -(dotC + dotA X + X dotB) nonumber\nendalign","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"So the pushforward is the solution to a different Sylvester equation:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"dotX = operatornamesylvester(A B dotC + dotA X + X dotB)","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"The frule can be implemented as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function frule((_, ΔA, ΔB, ΔC), ::typeof(sylvester), A, B, C)\n X = sylvester(A, B, C)\n return X, sylvester(A, B, ΔC + ΔA * X + X * ΔB)\nend","category":"page"},{"location":"maths/arrays.html#Reverse-mode-Rule","page":"Deriving array rules","title":"Reverse-mode Rule","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Like with the pushforward, it's easiest to work with the implicit function. We start by introducing some dummy -Z and taking its inner product with both sides of \\eqref{sylpfimplicit}:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"ip-ZA dotX + dotX B = ip-Z-dotC - dotA X - X dotB","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Then we expand","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"ip-ZA dotX + ip-ZdotX B = ipZdotC + ipZdotA X + ipZX dotB","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Now permute:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"ip-A^mathsfH ZdotX + ip-Z B^mathsfHdotX = ipZdotC + ipZ X^mathsfHdotA + ipX^mathsfH ZdotB","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Then combine:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"ip-(A^mathsfH Z + Z B^mathsfH)dotX = ipZ X^mathsfHdotA + ipX^mathsfH ZdotB + ipZdotC","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"This is almost exactly the identity we need to solve for overlineA, overlineB, and overlineC. To manipulate it to the right form, we need only define A^mathsfH Z + Z B^mathsfH = -overlineX. This yet another Sylvester equation, so letting Z = overlineC, our final pullback is:","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"beginalign*\noverlineC = operatornamesylvester(A^mathsfH B^mathsfH overlineX)\n = operatornamesylvester(B A overlineX^mathsfH)^mathsfH\noverlineA = overlineC X^mathsfH\noverlineB = X^mathsfH overlineC\nendalign*","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"The rrule can be implemented as","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"function rrule(::typeof(sylvester), A, B, C)\n X = sylvester(A, B, C)\n function sylvester_pullback(ΔX)\n ∂C = copy(sylvester(B, A, copy(ΔX'))')\n return NoTangent(), @thunk(∂C * X'), @thunk(X' * ∂C), ∂C\n end\n return X, sylvester_pullback\nend","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"Note, however, that the Sylvester equation is usually solved using the Schur decomposition of A and B. These Schur decompositions can be reused to solve the Sylvester equations in the pushforward and pullback. See the implementation in ChainRules for details.","category":"page"},{"location":"maths/arrays.html#More-examples","page":"Deriving array rules","title":"More examples","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"For more instructive examples of array rules, see [Giles2008ext] (real vector and matrix rules) and the LinearAlgebra rules in ChainRules. For differentiating the LU decomposition, see this blog post by Seth Axen.","category":"page"},{"location":"maths/arrays.html#References","page":"Deriving array rules","title":"References","text":"","category":"section"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"[Giles2008]: Giles M. B. Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation. In: Advances in Automatic Differentiation. Lecture Notes in Computational Science and Engineering, vol 64: pp 35-44. Springer, Berlin (2008). doi: 10.1007/978-3-540-68942-3_4. pdf","category":"page"},{"location":"maths/arrays.html","page":"Deriving array rules","title":"Deriving array rules","text":"[Giles2008ext]: Giles M. B. An Extended Collection of Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation. (unpublished). pdf","category":"page"},{"location":"rule_author/testing.html#Testing-your-rules","page":"Testing your rules","title":"Testing your rules","text":"","category":"section"},{"location":"rule_author/testing.html","page":"Testing your rules","title":"Testing your rules","text":"ChainRulesTestUtils.jl provides tools for writing tests based on FiniteDifferences.jl. Take a look at the documentation or the existing ChainRules.jl tests to see how to write the tests.","category":"page"},{"location":"rule_author/testing.html","page":"Testing your rules","title":"Testing your rules","text":"warning: Warning\nDon't use analytical derivations for derivatives in the tests. Those are what you use to define the rules, and so cannot be confidently used in the test. If you misread/misunderstood them, then your tests/implementation will have the same mistake. Use finite differencing methods instead, as they are based on the primal computation.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html#grad_acc","page":"Gradient accumulation","title":"Gradient Accumulation","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"Consider some function f(x) = g(x) + h(x). If we would like the derivative of f with respect to x we must compute it for each part and then sum them, i.e. fracpartial fpartial x = fracpartial gpartial x + fracpartial hpartial x. In general, we must accumulate (sum) gradients from each sub-part of a program where a variable is used.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"Consider for example:","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"function sum_first_and_second(X::Array{Float64})\n a = X[1]\n b = X[2]\n y = a + b\n return y\nend","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"The AD software must transform that into something which repeatedly sums up the gradient of each part: X̄ = ā + b̄.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"This requires that all tangent types D must implement +: +(::D, ::D)::D.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"We can note that in this particular case ā and b̄ will both be arrays. This operation (X̄ = ā + b̄) will allocate one array to hold ā, another one to hold b̄, and a third one to hold ā + b̄. This is three allocations. Allocations are not free, they increase the time the program takes to run by a nontrivial amount, even with a good allocator and a good garbage collector.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html#Maybe-mutating-accumulation-(add!!)","page":"Gradient accumulation","title":"Maybe-mutating accumulation (add!!)","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"We can note that in the above that neither ā nor b̄ are ever used again after accumulating to get X̄. Furthermore, Arrays are mutable. That means we could over-write either ā or b̄ and use the result as X̄:","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"ā .+= b̄\nX̄ = ā","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"This cuts our allocations down to 2, just ā and b̄.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"However, we have a bit of a problem that not all types are mutable, so this pattern is hard to apply in general. To deal with that ChainRulesCore provides add!!. Per the BangBang.jl convention, this is a maybe mutating addition. It may mutate its first argument (if it is mutable), but it will definitely return the correct result. We would write using that as X̄ = add!!(ā, b̄): which would in this case give us just 2 allocations. AD systems can generate add!! instead of + when accumulating gradient to take advantage of this.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html#Inplaceable-Thunks-(InplaceableThunks)-avoid-allocating-values-in-the-first-place.","page":"Gradient accumulation","title":"Inplaceable Thunks (InplaceableThunks) avoid allocating values in the first place.","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"We got down to two allocations from using add!!, but can we do better? We can think of having a tangent type which acts on a partially accumulated result, to mutate it to contain its current value plus the partial derivative being accumulated. Rather than having an actual computed value, we can just have a thing that will act on a value to perform the addition. Let's illustrate it with our example.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"b̄ is the partial for X[2] and its value can be computed by:","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"b̄ = zeros(size(X))\nb̄[2] = ȳ # the scalar sensitivity of the `sum_first_and_second` output","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"b̄ is a matrix entirely of zeros, except for at the index 2, where it is set to the output sensitivity ȳ. ā is similar, except with the non-zero at index 1.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"What is the action of b̄ upon ā, to get the same result as X̄ = add!!(ā, b̄) (or X̄ = ā + b̄ for that matter)? It is:","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"function b̄_add!(ā)\n ā[2] += ȳ\n return ā\nend","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"We don't need to worry about all those zeros since x + 0 == x.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"InplaceableThunk is the type we have to represent derivatives as gradient accumulating actions. We must note that to do this we do need a value form of ā for b̄ to act upon. For this reason every inplaceable thunk has both a val field holding the value representation, and a add! field holding the action representation. The val field use a plain Thunk to avoid the computation (and thus allocation) if it is unused.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"note: Do we need both representations?\nRight now every InplaceableThunk has two fields that need to be specified. The value form (represented as a the Thunk typed field), and the action form (represented as the add! field). It is possible in a future version of ChainRulesCore.jl we will work out a clever way to find the zero tangent for arbitrary primal values. Given that, we could always just determine the value form from inplaceable.add!(zero_tangent(primal)). There are some technical difficulties in finding the zero tangents, but this may be solved at some point.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"The + operation on InplaceableThunks is overloaded to unthunk that val field to get the value form. Where as the add!! operation is overloaded to call add! to invoke the action.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"With getindex defined to return an InplaceableThunk, we now get to X̄ = add!!(ā, b̄) requires only a single allocation. This allocation occurs when unthunking ā, which is then mutated to become X̄. This is basically as good as we can get: if we want X̄ to be an Array then at some point we need to allocate that array.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"note: Can we do more? Deferred accumulation\nWe could keep going further to drop allocations if we really wanted. If we didn't care about X̄ being an Array then we could defer its computation too. X̄ = @thunk add!!(ā, b̄). This kind of deferral will work fine and you can keep chaining it. It does start to burn stack space, and might make the compiler's optimization passes cry. But it's valid and should work fine.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html#Examples-of-InplaceableThunks","page":"Gradient accumulation","title":"Examples of InplaceableThunks","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html#getindex","page":"Gradient accumulation","title":"getindex","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"The aforementioned getindex is really the poster child for this. Consider something like:","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"function mysum(X::Array{Float64})\n total = 0.0\n for i in eachindex(X)\n total += X[i]\n end\n return total\nend","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"If one only has value representation of derivatives one ends up having to allocate a derivative array for every single element of the original array X. That's terrible. On the other hand, with the action representation that InplaceableThunks provide, there is just a single Array allocated. One can see the getindex rule in ChainRules.jl for the implementation.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html#matmul-etc-(*)","page":"Gradient accumulation","title":"matmul etc (*)","text":"","category":"section"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"Multiplication of scalars/vectors/matrices of compatible dimensions can all also have their derivatives represented as an InplaceableThunk. These tend to pivot around that add! action being defined along the lines of: X̄ -> mul!(X̄, A', Ȳ, true, true). Where 5-arg mul! is the in place multiply-add operation. mul!(X̄, A', Ȳ, true, true) has the same effect as (X̄ .+= A'*Ȳ) but avoids allocating the matrix A'*Ȳ This is one of the fundamental operations provided by BLAS – including the application of the conjugate transpose. e.g. the Matrix-Matrix form is GEMM (GEneralized Matrix-Matrix Multiplication), the Matrix-Vector form is GEMV (GEneralized Matrix-Vector Multiplication) etc. Under the hood doing it out of place is going to call one of these methods anyway, but on a freshly allocated output array. So we are going to hit a very efficient implementation and get the addition for free.","category":"page"},{"location":"rule_author/superpowers/gradient_accumulation.html","page":"Gradient accumulation","title":"Gradient accumulation","text":"One can see the * rules in ChainRules.jl for the implementations","category":"page"},{"location":"FAQ.html#FAQ","page":"FAQ","title":"FAQ","text":"","category":"section"},{"location":"FAQ.html#What-is-up-with-the-different-symbols?","page":"FAQ","title":"What is up with the different symbols?","text":"","category":"section"},{"location":"FAQ.html#Δx,-x,-dx","page":"FAQ","title":"Δx, ∂x, dx","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"ChainRules uses these perhaps atypically. As a notation that is the same across propagators, regardless of direction (in contrast see ẋ and x̄ below).","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Δx is the input to a propagator, (i.e a seed for a pullback; or a perturbation for a pushforward).\n∂x is the output of a propagator.\ndx could be either input or output.","category":"page"},{"location":"FAQ.html#dots-and-bars:-\\dot{y}-\\dfrac{y}{x}-\\overline{x}","page":"FAQ","title":"dots and bars: doty = dfracyx = overlinex","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"v̇ is a derivative of the input moving forward: v = fracvx for input x, intermediate value v.\nv̄ is a derivative of the output moving backward: v = fracyv for output y, intermediate value v.","category":"page"},{"location":"FAQ.html#others","page":"FAQ","title":"others","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Ω is often used as the return value of the function. Especially, but not exclusively, for scalar functions.\nΔΩ is thus a seed for the pullback.\n∂Ω is thus the output of a pushforward.","category":"page"},{"location":"FAQ.html#Why-does-rrule-return-the-primal-function-evaluation?","page":"FAQ","title":"Why does rrule return the primal function evaluation?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"You might wonder why frule(f, x) returns f(x) and the derivative of f at x, and similarly for rrule returning f(x) and the pullback for f at x. Why not just return the pushforward/pullback, and let the user call f(x) to get the answer separately?","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"There are three reasons the rules also calculate the f(x).","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"For some rules an alternative way of calculating f(x) can give the same answer while also generating intermediate values that can be used in the calculations required to propagate the derivative.\nFor many rrules the output value is used in the definition of the pullback. For example tan, sigmoid etc.\nFor some frules there exists a single, non-separable operation that will compute both derivative and primal result. For example, this is the case for many of the methods for differential equation sensitivity analysis.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"For more information and examples see the design notes on changing the primal.","category":"page"},{"location":"FAQ.html#Where-are-the-derivatives-for-keyword-arguments?","page":"FAQ","title":"Where are the derivatives for keyword arguments?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Pullbacks do not return a sensitivity for keyword arguments; similarly, pushforwards do not accept a perturbation for keyword arguments. This is because in practice functions are very rarely differentiable with respect to keyword arguments.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"As a rule, keyword arguments tend to control side-effects, like logging verbosity, or to be functionality-changing to perform a different operation, e.g. dims=3, and thus not differentiable.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"To the best of our knowledge no Julia AD system, with support for the definition of custom primitives, supports differentiating with respect to keyword arguments. At some point in the future ChainRules may support these. Maybe.","category":"page"},{"location":"FAQ.html#faq_abstract_zero","page":"FAQ","title":"What is the difference between ZeroTangent and NoTangent ?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"ZeroTangent and NoTangent act almost exactly the same in practice: they result in no change whenever added to anything. Odds are if you write a rule that returns the wrong one everything will just work fine. We provide both to allow for clearer writing of rules, and easier debugging.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"ZeroTangent() represents the fact that if one perturbs (adds a small change to) the matching primal, there will be no change in the behaviour of the primal function. For example, in fst(x, y) = x, the derivative of fst with respect to y is ZeroTangent(). fst(10, 5) == 10 and if we add 0.1 to 5 we still get fst(10, 5.1) == 10.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"NoTangent() represents the fact that if one perturbs the matching primal, the primal function will now error. For example, in access(xs, n) = xs[n], the derivative of access with respect to n is NoTangent(). access([10, 20, 30], 2) == 20, but if we add 0.1 to 2 we get access([10, 20, 30], 2.1) which errors as indexing can't be applied at fractional indexes.","category":"page"},{"location":"FAQ.html#Why-do-I-get-an-error-involving-nothing?","page":"FAQ","title":"Why do I get an error involving nothing?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"When no custom frule or rrule exists, if you try to call one of those, it will return nothing by default. As a result, you may encounter errors like","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"MethodError: no method matching iterate(::Nothing)","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Sometimes you think you have implemented the right rule, but it is called with a slightly different set of arguments than you expected. You can use Cthulhu.jl to dive into the call stack and figure out which method you are missing.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"An alternative is to call back into AD: read the documentation on rule configuration to know more.","category":"page"},{"location":"FAQ.html#When-to-use-ChainRules-vs-ChainRulesCore?","page":"FAQ","title":"When to use ChainRules vs ChainRulesCore?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"ChainRulesCore.jl is a light-weight dependency for defining rules for functions in your packages, without you needing to depend on ChainRules.jl itself. It has almost no dependencies of its own. If you only want to define rules, not use them, then you probably only want to load ChainRulesCore.jl.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"ChainRules.jl provides the full functionality for AD systems. In particular, it has all the rules for Base Julia and the standard libraries. It is thus a much heavier package to load. AD systems making use of frules and rrules should load ChainRules.jl.","category":"page"},{"location":"FAQ.html#Where-should-I-put-my-rules?","page":"FAQ","title":"Where should I put my rules?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"We recommend adding custom rules to your own packages with ChainRulesCore.jl. It is good to have them in the same package that defines the original function. This avoids type-piracy, and makes it easy to keep in-sync. ChainRulesCore is a very light-weight dependency.","category":"page"},{"location":"FAQ.html#How-do-I-test-my-rules?","page":"FAQ","title":"How do I test my rules?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"You can use ChainRulesTestUtils.jl to test your custom rules. ChainRulesTestUtils.jl has some dependencies, so it is a separate package from ChainRulesCore.jl. This means your package can depend on the light-weight ChainRulesCore.jl, and make ChainRulesTestUtils.jl a test-only dependency.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Remember to read the section On writing good rrule / frule methods.","category":"page"},{"location":"FAQ.html#Is-removing-a-thunk-a-breaking-change?","page":"FAQ","title":"Is removing a thunk a breaking change?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Removing thunks is not considered a breaking change. This is because (in principle) removing them changes the implementation of the values returned by an rrule, not the value that they represent. This is morally the same as similar issues discussed in ColPrac, such as details of floating point arithmetic changing.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"On a practical level, it's important that this is the case because thunks are a bit of a hack, and over time it is hoped that the need for them will reduce, as they increase code-complexity and place additional stress on the compiler.","category":"page"},{"location":"FAQ.html#Where-can-I-learn-more-about-AD-?","page":"FAQ","title":"Where can I learn more about AD ?","text":"","category":"section"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"There are not so many truly excellent learning resources for autodiff out there in the world, which is a bit sad. The list here is incomplete, but is vetted for quality.","category":"page"},{"location":"FAQ.html","page":"FAQ","title":"FAQ","text":"Automatic Differentiation for Dummies keynote video by Simon Peyton Jones: particularly good if you like pure math type thinking.\n\"What types work with differentiation? comment on DexLang GitHub issue by Dan Zheng: summarizes several years of insights from the Swift AD work.\nMIT 18337 lecture notes 8-10 by Christopher Rackauckas and David P. Sanders: moves fast from basic to advanced, particularly good if you like applicable mathematics\nAutomatic Differentiation and Application: Good introduction\nSolving Stiff Ordinary Differential Equations: ignore the ODE stuff, most of this is about Sparse AutoDiff, can skip/skim this one\nBasic Parameter Estimation, Reverse-Mode AD, and Inverse Problems: use in optimization, and details connections of other math.\nDifferentiable Programming and Neural Differential Equations: Includes custom primitive derivations for equation solvers.\nDiff-Zoo Jupyter Notebook Book by Mike Innes, has implementations and explanations.\n\"Evaluating Derivatives\" by Griewank and Walther is the best book at least for reverse-mode. It also covers forward-mode though (by its own admission) not as well, it never mentioned dual numbers which is an unfortunate lack.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html#Mutation-Support","page":"Mutation Support (experimental)","title":"Mutation Support","text":"","category":"section"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"ChainRulesCore.jl offers experimental support for mutation, targeting use in forward mode AD. (Mutation support in reverse mode AD is more complicated and will likely require more changes to the interface)","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"warning: Experimental\nThis page documents an experimental feature. Expect breaking changes in minor versions while this remains. It is not suitable for general use unless you are prepared to modify how you are using it each minor release. It is thus suggested that if you are using it to use tilde bounds on supported minor versions.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html#MutableTangent","page":"Mutation Support (experimental)","title":"MutableTangent","text":"","category":"section"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"The MutableTangent type is designed to be a partner to the Tangent type, with specific support for being mutated in place. It is required to be a structural tangent, having one tangent for each field of the primal object.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"Technically, not all mutable structs need to use MutableTangent to represent their tangents. Just like not all structs need to use Tangents. Common examples away from this are natural tangent types like for arrays. However, if one is setting up to use a custom tangent type for this it is sufficiently off the beaten path that we can not provide much guidance.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html#zero_tangent","page":"Mutation Support (experimental)","title":"zero_tangent","text":"","category":"section"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"The zero_tangent function functions to give you a zero (i.e. additive identity) for any primal value. The ZeroTangent type also does this. The difference is that zero_tangent is in general full structural tangent mirroring the structure of the primal. To be technical the promise of zero_tangent is that it will be a value that supports mutation. However, in practice[1] this is achieved through in a structural tangent For mutation support this is important, since it means that there is mutable memory available in the tangent to be mutated when the primal changes. To support this you thus need to make sure your zeros are created in various places with zero_tangent rather than []ZeroTangent](@ref).","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"It is also useful for reasons of type stability, since it forces a consistent type (generally a structural tangent) for any given primal type. For this reason AD system implementors might chose to use this to create the tangent for all literal values they encounter, mutable or not, and to process the output of frules to convert ZeroTangent into corresponding zero_tangents.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html#Writing-a-frule-for-a-mutating-function","page":"Mutation Support (experimental)","title":"Writing a frule for a mutating function","text":"","category":"section"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"It is relatively straight forward to write a frule for a mutating function. There are a few key points to follow:","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"There must be a mutable tangent input for every mutated primal input\nWhen the primal value is changed, the corresponding change must be made to its tangent partner\nWhen a value is returned, return its partnered tangent.\nIf (and only if) primal values alias, then their tangents must also alias.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html#Example","page":"Mutation Support (experimental)","title":"Example","text":"","category":"section"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"For example, consider the primal function with:","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"takes two Refs\ndoubles the first one in place\noverwrites the second one's value with the literal 5.0\nreturns the first one","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"function foo!(a::Base.RefValue, b::Base.RefValue)\n a[] *= 2\n b[] = 5.0\n return a\nend","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"The frule for this would be:","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"function ChainRulesCore.frule((_, ȧ, ḃ), ::typeof(foo!), a::Base.RefValue, b::Base.RefValue)\n @assert ȧ isa MutableTangent{typeof(a)}\n @assert ḃ isa MutableTangent{typeof(b)}\n\n a[] *= 2\n ȧ.x *= 2 # `.x` is the field that lives behind RefValues\n\n b[] = 5.0\n ḃ.x = zero_tangent(5.0) # or since we know that the zero for a Float64 is zero could write `ḃ.x = 0.0`\n\n return a, ȧ\nend","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"Then assuming the AD system does its part to makes sure you are indeed given mutable values to mutate (i.e. those @assertions are true) then all is well and this rule will make mutation correct.","category":"page"},{"location":"rule_author/superpowers/mutation_support.html","page":"Mutation Support (experimental)","title":"Mutation Support (experimental)","text":"[1]: Further, it is hard to achieve this promise of allowing mutation to be supported without returning a structural tangent. Except in the special case of where the struct is not mutable and has no nested fields that are mutable.","category":"page"},{"location":"rule_author/tips_for_packages.html#Tips-for-making-your-package-work-with-AD","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"","category":"section"},{"location":"rule_author/tips_for_packages.html#Ignoring-gradients-for-certain-expressions","page":"Tips for making your package work with AD","title":"Ignoring gradients for certain expressions","text":"","category":"section"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"There exists code that is not meant to be differentiated through, for example logging. In some cases, AD systems might work perfectly well with that code, but in others they might not. A convenience function ignore_derivatives is provided to get around this issue. It captures the functionality of both Zygote.ignore and Zygote.dropgrad.","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"For example, Zygote does not support mutation, so it will break if you try to store intermediate values as in the following example:","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"somes = []\nthings = []\n\nfunction loss(x, y)\n some = f(x, y)\n thing = g(x)\n \n # log\n push!(somes, some)\n push!(things, thing)\n\n return some + thing\nend","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"It is possible to get around this by using the ignore_derivatives function","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"ignore_derivatives() do\n push!(somes, some)\n push!(things, thing)\nend","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"or using a macro for one-liners","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"@ignore_derivatives push!(things, thing)","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"It is also possible to use this on individual objects, e.g.","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"ignore_derivatives(a) + b","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"will ignore the gradients for a only.","category":"page"},{"location":"rule_author/tips_for_packages.html","page":"Tips for making your package work with AD","title":"Tips for making your package work with AD","text":"Passing in instances of functors (callable structs), ignore_derivatives(functor), will make them behave like normal structs, i.e. propagate without being called and dropping their gradients. If you want to call a functor in the primal computation, wrap it in a closure: ignore_derivatives(() -> functor())","category":"page"},{"location":"rule_author/which_functions_need_rules.html#Which-functions-need-rules?","page":"Which functions need rules?","title":"Which functions need rules?","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"In principle, a perfect AD system only needs rules for basic operations and can infer the rules for more complicated functions automatically. In practice, performance needs to be considered as well.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Some functions use ccall internally, for example ^. These functions cannot be differentiated through by AD systems, and need custom rules.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Other functions can in principle be differentiated through by an AD system, but there exists a mathematical insight that can dramatically improve the computation of the derivative. An example is numerical integration, where writing a rule implementing the fundamental theorem of calculus removes the need to perform AD through numerical integration.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Furthermore, AD systems make different trade-offs in performance due to their design. This means that a certain rule will help one AD system, but not improve (and also not harm) another. Below, we list some patterns relevant for the Zygote.jl AD system.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Rules for functions which mutate its arguments, e.g. sort!, should not be written at the moment. While technically they are supported, they would break Zygote.jl such that it would sometimes quietly return the wrong answer. This may be resolved in the future by allowing AD systems to opt-in or opt-out of certain types of rules.","category":"page"},{"location":"rule_author/which_functions_need_rules.html#Patterns-that-need-rules-in-[Zygote.jl](https://github.com/FluxML/Zygote.jl)","page":"Which functions need rules?","title":"Patterns that need rules in Zygote.jl","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"There are a few classes of functions that Zygote cannot differentiate through. Custom rules will need to be written for these to make AD work.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Other patterns can be AD'ed through, but the backward pass performance can be greatly improved by writing a rule.","category":"page"},{"location":"rule_author/which_functions_need_rules.html#Functions-which-mutate-arrays","page":"Which functions need rules?","title":"Functions which mutate arrays","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"For example,","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function addone(a::AbstractArray)\n b = similar(a)\n b .= a .+ 1\n return sum(b)\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"complains that","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> using Zygote\njulia> gradient(addone, a)\nERROR: Mutating arrays is not supported","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"However, upon adding the rrule (restart the REPL after calling gradient)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function ChainRules.rrule(::typeof(addone), a)\n y = addone(a)\n function addone_pullback(ȳ)\n return NoTangent(), ones(length(a))\n end\n return y, addone_pullback\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"the gradient can be evaluated:","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> gradient(addone, a)\n([1.0, 1.0, 1.0],)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Notice that addone(a) mutates another array b internally, but not its input. This is commonly done in less trivial functions, and is often what Zygote's Mutating arrays is not supported error is telling you, even though you did not intend to mutate anything. Functions which mutate their own input are much more problematic. These are the ones named (by convention) with an exclamation mark, such as fill!(a, x) or push!(a, x). It is not possible to write rules which handle all uses of such a function correctly, on current Zygote.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"note: Why restarting REPL after calling `gradient`?\nWhen gradient is called in Zygote for a function with no rrule defined, a backward pass for the function call is generated and cached. When gradient is called for the second time on the same function signature, the backward pass is reused without checking whether an an rrule has been defined between the two calls to gradient.If an rrule is defined before the first call to gradient it should register the rule and use it, but that prevents comparing what happens before and after the rrule is defined. To compare both versions with and without an rrule in the REPL simultaneously, define a function f(x) = (no rrule), another function f_cr(x) = f(x), and an rrule for f_cr.Calling Zygote.refresh() will often have the same effect as restarting the REPL.","category":"page"},{"location":"rule_author/which_functions_need_rules.html#Exception-handling","page":"Which functions need rules?","title":"Exception handling","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Zygote does not support differentiating through try/catch statements. For example, differentiating through","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function exception(x)\n try\n return x^2\n catch e\n println(\"could not square input\")\n throw(e)\n end\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"does not work","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> gradient(exception, 3.0)\nERROR: Compiling Tuple{typeof(exception),Int64}: try/catch is not supported.","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"without an rrule defined (restart the REPL after calling gradient)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function ChainRulesCore.rrule(::typeof(exception), x)\n y = exception(x)\n function exception_pullback(ȳ)\n return NoTangent(), 2*x\n end\n return y, exception_pullback\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> gradient(exception, 3.0)\n(6.0,)","category":"page"},{"location":"rule_author/which_functions_need_rules.html#Loops","page":"Which functions need rules?","title":"Loops","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Julia runs loops fast. Unfortunately Zygote differentiates through loops slowly. So, for example, computing the mean squared error by using a loop","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function mse(y, ŷ)\n N = length(y)\n s = 0.0\n for i in 1:N\n s += (y[i] - ŷ[i])^2.0\n end\n return s/N\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"takes a lot longer to AD through","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> y = rand(30)\njulia> ŷ = rand(30)\njulia> @btime gradient(mse, $y, $ŷ)\n 38.180 μs (993 allocations: 65.00 KiB)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"than if we supply an rrule, (restart the REPL after calling gradient)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function ChainRules.rrule(::typeof(mse), x, x̂)\n output = mse(x, x̂)\n function mse_pullback(ȳ)\n N = length(x)\n g = (2 ./ N) .* (x .- x̂) .* ȳ\n return NoTangent(), g, -g\n end\n return output, mse_pullback\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"which is much faster","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> @btime gradient(mse, $y, $ŷ)\n 143.697 ns (2 allocations: 672 bytes)","category":"page"},{"location":"rule_author/which_functions_need_rules.html#In-place-accumulation","page":"Which functions need rules?","title":"In-place accumulation","text":"","category":"section"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"In-place accumulation of gradients is slow in Zygote. The issue, demonstrated in the following example, is that the gradient of getindex allocates an array of zeros with a single non-zero element. ","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function sum3(array)\n x = array[1]\n y = array[2]\n z = array[3]\n return x+y+z\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> @btime gradient(sum3, rand(30))\n 424.510 ns (9 allocations: 2.06 KiB)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"Computing the gradient with only a single array allocation using an rrule (restart the REPL after calling gradient)","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"function ChainRulesCore.rrule(::typeof(sum3), a)\n y = sum3(a)\n function sum3_pullback(ȳ)\n grad = zeros(length(a))\n grad[1:3] .+= ȳ\n return NoTangent(), grad\n end\n return y, sum3_pullback\nend","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"turns out to be significantly faster ","category":"page"},{"location":"rule_author/which_functions_need_rules.html","page":"Which functions need rules?","title":"Which functions need rules?","text":"julia> @btime gradient(sum3, rand(30))\n 192.818 ns (3 allocations: 784 bytes)","category":"page"},{"location":"api.html#API-Documentation","page":"API","title":"API Documentation","text":"","category":"section"},{"location":"api.html#Rules","page":"API","title":"Rules","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"Modules = [ChainRulesCore]\nPages = [\"rules.jl\"]\nPrivate = false","category":"page"},{"location":"api.html#ChainRulesCore.frule-Tuple{Any, Any, Vararg{Any, N} where N}","page":"API","title":"ChainRulesCore.frule","text":"frule([::RuleConfig,] (Δf, Δx...), f, x...)\n\nExpressing the output of f(x...) as Ω, return the tuple:\n\n(Ω, ΔΩ)\n\nThe second return value is the tangent w.r.t. the output.\n\nIf no method matching frule((Δf, Δx...), f, x...) has been defined, then return nothing.\n\nExamples:\n\nunary input, unary output scalar function:\n\njulia> dself = NoTangent();\n\njulia> x = rand()\n0.8236475079774124\n\njulia> sinx, Δsinx = frule((dself, 1), sin, x)\n(0.7336293678134624, 0.6795498147167869)\n\njulia> sinx == sin(x)\ntrue\n\njulia> Δsinx == cos(x)\ntrue\n\nUnary input, binary output scalar function:\n\njulia> sincosx, Δsincosx = frule((dself, 1), sincos, x);\n\njulia> sincosx == sincos(x)\ntrue\n\njulia> Δsincosx[1] == cos(x)\ntrue\n\njulia> Δsincosx[2] == -sin(x)\ntrue\n\nNote that techically speaking julia does not have multiple output functions, just functions that return a single output that is iterable, like a Tuple. So this is actually a Tangent:\n\njulia> Δsincosx\nTangent{Tuple{Float64, Float64}}(0.6795498147167869, -0.7336293678134624)\n\nThe optional RuleConfig option allows specifying frules only for AD systems that support given features. If not needed, then it can be omitted and the frule without it will be hit as a fallback. This is the case for most rules.\n\nSee also: rrule, @scalar_rule, RuleConfig\n\n\n\n\n\n","category":"method"},{"location":"api.html#ChainRulesCore.rrule-Tuple{Any, Vararg{Any, N} where N}","page":"API","title":"ChainRulesCore.rrule","text":"rrule([::RuleConfig,] f, x...)\n\nExpressing x as the tuple (x₁, x₂, ...) and the output tuple of f(x...) as Ω, return the tuple:\n\n(Ω, (Ω̄₁, Ω̄₂, ...) -> (s̄elf, x̄₁, x̄₂, ...))\n\nWhere the second return value is the the propagation rule or pullback. It takes in cotangents corresponding to the outputs (x̄₁, x̄₂, ...), and s̄elf, the internal values of the function itself (for closures)\n\nIf no method matching rrule(f, xs...) has been defined, then return nothing.\n\nExamples:\n\nunary input, unary output scalar function:\n\njulia> x = rand();\n\njulia> sinx, sin_pullback = rrule(sin, x);\n\njulia> sinx == sin(x)\ntrue\n\njulia> sin_pullback(1) == (NoTangent(), cos(x))\ntrue\n\nbinary input, unary output scalar function:\n\njulia> x, y = rand(2);\n\njulia> hypotxy, hypot_pullback = rrule(hypot, x, y);\n\njulia> hypotxy == hypot(x, y)\ntrue\n\njulia> hypot_pullback(1) == (NoTangent(), (x / hypot(x, y)), (y / hypot(x, y)))\ntrue\n\nThe optional RuleConfig option allows specifying rrules only for AD systems that support given features. If not needed, then it can be omitted and the rrule without it will be hit as a fallback. This is the case for most rules.\n\nSee also: frule, @scalar_rule, RuleConfig\n\n\n\n\n\n","category":"method"},{"location":"api.html#Rule-Definition-Tools","page":"API","title":"Rule Definition Tools","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"Modules = [ChainRulesCore]\nPages = [\"rule_definition_tools.jl\"]\nPrivate = false","category":"page"},{"location":"api.html#ChainRulesCore.@non_differentiable-Tuple{Any}","page":"API","title":"ChainRulesCore.@non_differentiable","text":"@non_differentiable(signature_expression)\n\nA helper to make it easier to declare that a method is not differentiable. This is a short-hand for defining an frule and rrule that return NoTangent() for all partials (even for the function s̄elf-partial itself)\n\nKeyword arguments should not be included.\n\njulia> @non_differentiable Base.:(==)(a, b)\n\njulia> _, pullback = rrule(==, 2.0, 3.0);\n\njulia> pullback(1.0)\n(NoTangent(), NoTangent(), NoTangent())\n\nYou can place type-constraints in the signature:\n\njulia> @non_differentiable Base.length(xs::Union{Number, Array})\n\njulia> frule((ZeroTangent(), 1), length, [2.0, 3.0])\n(2, NoTangent())\n\nwarning: Warning\nThis helper macro covers only the simple common cases. It does not support where-clauses. For these you can declare the rrule and frule directly\n\n\n\n\n\n","category":"macro"},{"location":"api.html#ChainRulesCore.@opt_out-Tuple{Any}","page":"API","title":"ChainRulesCore.@opt_out","text":"@opt_out frule([config], _, f, args...)\n@opt_out rrule([config], f, args...)\n\nThis allows you to opt-out of an frule or an rrule by providing a more specific method, that says to use the AD system to differentiate it.\n\nFor example, consider some function foo(x::AbtractArray). In general, you know an efficient and generic way to implement its rrule. You do so, (likely making use of ProjectTo). But it actually turns out that for some FancyArray type it is better to let the AD do its thing.\n\nThen you would write something like:\n\nfunction rrule(::typeof(foo), x::AbstractArray)\n foo_pullback(ȳ) = ...\n return foo(x), foo_pullback\nend\n\n@opt_out rrule(::typeof(foo), ::FancyArray)\n\nThis will generate an rrule that returns nothing, and will also add a similar entry to ChainRulesCore.no_rrule.\n\nSimilar applies for frule and ChainRulesCore.no_frule\n\nFor more information see the documentation on opting out of rules.\n\n\n\n\n\n","category":"macro"},{"location":"api.html#ChainRulesCore.@scalar_rule-Tuple{Any, Any, Vararg{Any, N} where N}","page":"API","title":"ChainRulesCore.@scalar_rule","text":"@scalar_rule(f(x₁, x₂, ...),\n @setup(statement₁, statement₂, ...),\n (∂f₁_∂x₁, ∂f₁_∂x₂, ...),\n (∂f₂_∂x₁, ∂f₂_∂x₂, ...),\n ...)\n\nA convenience macro that generates simple scalar forward or reverse rules using the provided partial derivatives. Specifically, generates the corresponding methods for frule and rrule:\n\nfunction ChainRulesCore.frule((NoTangent(), Δx₁, Δx₂, ...), ::typeof(f), x₁::Number, x₂::Number, ...)\n Ω = f(x₁, x₂, ...)\n $(statement₁, statement₂, ...)\n return Ω, (\n (∂f₁_∂x₁ * Δx₁ + ∂f₁_∂x₂ * Δx₂ + ...),\n (∂f₂_∂x₁ * Δx₁ + ∂f₂_∂x₂ * Δx₂ + ...),\n ...\n )\nend\n\nfunction ChainRulesCore.rrule(::typeof(f), x₁::Number, x₂::Number, ...)\n Ω = f(x₁, x₂, ...)\n $(statement₁, statement₂, ...)\n return Ω, ((ΔΩ₁, ΔΩ₂, ...)) -> (\n NoTangent(),\n ∂f₁_∂x₁ * ΔΩ₁ + ∂f₂_∂x₁ * ΔΩ₂ + ...),\n ∂f₁_∂x₂ * ΔΩ₁ + ∂f₂_∂x₂ * ΔΩ₂ + ...),\n ...\n )\nend\n\nIf no type constraints in f(x₁, x₂, ...) within the call to @scalar_rule are provided, each parameter in the resulting frule/rrule definition is given a type constraint of Number. Constraints may also be explicitly be provided to override the Number constraint, e.g. f(x₁::Complex, x₂), which will constrain x₁ to Complex and x₂ to Number.\n\nAt present this does not support defining for closures/functors. Thus in reverse-mode, the first returned partial, representing the derivative with respect to the function itself, is always NoTangent(). And in forward-mode, the first input to the returned propagator is always ignored.\n\nThe result of f(x₁, x₂, ...) is automatically bound to Ω. This allows the primal result to be conveniently referenced (as Ω) within the derivative/setup expressions.\n\nThis macro assumes complex functions are holomorphic. In general, for non-holomorphic functions, the frule and rrule must be defined manually.\n\nIf the derivative is one, (e.g. for identity functions) true can be used as the most general multiplicative identity.\n\nThe @setup argument can be elided if no setup code is need. In other words:\n\n@scalar_rule(f(x₁, x₂, ...),\n (∂f₁_∂x₁, ∂f₁_∂x₂, ...),\n (∂f₂_∂x₁, ∂f₂_∂x₂, ...),\n ...)\n\nis equivalent to:\n\n@scalar_rule(f(x₁, x₂, ...),\n @setup(nothing),\n (∂f₁_∂x₁, ∂f₁_∂x₂, ...),\n (∂f₂_∂x₁, ∂f₂_∂x₂, ...),\n ...)\n\nFor examples, see ChainRules' rulesets directory.\n\nSee also: frule, rrule.\n\n\n\n\n\n","category":"macro"},{"location":"api.html#Tangent-Types","page":"API","title":"Tangent Types","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"Modules = [ChainRulesCore]\nPages = [\n \"tangent_types/abstract_zero.jl\",\n \"tangent_types/one.jl\",\n \"tangent_types/structural_tangent.jl\",\n \"tangent_types/thunks.jl\",\n \"tangent_types/abstract_tangent.jl\",\n \"tangent_types/notimplemented.jl\",\n]\nPrivate = false","category":"page"},{"location":"api.html#ChainRulesCore.AbstractZero","page":"API","title":"ChainRulesCore.AbstractZero","text":"AbstractZero <: AbstractTangent\n\nSupertype for zero-like tangents—i.e., tangents that act like zero when added or multiplied to other values. If an AD system encounters a propagator that takes as input only subtypes of AbstractZero, then it can stop performing AD operations. All propagators are linear functions, and thus the final result will be zero.\n\nAll AbstractZero subtypes are singleton types. There are two of them: ZeroTangent() and NoTangent().\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.NoTangent","page":"API","title":"ChainRulesCore.NoTangent","text":"NoTangent() <: AbstractZero\n\nThis tangent indicates that the derivative does not exist. It is the tangent type for primal types that are not differentiable, such as integers or booleans (when they are not being used to represent floating-point values). The only valid way to perturb such values is to not change them at all. As a consequence, NoTangent is functionally identical to ZeroTangent(), but it provides additional semantic information.\n\nAdding NoTangent() to a primal is generally wrong: gradient-based methods cannot be used to optimize over discrete variables. An optimization package making use of this might want to check for such a case.\n\nnote: Note\nThis does not indicate that the derivative is not implemented, but rather that mathematically it is not defined.\n\nThis mostly shows up as the derivative with respect to dimension, index, or size arguments.\n\n function rrule(fill, x, len::Int)\n y = fill(x, len)\n fill_pullback(ȳ) = (NoTangent(), @thunk(sum(Ȳ)), NoTangent())\n return y, fill_pullback\n end\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.ZeroTangent","page":"API","title":"ChainRulesCore.ZeroTangent","text":"ZeroTangent() <: AbstractZero\n\nThe additive identity for tangents. This is basically the same as 0. A derivative of ZeroTangent() does not propagate through the primal function.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.zero_tangent","page":"API","title":"ChainRulesCore.zero_tangent","text":"zero_tangent(primal)\n\nThis returns an appropriate zero tangent suitable for accumulating tangents of the primal. For mutable composites types this is a structural MutableTangent For Arrays, it is applied recursively for each element. For other types, in particular immutable types, we do not make promises beyond that it will be iszero and suitable for accumulating against. For types without a tangent space (e.g. singleton structs) this returns NoTangent(). In general, it is more likely to produce a structural tangent.\n\n!!! warning Exprimental zero_tangentis an experimental feature, and is part of the mutation support featureset. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore. Exactly how it should be used (e.g. is it forward-mode only?)\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.MutableTangent","page":"API","title":"ChainRulesCore.MutableTangent","text":"MutableTangent{P}(fields) <: StructuralTangent{P} <: AbstractTangent\n\nThis type represents the tangent to a mutable struct. It itself is also mutable.\n\n!!! warning Exprimental MutableTangent is an experimental feature, and is part of the mutation support featureset. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore. Exactly how it should be used (e.g. is it forward-mode only?)\n\n!!! warning Do not directly mess with the tangent backing data It is relatively straight forward for a forwards-mode AD to work correctly in the presence of mutation and aliasing of primal values. However, this requires that the tangent is aliased in turn and conversely that it is copied when the primal is). If you seperately alias the backing data, etc by using the internal ChainRulesCore.backing function you can break this.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.StructuralTangent","page":"API","title":"ChainRulesCore.StructuralTangent","text":"StructuralTangent{P} <: AbstractTangent\n\nRepresenting the type of the tangent of a struct P (or a Tuple/NamedTuple). as an object with mirroring fields.\n\n!!!!!! warning Exprimental StructuralTangent is an experimental feature, and is part of the mutation support featureset. The StructuralTangent constructor returns a MutableTangent for mutable structs. MutableTangent is an experimental feature. Thus use of StructuralTangent (rather than Tangent directly) is also experimental. While this notice remains it may have changes in behavour, and interface in any minor version of ChainRulesCore.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.Tangent","page":"API","title":"ChainRulesCore.Tangent","text":"Tangent{P, T} <: StructuralTangent{P} <: AbstractTangent\n\nThis type represents the tangent for a struct/NamedTuple, or Tuple. P is the the corresponding primal type that this is a tangent for.\n\nTangent{P} should have fields (technically properties), that match to a subset of the fields of the primal type; and each should be a tangent type matching to the primal type of that field. Fields of the P that are not present in the Tangent are treated as Zero.\n\nT is an implementation detail representing the backing data structure. For Tuple it will be a Tuple, and for everything else it will be a NamedTuple. It should not be passed in by user.\n\nFor Tangents of Tuples, iterate and getindex are overloaded to behave similarly to for a tuple. For Tangents of structs, getproperty is overloaded to allow for accessing values via tangent.fieldname. Any fields not explictly present in the Tangent are treated as being set to ZeroTangent(). To make a Tangent have all the fields of the primal the canonicalize function is provided.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.canonicalize-Union{Tuple{Tangent{P, var\"#s9\"} where var\"#s9\"<:(NamedTuple{L, T} where T<:Tuple)}, Tuple{L}, Tuple{P}} where {P, L}","page":"API","title":"ChainRulesCore.canonicalize","text":"canonicalize(tangent::Tangent{P}) -> Tangent{P}\n\nReturn the canonical Tangent for the primal type P. The property names of the returned Tangent match the field names of the primal, and all fields of P not present in the input tangent are explictly set to ZeroTangent().\n\n\n\n\n\n","category":"method"},{"location":"api.html#ChainRulesCore.InplaceableThunk","page":"API","title":"ChainRulesCore.InplaceableThunk","text":"InplaceableThunk(add!::Function, val::Thunk)\n\nA wrapper for a Thunk, that allows it to define an inplace add! function.\n\nadd! should be defined such that: ithunk.add!(Δ) = Δ .+= ithunk.val but it should do this more efficently than simply doing this directly. (Otherwise one can just use a normal Thunk).\n\nMost operations on an InplaceableThunk treat it just like a normal Thunk; and destroy its inplacability.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.Thunk","page":"API","title":"ChainRulesCore.Thunk","text":"Thunk(()->v)\n\nA thunk is a deferred computation. It wraps a zero argument closure that when invoked returns a tangent. @thunk(v) is a macro that expands into Thunk(()->v).\n\nTo evaluate the wrapped closure, call unthunk which is a no-op when the argument is not a Thunk.\n\njulia> t = @thunk(3)\nThunk(var\"#4#5\"())\n\njulia> unthunk(t)\n3\n\nWhen to @thunk?\n\nWhen writing rrules (and to a lesser exent frules), it is important to @thunk appropriately. Propagation rules that return multiple derivatives may not have all deriviatives used. By @thunking the work required for each derivative, they then compute only what is needed.\n\nHow do thunks prevent work?\n\nIf we have res = pullback(...) = @thunk(f(x)), @thunk(g(x)) then if we did dx + res[1] then only f(x) would be evaluated, not g(x). Also if we did ZeroTangent() * res[1] then the result would be ZeroTangent() and f(x) would not be evaluated.\n\nSo why not thunk everything?\n\n@thunk creates a closure over the expression, which (effectively) creates a struct with a field for each variable used in the expression, and call overloaded.\n\nDo not use @thunk if this would be equal or more work than actually evaluating the expression itself. This is commonly the case for scalar operators.\n\nFor more details see the manual section on using thunks effectively.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.unthunk-Tuple{Any}","page":"API","title":"ChainRulesCore.unthunk","text":"unthunk(x)\n\nOn AbstractThunks this removes 1 layer of thunking. On any other type, it is the identity operation.\n\n\n\n\n\n","category":"method"},{"location":"api.html#ChainRulesCore.@thunk-Tuple{Any}","page":"API","title":"ChainRulesCore.@thunk","text":"@thunk expr\n\nDefine a Thunk wrapping the expr, to lazily defer its evaluation.\n\n\n\n\n\n","category":"macro"},{"location":"api.html#ChainRulesCore.@not_implemented-Tuple{Any}","page":"API","title":"ChainRulesCore.@not_implemented","text":"@not_implemented(info)\n\nCreate a tangent that indicates that the derivative is not implemented.\n\nThe info should be useful information about the missing tangent for debugging.\n\nnote: Note\nThis macro should be used only if the automatic differentiation would error otherwise. It is mostly useful if the function has multiple inputs or outputs, and one has worked out analytically and implemented some but not all tangents.\n\nnote: Note\nIt is good practice to include a link to a GitHub issue about the missing tangent in the debugging information.\n\n\n\n\n\n","category":"macro"},{"location":"api.html#Accumulation","page":"API","title":"Accumulation","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"add!!\nChainRulesCore.is_inplaceable_destination","category":"page"},{"location":"api.html#ChainRulesCore.add!!","page":"API","title":"ChainRulesCore.add!!","text":"add!!(x, y)\n\nReturns x+y, potentially mutating x in-place to hold this value. This avoids allocations when x can be mutated in this way.\n\n\n\n\n\nadd!!(x, t::InplacableThunk)\n\nThe specialization of add!! for InplaceableThunk promises to only call t.add! on x if x is suitably mutable; otherwise it will be out of place.\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.is_inplaceable_destination","page":"API","title":"ChainRulesCore.is_inplaceable_destination","text":"is_inplaceable_destination(x) -> Bool\n\nReturns true if x is suitable for for storing inplace accumulation of gradients. For arrays this means x .= y will mutate x, if y is an appropriate tangent.\n\nHere \"appropriate\" means that y cannot be complex unless x is too, and that for structured matrices like x isa Diagonal, y shares this structure.\n\nnote: history\nWrapper array types should overload this function if they can be written into. Before ChainRulesCore 1.16, it would guess true for most wrappers based on parent, but this is not safe, e.g. it will lead to an error with ReadOnlyArrays.jl. \n\nThere must always be a correct non-mutating path, so in uncertain cases, this function returns false.\n\n\n\n\n\n","category":"function"},{"location":"api.html#RuleConfig","page":"API","title":"RuleConfig","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"Modules = [ChainRulesCore]\nPages = [\"config.jl\"]\nPrivate = false","category":"page"},{"location":"api.html#ChainRulesCore.HasForwardsMode","page":"API","title":"ChainRulesCore.HasForwardsMode","text":"HasForwardsMode\n\nThis trait indicates that a RuleConfig{>:HasForwardsMode} can perform forward mode AD. If it is set then frule_via_ad must be implemented.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.HasReverseMode","page":"API","title":"ChainRulesCore.HasReverseMode","text":"HasReverseMode\n\nThis trait indicates that a RuleConfig{>:HasReverseMode} can perform reverse mode AD. If it is set then rrule_via_ad must be implemented.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.NoForwardsMode","page":"API","title":"ChainRulesCore.NoForwardsMode","text":"NoForwardsMode\n\nThis is the complement to HasForwardsMode. To avoid ambiguities [RuleConfig]s that do not support performing forwards mode AD should be RuleConfig{>:NoForwardsMode}.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.NoReverseMode","page":"API","title":"ChainRulesCore.NoReverseMode","text":"NoReverseMode\n\nThis is the complement to HasReverseMode. To avoid ambiguities [RuleConfig]s that do not support performing reverse mode AD should be RuleConfig{>:NoReverseMode}.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.RuleConfig","page":"API","title":"ChainRulesCore.RuleConfig","text":"RuleConfig{T}\n\nThe configuration for what rules to use. T: traits. This should be a Union of all special traits needed for rules to be allowed to be defined for your AD. If nothing special this should be set to Union{}.\n\nAD authors should define a subtype of RuleConfig to use when calling frule/rrule.\n\nRule authors can dispatch on this config when defining rules. For example:\n\n# only define rrule for `pop!` on AD systems where mutation is supported.\nrrule(::RuleConfig{>:SupportsMutation}, typeof(pop!), ::Vector) = ...\n\n# this definition of map is for any AD that defines a forwards mode\nrrule(conf::RuleConfig{>:HasForwardsMode}, typeof(map), ::Vector) = ...\n\n# this definition of map is for any AD that only defines a reverse mode.\n# It is not as good as the rrule that can be used if the AD defines a forward-mode as well.\nrrule(conf::RuleConfig{>:Union{NoForwardsMode, HasReverseMode}}, typeof(map), ::Vector) = ...\n\nFor more details see rule configurations and calling back into AD.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.frule_via_ad","page":"API","title":"ChainRulesCore.frule_via_ad","text":"frule_via_ad(::RuleConfig{>:HasForwardsMode}, ȧrgs, f, args...; kwargs...)\n\nThis function has the same API as frule, but operates via performing forwards mode automatic differentiation. Any RuleConfig subtype that supports the HasForwardsMode special feature must provide an implementation of it.\n\nSee also: rrule_via_ad, RuleConfig and the documentation on rule configurations and calling back into AD\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.rrule_via_ad","page":"API","title":"ChainRulesCore.rrule_via_ad","text":"rrule_via_ad(::RuleConfig{>:HasReverseMode}, f, args...; kwargs...)\n\nThis function has the same API as rrule, but operates via performing reverse mode automatic differentiation. Any RuleConfig subtype that supports the HasReverseMode special feature must provide an implementation of it.\n\nSee also: frule_via_ad, RuleConfig and the documentation on rule configurations and calling back into AD\n\n\n\n\n\n","category":"function"},{"location":"api.html#ProjectTo","page":"API","title":"ProjectTo","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"ProjectTo","category":"page"},{"location":"api.html#ChainRulesCore.ProjectTo","page":"API","title":"ChainRulesCore.ProjectTo","text":"(p::ProjectTo{T})(dx)\n\nProjects the tangent dx onto a specific tangent space.\n\nThe type T is meant to encode the largest acceptable space, so usually this enforces p(dx)::T. But some subspaces which aren't subtypes of T may be allowed, and in particular dx::AbstractZero always passes through.\n\nUsually T is the \"outermost\" part of the type, and p stores additional properties such as projectors for each constituent field. Arrays have either one projector p.element expressing the element type for an array of numbers, or else an array of projectors p.elements. These properties can be supplied as keyword arguments on construction, p = ProjectTo{T}(; field=data, element=Projector(x)). For each T in use, corresponding methods should be written for ProjectTo{T}(dx) with nonzero dx.\n\nWhen called on dx::Thunk, the projection is inserted into the thunk.\n\n\n\n\n\n","category":"type"},{"location":"api.html#Ignoring-gradients","page":"API","title":"Ignoring gradients","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"ignore_derivatives\n@ignore_derivatives","category":"page"},{"location":"api.html#ChainRulesCore.ignore_derivatives","page":"API","title":"ChainRulesCore.ignore_derivatives","text":"ignore_derivatives(f::Function)\n\nTells the AD system to ignore the gradients of the wrapped closure. The primal computation (forward pass) is executed normally.\n\nignore_derivatives() do\n value = rand()\n push!(collection, value)\nend\n\nUsing this incorrectly could lead to incorrect gradients. For example, the following function will have zero gradients with respect to its argument:\n\nfunction wrong_grads(x)\n y = ones(3)\n ignore_derivatives() do\n push!(y, x)\n end\n return sum(y)\nend\n\n\n\n\n\nignore_derivatives(x)\n\nTells the AD system to ignore the gradients of the argument. Can be used to avoid unnecessary computation of gradients.\n\nignore_derivatives(x) * w\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.@ignore_derivatives","page":"API","title":"ChainRulesCore.@ignore_derivatives","text":"@ignore_derivatives (...)\n\nTells the AD system to ignore the expression. Equivalent to ignore_derivatives() do (...) end.\n\n\n\n\n\n","category":"macro"},{"location":"api.html#Internal","page":"API","title":"Internal","text":"","category":"section"},{"location":"api.html","page":"API","title":"API","text":"ChainRulesCore.AbstractTangent\nChainRulesCore.debug_mode\nChainRulesCore.no_rrule\nChainRulesCore.no_frule","category":"page"},{"location":"api.html#ChainRulesCore.AbstractTangent","page":"API","title":"ChainRulesCore.AbstractTangent","text":"The subtypes of AbstractTangent define a custom \"algebra\" for chain rule evaluation that attempts to factor various features like complex derivative support, broadcast fusion, zero-elision, etc. into nicely separated parts.\n\nIn general a tangent type is the type of a derivative of a value. The type of the value is for contrast called the primal type. Differential types correspond to primal types, although the relation is not one-to-one. Subtypes of AbstractTangent are not the only tangent types. In fact for the most common primal types, such as Real or AbstractArray{Real} the the tangent type is the same as the primal type.\n\nIn a circular definition: the most important property of a tangent is that it should be able to be added (by defining +) to another tangent of the same primal type. That allows for gradients to be accumulated.\n\nIt generally also should be able to be added to a primal to give back another primal, as this facilitates gradient descent.\n\nAll subtypes of AbstractTangent implement the following operations:\n\n+(a, b): linearly combine tangent a and tangent b\n*(a, b): multiply the tangent b by the scaling factor a\nBase.zero(x) = ZeroTangent(): a zero.\n\nFurther, they often implement other linear operators, such as conj, adjoint, dot. Pullbacks/pushforwards are linear operators, and their inputs are often AbstractTangent subtypes. Pullbacks/pushforwards in-turn call other linear operators on those inputs. Thus it is desirable to have all common linear operators work on AbstractTangents.\n\n\n\n\n\n","category":"type"},{"location":"api.html#ChainRulesCore.debug_mode","page":"API","title":"ChainRulesCore.debug_mode","text":"debug_mode() -> Bool\n\nDetermines if ChainRulesCore is in debug_mode. Defaults to false, but if the user redefines it to return true then extra information will be shown when errors occur.\n\nEnable via:\n\nChainRulesCore.debug_mode() = true\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.no_rrule","page":"API","title":"ChainRulesCore.no_rrule","text":"no_rrule\n\nThis is an piece of infastructure supporting opting out of rrule. It follows the signature for rrule exactly. A collection of type-tuples is stored in its method-table. If something has this defined, it means that it must having a must also have a rrule, defined that returns nothing.\n\nwarning: Do not overload no_rrule directly\nIt is fine and intended to query the method table of no_rrule. It is not safe to add to that directly, as corresponding changes also need to be made to rrule. The @opt_out macro does both these things, and so should almost always be used rather than defining a method of no_rrule directly.\n\nMechanics\n\nnote: when the text below says methods == it actually means: parameters(m.sig)[2:end] (i.e. the signature type tuple) rather than the method object m itself.\n\nTo decide if should opt-out using this mechanism.\n\nfind the most specific method of rrule and no_rule e.g with Base.which\nif the method of no_rrule == the method of rrule, then should opt-out\n\nTo just ignore the fact that rules can be opted-out from, and that some rules thus return nothing, then filter the list of methods of rrule to remove those that are == to ones that occur in the method table of no_rrule.\n\nNote also when doing this you must still also handle falling back from rule with config, to rule without config.\n\nOn the other-hand if your AD can work with rrules that return nothing, then it is simpler to just use that mechanism for opting out; and you don't need to worry about this at all.\n\nFor more information see the documentation on opting out of rules\n\nSee also ChainRulesCore.no_frule.\n\n\n\n\n\n","category":"function"},{"location":"api.html#ChainRulesCore.no_frule","page":"API","title":"ChainRulesCore.no_frule","text":"no_frule\n\nThis is an piece of infastructure supporting opting out of frule. It follows the signature for frule exactly. A collection of type-tuples is stored in its method-table. If something has this defined, it means that it must having a must also have a frule, defined that returns nothing.\n\nwarning: Do not overload no_frule directly\nIt is fine and intended to query the method table of no_frule. It is not safe to add to that directly, as corresponding changes also need to be made to frule. The @opt_out macro does both these things, and so should almost always be used rather than defining a method of no_frule directly.\n\nMechanics\n\nnote: when the text below says methods == it actually means: parameters(m.sig)[2:end] (i.e. the signature type tuple) rather than the method object m itself.\n\nTo decide if should opt-out using this mechanism.\n\nfind the most specific method of frule and no_rule e.g with Base.which\nif the method of no_frule == the method of frule, then should opt-out\n\nTo just ignore the fact that rules can be opted-out from, and that some rules thus return nothing, then filter the list of methods of frule to remove those that are == to ones that occur in the method table of no_frule.\n\nNote also when doing this you must still also handle falling back from rule with config, to rule without config.\n\nOn the other-hand if your AD can work with frules that return nothing, then it is simpler to just use that mechanism for opting out; and you don't need to worry about this at all.\n\nFor more information see the documentation on opting out of rules\n\nSee also ChainRulesCore.no_rrule.\n\n\n\n\n\n","category":"function"},{"location":"rule_author/writing_good_rules.html#On-writing-good-rrule-/-frule-methods","page":"Writing good rules","title":"On writing good rrule / frule methods","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html#Code-Style","page":"Writing good rules","title":"Code Style","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Use named local functions for the pullback in an rrule.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"# good:\nfunction rrule(::typeof(foo), x)\n Y = foo(x)\n function foo_pullback(Ȳ)\n return NoTangent(), bar(Ȳ)\n end\n return Y, foo_pullback\nend\n#== output\njulia> rrule(foo, 2)\n(4, var\"#foo_pullback#11\"())\n==#\n\n# bad:\nfunction rrule(::typeof(foo), x)\n return foo(x), x̄ -> (NoTangent(), bar(x̄))\nend\n#== output:\njulia> rrule(foo, 2)\n(4, var\"##9#10\"())\n==#","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"While this is more verbose, it ensures that if an error is thrown during the pullback the gensym name of the local function will include the name you gave it. This makes it a lot simpler to debug from the stacktrace.","category":"page"},{"location":"rule_author/writing_good_rules.html#Use-ZeroTangent()-as-the-return-value","page":"Writing good rules","title":"Use ZeroTangent() as the return value","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"The ZeroTangent() object exists as an alternative to directly returning 0 or zeros(n). It allows more optimal computation when chaining pullbacks/pushforwards, to avoid work. They should be used where possible.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"However, sometimes for performance reasons this is not ideal. Especially, if it is to replace a scalar, and is in a type-unstable way. It causes problems if mapping over such pullbacks/pushforwards. This would be solved once JuliaLang/julia#38241 has been addressed.","category":"page"},{"location":"rule_author/writing_good_rules.html#Use-Thunks-appropriately","page":"Writing good rules","title":"Use Thunks appropriately","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"If work is only required for one of the returned tangents, then it should be wrapped in a @thunk (potentially using a begin-end block).","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"If there are multiple return values, their computation should almost always be wrapped in a @thunk.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Do not wrap variables in a @thunk; wrap the computations that fill those variables in @thunk:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"# good:\n∂A = @thunk(foo(x))\nreturn ∂A\n\n# bad:\n∂A = foo(x)\nreturn @thunk(∂A)","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"In the bad example foo(x) gets computed eagerly, and all that the thunk is doing is wrapping the already calculated result in a function that returns it.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Do not use @thunk if this would be equal or more work than actually evaluating the expression itself. Examples being:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"The expression being a constant\nThe expression is merely wrapping something in a struct, such as Adjoint(x) or Diagonal(x)\nThe expression being itself a thunk\nThe expression being from another rrule or frule; it would be @thunked if required by the defining rule already.\nThere is only one derivative being returned, so from the fact that the user called frule/rrule they clearly will want to use that one.","category":"page"},{"location":"rule_author/writing_good_rules.html#structs","page":"Writing good rules","title":"Structs: constructors and functors","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"To define an frule or rrule for a function foo we dispatch on the type of foo, which is typeof(foo). For example, the rrule signature would be like:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"function rrule(::typeof(foo), args...; kwargs...)\n ...\n return y, foo_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"For a struct Bar,","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"struct Bar\n a::Float64\nend\n\n(bar::Bar)(x, y) = return bar.a + x + y # functor (i.e. callable object, overloading the call action)","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"we can define an frule/rrule for the Bar constructor(s), as well as any Bar functors.","category":"page"},{"location":"rule_author/writing_good_rules.html#Constructors","page":"Writing good rules","title":"Constructors","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"To define an rrule for a constructor for a type Bar we need to be careful to dispatch only on Type{Bar}. For example, the rrule signature for a Bar constructor would be like:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"function ChainRulesCore.rrule(::Type{Bar}, a)\n Bar_pullback(Δbar) = NoTangent(), Δbar.a\n return Bar(a), Bar_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Use Type{<:Bar} (with the <:) for non-concrete types, such that the rrule is defined for all subtypes. In particular, be careful not to use typeof(Bar) here. Because typeof(Bar) is DataType, using this to define an rrule/frule will define an rrule/frule for all constructors.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"You can check which to use with Core.Typeof:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"julia> function foo end\nfoo (generic function with 0 methods)\n\njulia> typeof(foo)\ntypeof(foo)\n\njulia> Core.Typeof(foob)\ntypeof(foo)\n\njulia> typeof(Bar)\nDataType\n\njulia> Core.Typeof(Bar)\nType{Bar}\n\njulia> abstract type AbstractT end\n\njulia> typeof(AbstractT)\nDataType\n\njulia> Core.Typeof(AbstractT)\nType{AbstractT}","category":"page"},{"location":"rule_author/writing_good_rules.html#Functors-(callable-objects)","page":"Writing good rules","title":"Functors (callable objects)","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"In contrast to defining a rule for a constructor, it is possible to define rules for calling an instance of an object. In that case, use bar::Bar, i.e.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"function ChainRulesCore.rrule(bar::Bar, x, y)\n # Notice the first return is not `NoTangent()`\n Bar_pullback(Δy) = Tangent{Bar}(;a=Δy), Δy, Δy\n return bar(x, y), Bar_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"to define the rules.","category":"page"},{"location":"rule_author/writing_good_rules.html#Ensure-your-pullback-can-accept-the-right-types","page":"Writing good rules","title":"Ensure your pullback can accept the right types","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"As a rule the number of types you need to accept in a pullback is theoretically unlimitted, but practically highly constrained to be in line with the primal return type. The three kinds of inputs you will practically need to accept one or more of: natural tangents, structural tangents, and thunks. You do not in general have to handle AbstractZeros as the AD system will not call the pullback if the input is a zero, since the output will also be. Some more background information on these types can be found in the design notes. In many cases all these tangents can be treated the same: tangent types overload a bunch of linear-operators, and the majority of functions used inside a pullback are linear operators. If you find linear operators from Base/stdlibs that are not supported, consider opening an issue or a PR on the ChainRulesCore.jl repo.","category":"page"},{"location":"rule_author/writing_good_rules.html#Natural-tangents","page":"Writing good rules","title":"Natural tangents","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Natural tangent types are the types you might feel the tangent should be, to represent a small change in the primal value. For example, if the primal is a Float32, the natural tangent is also a Float32. Slightly more complex, for a ComplexF64 the natural tangent is again also a ComplexF64, we almost never want to use the structural tangent Tangent{ComplexF64}(re=..., im=...) which is defined. For other cases, this gets a little more complicated, see below. These are a purely human notion, they are the types the user wants to use because they make the math easy. There is currently no formal definition of what constitutes a natural tangent, but there are a few heuristics. For example, if a primal type P overloads subtraction (-(::P,::P)) then that generally returns a natural tangent type for P; but this is not required to be defined and sometimes it is defined poorly.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Common cases for types that represent a vector-space (e.g. Float64, Array{Float64}) is that the natural tangent type is the same as the primal type. However, this is not always the case. For example for a PDiagMat a natural tangent is Diagonal since there is no requirement that a positive definite diagonal matrix has a positive definite tangent. Another example is for a DateTime, any Period subtype, such as Millisecond or Nanosecond is a natural tangent. There are often many different natural tangent types for a given primal type. However, they are generally closely related and duck-type the same. For example, for most AbstractArray subtypes, most other AbstractArrays (of right size and element type) can be considered as natural tangent types.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Not all types have natural tangent types. For example there is no natural tangent for a Tuple. It is not a Tuple since that doesn't have any method for +. Similar is true for many structs. For those cases there is only a structural tangent.","category":"page"},{"location":"rule_author/writing_good_rules.html#Structural-tangents","page":"Writing good rules","title":"Structural tangents","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Structural tangents are tangent types that shadow the structure of the primal type. They are represented by the Tangent type. They can represent any composite type, such as a tuple, or a structure (or a NamedTuple) etc.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"info: Do I have to support the structural tangents as well?\nTechnically, you might not actually have to write rules to accept structural tangents; if the AD system never has to decompose down to the level of getfield. This is common for types that don't support user getfield/getproperty access, and that have a lot of rules for the ways they are accessed (such cases include some AbstractArray subtypes). You really should support it just in case; especially if the primal type in question is not restricted to a well-tested concrete type. But if it is causing struggles, then you can leave it off til someone complains.","category":"page"},{"location":"rule_author/writing_good_rules.html#Thunks","page":"Writing good rules","title":"Thunks","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"A thunk (either a Thunk, or a InplaceableThunk), represents a delayed computation. They can be thought of as a wrapper of the value the computation returns. In this sense they wrap either a natural or structural tangent.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"warning: You should support AbstractThunk inputs even if you don't use thunks\nUnfortunately the AD sytems do not know which rules support thunks and which do not. So all rules have to; at least if they want to play nicely with arbitrary AD systems. Luckily it is not hard: much of the time they will duck-type as the object they wrap. If not, then just add a unthunk after the start of your pullback. (Even when they do duck-type, if they are used multiple times then unthunking at the start will prevent them from being recomputed.) If you are using @thunk and the input is only needed for one of them then the unthunk should be in that one. If not, and you have a bunch of pullbacks you might like to write a little helper unthunking(f) = x̄ -> f(unthunk(x̄)) that you can wrap your pullback function in before returning it from the rrule. Yes, this is a bit of boiler-plate, and it is unfortunate. Sadly, it is needed because if the AD wants to benefit it can't get that benifit unless things are not unthunked unnecessarily. Which eventually allows them in some cases to never be unthunked at all. There are two ways common things are never unthunked. One is if the unthunking happens inside a @thunk which is never unthunked itself because it is the tangent for a primal input that never has it's tangent queried. The second is if they are not unthunked because the rule does not need to know what is inside: consider the pullback for identity: x̄ -> (NoTangent(), x̄).","category":"page"},{"location":"rule_author/writing_good_rules.html#Use-@not_implemented-appropriately","page":"Writing good rules","title":"Use @not_implemented appropriately","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"You can use @not_implemented to mark missing tangents. This is helpful if the function has multiple inputs or outputs, and you have worked out analytically and implemented some but not all tangents.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"It is recommended to include a link to a GitHub issue about the missing tangent in the debugging information:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"@not_implemented(\n \"\"\"\n derivatives of Bessel functions with respect to the order are not implemented:\n https://github.com/JuliaMath/SpecialFunctions.jl/issues/160\n \"\"\"\n)","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Do not use @not_implemented if the tangent does not exist mathematically (use NoTangent() instead).","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Note: ChainRulesTestUtils.jl marks @not_implemented tangents as \"test broken\".","category":"page"},{"location":"rule_author/writing_good_rules.html#Use-rule-definition-tools","page":"Writing good rules","title":"Use rule definition tools","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Rule definition tools can help you write more frules and the rrules with less lines of code. See using rule definition tools section for more details.","category":"page"},{"location":"rule_author/writing_good_rules.html#Be-careful-about-pullback-closures-calling-other-methods-of-themselves","page":"Writing good rules","title":"Be careful about pullback closures calling other methods of themselves","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Due to JuliaLang/Julia#40990, a closure calling another (or the same) method of itself often comes out uninferable (and thus effectively type-unstable). This can be avoided by moving the pullback definition outside the function, so that it is no longer a closure. For example:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"double_it(x::AbstractArray) = 2 .* x\n\nfunction ChainRulesCore.rrule(::typeof(double_it), x)\n double_it_pullback(ȳ::AbstractArray) = (NoTangent(), 2 .* ȳ)\n double_it_pullback(ȳ::AbstractThunk) = double_it_pullback(unthunk(ȳ))\n return double_it(x), double_it_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Ends up infering a return type of Any","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"julia> _, pullback = rrule(double_it, [2.0, 3.0])\n([4.0, 6.0], var\"#double_it_pullback#8\"(Core.Box(var\"#double_it_pullback#8\"(#= circular reference @-2 =#))))\n\njulia> @code_warntype pullback(@thunk([10.0, 10.0]))\nVariables\n #self#::var\"#double_it_pullback#8\"\n ȳ::Core.Const(Thunk(var\"#9#10\"()))\n double_it_pullback::Union{}\n\nBody::Any\n1 ─ %1 = Core.getfield(#self#, :double_it_pullback)::Core.Box\n│ %2 = Core.isdefined(%1, :contents)::Bool\n└── goto #3 if not %2\n2 ─ goto #4\n3 ─ Core.NewvarNode(:(double_it_pullback))\n└── double_it_pullback\n4 ┄ %7 = Core.getfield(%1, :contents)::Any\n│ %8 = Main.unthunk(ȳ)::Vector{Float64}\n│ %9 = (%7)(%8)::Any\n└── return %9","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"This can be solved by moving the pullbacks outside the function so they are not closures, and thus to not run into this upstream issue. In this case that is fairly simple, since this example doesn't close over anything (if it did then would need a closure calling an outside function that calls itself. See this example.).","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"_double_it_pullback(ȳ::AbstractArray) = (NoTangent(), 2 .* ȳ)\n_double_it_pullback(ȳ::AbstractThunk) = _double_it_pullback(unthunk(ȳ))\n\nfunction ChainRulesCore.rrule(::typeof(double_it), x)\n return double_it(x), _double_it_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"This infers just fine:","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"julia> _, pullback = rrule(double_it, [2.0, 3.0])\n([4.0, 6.0], _double_it_pullback)\n\njulia> @code_warntype pullback(@thunk([10.0, 10.0]))\nVariables\n #self#::Core.Const(_double_it_pullback)\n ȳ::Core.Const(Thunk(var\"#7#8\"()))\n\nBody::Tuple{NoTangent, Vector{Float64}}\n1 ─ %1 = Main.unthunk(ȳ)::Vector{Float64}\n│ %2 = Main._double_it_pullback(%1)::Core.PartialStruct(Tuple{NoTangent, Vector{Float64}}, Any[Core.Const(NoTangent()), Vector{Float64}])\n└── return %2","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"Though in this particular case, it can also be solved by taking advantage of duck-typing and just writing one method. Thus avoiding the call that confuses the compiler. Thunks duck-type as the type they wrap in most cases: including broadcast multiplication.","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"function ChainRulesCore.rrule(::typeof(double_it), x)\n double_it_pullback(ȳ) = (NoTangent(), 2 .* ȳ)\n return double_it(x), double_it_pullback\nend","category":"page"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"This infers perfectly.","category":"page"},{"location":"rule_author/writing_good_rules.html#CAS-systems-are-your-friends.","page":"Writing good rules","title":"CAS systems are your friends.","text":"","category":"section"},{"location":"rule_author/writing_good_rules.html","page":"Writing good rules","title":"Writing good rules","text":"It is very easy to check gradients or derivatives with a computer algebra system (CAS) like WolframAlpha.","category":"page"},{"location":"rule_author/superpowers/opt_out.html#opt_out","page":"@opt_out","title":"Opting out of rules","text":"","category":"section"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"It is common to define rules fairly generically. Often matching (or exceeding) how generic the matching original primal method is. Sometimes this is not the correct behaviour. Sometimes the AD can do better than this human defined rule. If this is generally the case, then we should not have the rule defined at all. But if it is only the case for a particular set of types, then we want to opt-out just that one. This is done with the @opt_out macro.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"Consider one a rrule for sum (the following simplified from the one in ChainRules.jl itself)","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"function rrule(::typeof(sum), x::AbstractArray{<:Number})\n y = sum(x; dims=dims)\n project = ProjectTo(x)\n function sum_pullback(ȳ)\n x̄ = project(fill(ȳ, size(x)))\n return (NoTangent(), x̄)\n end\n return y, sum_pullback\nend","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"That is a fairly reasonable rrule for the vast majority of cases. You might have a custom array type for which you could write a faster rule. In which case you would do that, by writing a faster, more specific, rrule. But sometimes, it is the case that ADing the (faster, more specific) primal for your custom array type would yeild the faster pullback without you having to write a rrule by hand.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"Consider a summing SkewSymmetric (anti-symmetric) matrix. The skew symmetric matrix has structural zeros on the diagonal, and off-diagonals are paired with their negation. Thus the sum is always going to be zero. As such the author of that matrix type would probably have overloaded sum(x::SkewSymmetric{T}) where T = zero(T). ADing this would result in the tangent computed for x as ZeroTangent() and it would be very fast since AD can see that x is never used in the right-hand side. In contrast the generic method for AbstractArray defined above would have to allocate the fill array, and then compute the skew projection. Only to find out the output would be projected to SkewSymmetric(zeros(T)) anyway (slower, and a less useful type).","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"To opt-out of using the generic rrule and to allow the AD system to do its own thing we use the @opt_out macro, to say to not use it for sum of SkewSymmetric.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"@opt_out rrule(::typeof(sum), ::SkewSymmetric)","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"Perhaps we might not want to ever use rules for SkewSymmetric, because we have determined that it is always better to leave it to the AD, unless a very specific rule has been written[1]. We could then opt-out for all 1 arg functions.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"@opt_out rrule(::Any, ::SkewSymmetric)","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"Though this is likely to cause some method-ambiguities, if we do it for more, but we can resolve those.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"Similar can be done @opt_out frule. It can also be done passing in a RuleConfig.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"warning: If the general rule uses a config, the opt-out must also\nFollowing the same principles as for rules with config, a rule with a RuleConfig argument will take precedence over one without, including if that one is a opt-out rule. But if the general rule does not use a config, then the opt-out rule can use a config. This allows, for example, you to use opt-out to avoid a particular AD system using a opt-out rule that takes that particular AD's config.","category":"page"},{"location":"rule_author/superpowers/opt_out.html","page":"@opt_out","title":"@opt_out","text":"[1]: seems unlikely, but it is possible, there is a lot of structure that can be taken advantage of for some matrix types.","category":"page"},{"location":"design/changing_the_primal.html#change_primal","page":"Changing the Primal","title":"Design Notes: Why can you change the primal computation?","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"These design notes are to help you understand ChainRules.jl's rrule function. It explains why we have a rrule function that returns both the primal result (i.e. the output for the forward pass) and the pullback as a closure. It might be surprising to some AD authors, who might expect just a function that performs the pullback, that the rrule function computes the primal result as well as the pullback. In particularly, rrule allows you to change how the primal result is computed. We will illustrate in this document why being able to change the computation of the primal is crucial for efficient AD.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"note: What about `frule`?\nDiscussion here is focused on on reverse mode and rrule. Similar concerns do apply to forward mode and frule. In forward mode these concerns lead to the fusing of the pushforward into frule. All the examples given here also apply in forward mode. In fact in forward mode there are even more opportunities to take advantage of sharing work between the primal and derivative computations. A particularly notable example is in efficiently calculating the pushforward of solving a differential equation via expanding the system of equations to also include the derivatives before solving it.","category":"page"},{"location":"design/changing_the_primal.html#The-Journey-to-rrule","page":"Changing the Primal","title":"The Journey to rrule","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Let's imagine a different system for rules, one that doesn't let you define the computation of the primal. This system is what a lot of AD systems have. It is what Nabla.jl had originally.[1] We will have a primal (i.e. forward) pass that directly executes the primal function and just records the primal function, its inputs and its output onto the tape.[2]. Then during the gradient (i.e. reverse) pass it has a function which receives those records from the tape along with the sensitivity of the output, and gives back the sensitivity of the input. We will call this function pullback_at, as it pulls back the sensitivity at a given primal point. To make this concrete:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"y = f(x) # primal program\nx̄ = pullback_at(f, x, y, ȳ)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Let's illustrate this with examples for sin and for the logistic sigmoid.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"y = sin(x)\npullback_at(::typeof(sin), x, y, ȳ) = ȳ * cos(x)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"pullback_at uses the primal input x, and the sensitivity being pulled back ȳ.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"σ(x) = 1/(1 + exp(-x)) # = exp(x) / (1 + exp(x))\ny = σ(x)\npullback_at(::typeof(σ), x, y, ȳ) = ȳ * y * σ(-x) # = ȳ * σ(x) * σ(-x)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Notice that in pullback_at we are not only using input x but also using the primal output y . This is a nice bit of symmetry that shows up around exp.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Now let's consider why we implement rrules in the first place. One key reason is to insert domain knowledge so as to compute the derivative more efficiently than AD would just by breaking everything down into +, *, etc.[3] What insights do we have about sin and cos? What about using sincos?","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"julia> using BenchmarkTools\n\njulia> @btime sin(x) setup=(x=rand());\n 3.838 ns (0 allocations: 0 bytes)\n\njulia> @btime cos(x) setup=(x=rand());\n 4.795 ns (0 allocations: 0 bytes)\n\njulia> 3.838 + 4.795\n8.633","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"vs computing both together:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"julia> @btime sincos(x) setup=(x=rand());\n 6.028 ns (0 allocations: 0 bytes)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"What about the logistic sigmoid? We note that the two values we need are σ(x) and σ(-x) If we write these as: sigma(x) = frace^x1+e^x and sigma(-x) = frac11+e^x then we see they have the common term e^x. exp(x) is a much more expensive operation than + and /. So we can save time, if we can reuse that exp(x).","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"If we have to computing separately:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"julia> @btime 1/(1+exp(x)) setup=(x=rand());\n 5.622 ns (0 allocations: 0 bytes)\n\njulia> @btime 1/(1+exp(-x)) setup=(x=rand());\n 6.036 ns (0 allocations: 0 bytes)\n\njulia> 5.622 + 6.036\n11.658","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"vs reusing exp(x):","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"julia> @btime exp(x) setup=(x=rand());\n 5.367 ns (0 allocations: 0 bytes)\n\njulia> @btime ex/(1+ex) setup=(ex=exp(rand()));\n 1.255 ns (0 allocations: 0 bytes)\n\njulia> @btime 1/(1+ex) setup=(ex=exp(rand()));\n 1.256 ns (0 allocations: 0 bytes)\n\njulia> 5.367 + 1.255 + 1.256\n7.878","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"So we are talking about a 30-40% speed-up from these optimizations.[4]","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"It is faster to compute sin and cos at the same time via sincos than it is to compute them one after the other. And it is faster to reuse the exp(x) in computing σ(x) and σ(-x). How can we incorporate this insight into our system? We know we can compute both of these in the primal — because they only depend on x and not on ȳ — but there is nowhere to put them that is accessible both to the primal pass and the gradient pass code.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"What if we introduced some variable called intermediates that is also recorded onto the tape during the primal pass? We would need to be able to modify the primal pass to do this, so that we can actually put the data into the intermediates. So we will introduce a function: augmented_primal, that will return the primal output plus the intermediates that we want to reuse in the gradient pass. Then we will make our AD system replace calls to the primal with calls to the augmented_primal of the primal function and take care of all the bookkeeping. So that would look like:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"y = f(x) # primal program\ny, intermediates = augmented_primal(f, x)\nx̄ = pullback_at(f, x, y, ȳ, intermediates)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y, cx = sincos(x)\n return y, (; cx=cx) # use a NamedTuple for the intermediates\nend\n\npullback_at(::typeof(sin), x, y, ȳ, intermediates) = ȳ * intermediates.cx","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n ex = exp(x)\n y = ex / (1 + ex)\n return y, (; ex=ex) # use a NamedTuple for the intermediates\nend\n\npullback_at(::typeof(σ), x, y, ȳ, intermediates) = ȳ * y / (1 + intermediates.ex)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Cool! That lets us do what we wanted. We net decreased the time it takes to run the primal and gradient passes. We have now demonstrated the title question of why we want to be able to modify the primal pass. We will go into that more later and have some more usage examples, but first let's continue to see how we go from augmented_primal and pullback_at to rrule.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"One thing we notice when looking at pullback_at is it really is starting to have a lot of arguments. It had a fair few already, and now we are adding intermediates as well, making it even more unwieldy. Not to mention these are fairly simple example, the sin and σ functions have 1 input and no keyword arguments. Furthermore, we often don't even use all of the arguments to pullback_at. The new code for pulling back sin — which uses sincos and intermediates — no longer needs x, and it never needed y (though sigmoid σ does). And storing all these things on the tape — inputs, outputs, sensitivities, intermediates — is using up extra memory. What if we generalized the idea of the intermediate named tuple, and had augmented_primal return a struct that just held anything we might want put on the tape.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"struct PullbackMemory{P, S}\n primal_function::P\n state::S\nend\n# convenience constructor:\nPullbackMemory(primal_function; state...) = PullbackMemory(primal_function, state)\n# convenience accessor so that `m.x` is same as `m.state.x`\nBase.getproperty(m::PullbackMemory, propname) = getproperty(getfield(m, :state), propname)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"So changing our API we have:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"y = f(x) # primal program\ny, pb = augmented_primal(f, x)\nx̄ = pullback_at(pb, ȳ)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"which is much cleaner.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y, cx = sincos(x)\n return y, PullbackMemory(sin; cx=cx)\nend\n\npullback_at(pb::PullbackMemory{typeof(sin)}, ȳ) = ȳ * pb.cx","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n ex = exp(x)\n y = ex / (1 + ex)\n return y, PullbackMemory(σ; y=y, ex=ex)\nend\n\npullback_at(pb::PullbackMemory{typeof(σ)}, ȳ) = ȳ * pb.y / (1 + pb.ex)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"That now looks much simpler; pullback_at only ever has 2 arguments.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"One way we could make it nicer to use is by making PullbackMemory a callable object. Conceptually, for a particular evaluation of an operation, the PullbackMemory is fixed. It is fully determined by the end of the primal pass. The during the gradient (reverse) pass the PullbackMemory is used to successively compute the ȳ argument. So it makes sense to make PullbackMemory a callable object that acts on the sensitivity. We can do that via call overloading:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"y = f(x) # primal program\ny, pb = augmented_primal(f, x)\nx̄ = pb(ȳ)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y, cx = sincos(x)\n return y, PullbackMemory(sin; cx=cx)\nend\n(pb::PullbackMemory{typeof(sin)})(ȳ) = ȳ * pb.cx","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n ex = exp(x)\n y = ex / (1 + ex)\n return y, PullbackMemory(σ; y=y, ex=ex)\nend\n\n(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y / (1 + pb.ex)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Let's recap what we have done here. We now have an object pb that acts on the cotangent of the output of the primal ȳ to give us the cotangent of the input of the primal function x̄. pb is not just the memory of state required for the pullback, it is the pullback.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"We have one final thing to do, which is to think about how we make the code easy to modify. Let's go back and think about the changes we would have make to go from our original way of writing that only used the inputs/outputs, to one that used the intermediate state.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"To rewrite that original formulation in the new pullback form we have:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y = sin(x)\n return y, PullbackMemory(sin; x=x)\nend\n(pb::PullbackMemory)(ȳ) = ȳ * cos(pb.x)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"To go from that to:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y, cx = sincos(x)\n return y, PullbackMemory(sin; cx=cx)\nend\n(pb::PullbackMemory)(ȳ) = ȳ * pb.cx","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n y = σ(x)\n return y, PullbackMemory(σ; y=y, x=x)\nend\n(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y * σ(-pb.x)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"to get to:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n ex = exp(x)\n y = ex/(1 + ex)\n return y, PullbackMemory(σ; y=y, ex=ex)\nend\n(pb::PullbackMemory{typeof(σ)})(ȳ) = ȳ * pb.y/(1 + pb.ex)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"We should think about how we might want to make future changes to this code.[6]","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"We need to make a series of changes:","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"update what work is done in the primal, to compute the intermediate values.\nupdate what is stored in the PullbackMemory.\nupdate the function that applies the pullback so it uses the new thing that was stored.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"It's important these parts all stay in sync. It's not too bad for this simple example with just one or two things to remember. For more complicated multi-argument functions, which we will show below, you often end up needing to remember half a dozen things, like sizes and indices relating to each input/output, so it gets a little more fiddly to make sure you remember all the things you need to and give them the same name in both places. Is there a way we can automatically just have all the things we use remembered for us? Surprisingly for such a specific request, there actually is: a closure.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"A closure in Julia is a callable structure that automatically contains a field for every object from its parent scope that is used in its body. There are incredible ways to abuse this; but here we can use closures exactly as they are intended. Replacing PullbackMemory with a closure that works the same way lets us avoid having to manually control what is remembered and lets us avoid separately writing the call overload.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for `sin`","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(sin), x)\n y, cx = sincos(x)\n pb = ȳ -> cx * ȳ # pullback closure. closes over `cx`\n return y, pb\nend","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
Example for the logistic sigmoid","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"function augmented_primal(::typeof(σ), x)\n ex = exp(x)\n y = ex / (1 + ex)\n pb = ȳ -> ȳ * y / (1 + ex) # pullback closure. closes over `y` and `ex`\n return y, pb\nend","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"
","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"This is pretty clean now.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Our augmented_primal is now within spitting distance of rrule. All that is left is a rename and some extra conventions around multiple outputs and gradients with respect to callable objects.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"This has been a journey into how we get to rrule as it is defined in ChainRulesCore. We started with an unaugmented primal function and a pullback_at function that only saw the inputs and outputs of the primal. We realized a key limitation of this was that we couldn't share computational work between the primal and gradient passes. To solve this we introduced the notation of some intermediate that is shared from the primal to the pullback. We successively improved that idea, first by making it a type that held everything that is needed for the pullback: the PullbackMemory, which we then made callable, so it was itself the pullback. Finally, we replaced that separate callable structure with a closure, which kept everything in one place and made it more convenient.","category":"page"},{"location":"design/changing_the_primal.html#More-Shared-Work-Examples","page":"Changing the Primal","title":"More Shared Work Examples","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"sin and the logistic sigmoid are nice, simple examples of when it is useful to share work between the primal and the pullback. There are many others though. It is actually surprising that in so many cases it is reasonable to write the rules where the only shared information between the primal and the pullback is the primal inputs (like our original sin), or primal outputs (like our original logistic sigmoid). Under our formulation above, those primal inputs/outputs are shared information just like any other. Beyond this, there are a number of other decent applications.","category":"page"},{"location":"design/changing_the_primal.html#getindex","page":"Changing the Primal","title":"getindex","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"In Julia (and many other numerical languages) indexing can take many more arguments than simply a couple of integers, such as boolean masking arrays (logical indexing), ranges for slices, etc. Converting the arguments to plain integers, arrays of integers, and ranges with Base.to_indices is the first thing that getindex does. It then re-calls getindex with these simpler types to get the result.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"The result of pulling back the getindex operation is always an array that is all zeros, except for the elements that are selected, which are set to the appropriate sensitivities being pulled back. To identify which actual positions in the array are being gotten/set is common work to both primal and gradient computations. We really don't want to deal with fancy indexing types during the pullback, because there are weird edge cases like indexing in such a way that the same element is output twice (and thus we have 2 sensitivities we need to add to it). We can pull the to_indices out of the primal computation and remember the plain indexes used, then can reuse them to set gradients during the pullback.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"See the code for this in ChainRules.jl","category":"page"},{"location":"design/changing_the_primal.html#exp(::Matrix)","page":"Changing the Primal","title":"exp(::Matrix)","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Matrix Functions are generalizations of scalar functions to operate on matrices. Note that this is distinct from simply element-wise application of the function to the matrix's elements. The Matrix Exponential exp(::Matrix) is a particularly important matrix function.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Al-Mohy and Higham (2009)[7], published a method for computing the pullback of exp(::Matrix). It is pretty complex and very cool. To quote its abstract (emphasis mine):","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"The algorithm is derived from the scaling and squaring method by differentiating the Padé approximants and the squaring recurrence, re-using quantities computed during the evaluation of the Padé approximant, and intertwining the recurrences in the squaring phase.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Julia does in fact use a Padé approximation to compute exp(::Matrix). So we can extract the code for that into our augmented primal, and add remembering the intermediate quantities that are to be used. See the code for this in ChainRules.jl","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"An interesting scenario here that may be of concern to some: if Julia changes the algorithm it uses to compute exp(::Matrix), then during an AD primal pass, it will continue to use the old Padé approximation based algorithm. This may actually happen, as there are many other algorithms that can compute the matrix exponential. Further, perhaps there might be an improvement to the exact coefficient or cut-offs used by Julia's current Padé approximation. If Julia made this change it would not be considered breaking. Exact floating point numerical values are not generally considered part of the SemVer-bound API. Rather only the general accuracy of the computed value relative to the true mathematical value (e.g. for common scalar operations Julia promises 1 ULP).","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"This change will result in the output of the AD primal pass not being exactly equal to what would be seen from just running the primal code. It will still be accurate because the current implementation is accurate, but it will be different. It is our argument that in general this should be considered acceptable, as long as the AD primal pass is in general about as accurate as the unaugmented primal. E.g. it might overshoot for some values the unaugmented primal undershoots for.","category":"page"},{"location":"design/changing_the_primal.html#eigvals","page":"Changing the Primal","title":"eigvals","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"eigvals is a real case where the algorithm for the augmented primal and the original primal is already different today. To compute the pullback of eigvals you need to know not only the eigenvalues but also the eigenvectors. The eigen function computes both, so that is used in the augmented primal. See the code for this in ChainRules.jl. If we could not compute and remember the eigenvectors in the primal pass, we would have to call eigen in the gradient pass anyway and fully recompute eigenvectors and eigenvalues, more than doubling the total work.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"However, if you trace this down, it actually uses a different algorithm.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"eigvals basically wraps LAPACK.syevr!('N', ...), which goes through DSYEVR and eventually calls DSTERF, which uses \"Pal-Walker-Kahan variant of the QL or QR algorithm.\" to compute eigenvalues","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"In contrast, eigen wraps LAPACK.syevr!('V',...) which also goes through DSYEVR but eventually calls DSTEMR, which calculates eigenvalues \"either by bisection or the dqds algorithm.\".","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Both of these are very good algorithms. LAPACK has had decades of work by experts and is one of the most trusted libraries for linear algebra. But they are different algorithms that give different results. The differences in practice are around 10^-15, which while very small on absolute terms are as far as Float64 is concerned a very real difference.","category":"page"},{"location":"design/changing_the_primal.html#Matrix-Division","page":"Changing the Primal","title":"Matrix Division","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"Roughly speaking: Y=A\\B is the function that finds the least-square solution to YA ≈ B. When solving such a system, the efficient way to do so is to factorize A into an appropriate factorized form such as Cholesky or QR, then perform the \\ operation on the factorized form. The pullback of A\\B with respect to B is Ȳ -> A' \\ Ȳ. It should be noted that this involves computing the factorization of A' (the adjoint of A).[8] In this computation the factorization of the original A can reused. Doing so can give a 4x speed-up.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"We don't have this in ChainRules.jl yet, because Julia is missing some definitions of adjoint of factorizations (JuliaLang/julia#38293).[8] We have been promised them for Julia v1.7 though. You can see what the code would look like in PR #302.","category":"page"},{"location":"design/changing_the_primal.html#Conclusion","page":"Changing the Primal","title":"Conclusion","text":"","category":"section"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"This document has explained why rrule is the way it is. In particular it has highlighted why the primal computation is able to be changed from simply calling the function. Further, it has explained why rrule returns a closure for the pullback, rather than it being a separate function. It has highlighted several places in ChainRules.jl where this has allowed us to significantly improve performance. Being able to change the primal computation is practically essential for a high performance AD system.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[1]: I am not just picking on Nabla randomly. Many of the core developers of ChainRules worked on Nabla prior. It's a good AD, but ChainRules incorporates lessons learned from working on Nabla.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[2]: which may be an explicit tape or an implicit tape that is actually incorporated into generated code (à la Zygote)","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[3]: Another key reason is if the operation is a primitive that is not defined in terms of more basic operations. In many languages this is the case for sin; where the actual implementation is in some separate libm.so. But actually sin in Julia is defined in terms of a polynomial. It's fairly vanilla Julia code. It shouldn't be too hard for an AD that only knows about basic operations like + and * to AD through it. In any case, that is another discussion for another day.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[4]: Sure, this is small fries and depending on Julia version might just get solved by the optimizer[5], but go with it for the sake of example.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[5]: To be precise, this is very likely to be solved by the optimizer inlining both and then performing common subexpression elimination, with the result that it generates the code for sincos just from having sin and cos inside the same function. However, this actually doesn't apply in the case of AD, as it is not possible to inline code called in the gradient pass into the primal pass. Those are separate functions called at very different times. This is something opaque closures should help solve.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[6]: One change we might consider is to have logistic sigmoid to only remember one thing. Rather than remembering y and ex to use in the pullback, we could compute y / (1 + ex) during the augmented primal, and just remember that.","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[7]: Al-Mohy, Awad H. and Higham, Nicholas J. (2009) Computing the Fréchet Derivative of the Matrix Exponential, with an application to Condition Number Estimation. SIAM Journal On Matrix Analysis and Applications., 30 (4). pp. 1639-1657. ISSN 1095-7162","category":"page"},{"location":"design/changing_the_primal.html","page":"Changing the Primal","title":"Changing the Primal","text":"[8]: To be clear here we mean adjoint as in the conjugate transpose of a matrix, rather than in the sense of reverse mode AD.","category":"page"},{"location":"design/many_tangents.html#manytypes","page":"Many Tangent Types","title":"Design Notes: The many-to-many relationship between tangent types and primal types","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"ChainRules has a system where one primal type (the type having its derivative taken) can have multiple possible tangent types (the type of the derivative); and where one tangent type can correspond to multiple primal types. This is in-contrast to the Swift AD efforts, which has one tangent type per primal type (Swift uses the term associated tangent type).","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"terminology: tangent and associated tangent type\nThe use of “associated tangent type” in AD is not technically correct, as they live in the cotangent plane instead of the tangent plane. However it is often reasonable for AD to treat the cotangent plane and tangent plane as the same thing, and this was an intentional choice by the Swift team. In ChainRules we use the term “tangent type” to refer to both tangents and cotangents.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"One thing to understand about tangents is that they have to form a vector space (or something very like them). They need to support addition to each other, they need a zero which doesn't change what it is added to, and they need to support scalar multiplication (this isn't really required, but it is handy for things like gradient descent). Beyond being a vector space, tangents need to be able to be added to a primal value to get back another primal value. Or roughly equivalently a tangent is a difference between two primal values.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"One thing to note in this example is that the primal does not have to be a vector. As an example, consider DateTime. A DateTime is not a vector space: there is no origin point, and DateTimes cannot be added to each other. The corresponding tangent type is any subtype of Period, such as Millisecond, Hour, Day etc.","category":"page"},{"location":"design/many_tangents.html#Natural-tangent","page":"Many Tangent Types","title":"Natural tangent","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"For a given primal type, we say a natural tangent type is one which people would intuitively think of as representing the difference between two primal values. It tends to already exist outside of the context of AD. So Millisecond, Hour, Day etc. are examples of natural tangents for the DateTime primal.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Note here that we already have a one primal type to many tangent types relationship. We have Millisecond and Hour and Day all being valid tangent types for DateTime. In this case we could convert them all to a single tangent type, such as Nanoseconds, but that is not always a reasonable decision: we may run in to overflow, or lots of allocations if we need to use a BigInt to represent the number of Nanosecond since the start of the universe. For types with more complex semantics, such as array types, these considerations are much more important.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Natural tangent types are the types people tend to think in, and thus the type they tend to write custom sensitivity rules in. An important special case of natural tangents is when the primal type is a vector space (e.g. Real,AbstractMatrix) in which case it is common for the natural tangent type to be the same as the primal type. One exception to this is getindex. The ideal choice of tangent type for getindex on a dense array would be some type of sparse array, due to the fact the derivative will have only one non-zero element. This actually further brings us to a weirdness of tangent types not actually being closed under addition, as it would be ideal for the sparse array to become a dense array if summed over all elements.","category":"page"},{"location":"design/many_tangents.html#Structural-tangent-types","page":"Many Tangent Types","title":"Structural tangent types","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"AD cannot automatically determine the natural tangent types for a primal. For some types we may be able to declare manually their natural tangent type. Other types will not have natural tangent types at all - e.g. NamedTuple, Tuple, WebServer, Flux.Dense - so we are destined to make some up. So beyond natural tangent types, we also have structural tangent types. ChainRules uses Tangent{P, <:NamedTuple} to represent a structural tangent type corresponding to primal type P. Zygote v0.4 uses NamedTuple.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Structural tangents are derived from the structure of the input. Either automatically, as part of the AD, or manually, as part of a custom rule.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Consider the structure of DateTime:","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"julia> dump(now())\nDateTime\n instant: UTInstant{Millisecond}\n periods: Millisecond\n value: Int64 63719890305605","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"The corresponding structural tangent is:","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Tangent{DateTime}(\n instant::Tangent{UTInstant{Millisecond}}(\n periods::Tangent{Millisecond}(\n value::Int64\n )\n )\n)","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"note: One must be allowed to take derivatives of integer arguments\nThis brings up another contrast to Swift. In Swift Int is considered non-differentiable, which is quite reasonable; it doesn’t have a very good definition of the limit of a small step (as that would be some floating/fixed point type). Int is intrinsically discrete. It is commonly used for indexing, and if one takes a gradient step, say turning x[2] into x[2.1] then that is an error. However, disallowing Int to be used as a tangent means we cannot handle cases like DateTime having an inner field of milliseconds counted as an integer from the unix epoch or other cases where an integer is used as a convenience for computational efficiency. In the case where a custom sensitivity rule claims that there is a non-zero derivative for an Int argument that is being used for indexing, that code is simply wrong. We can’t handle incorrect code and trying to is a path toward madness. Julia, unlike Swift, is not well suited to handling rules about what you can and can’t do with particular types.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"So the structural tangent is another type of tangent. We must support both natural and structural tangents because AD can only create structural tangents (unless using custom sensitivity rules) and all custom sensitivities are only written in terms of natural tangents, as that is what is used in papers about derivatives.","category":"page"},{"location":"design/many_tangents.html#Semi-structural-tangents","page":"Many Tangent Types","title":"Semi-structural tangents","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Where there is no natural tangent type for the outermost type but there is for some of its fields, we call this a \"semi-structural\" tangent.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Consider if we had a representation of a country's GDP as output by some continuous time model like a Gaussian Process, where that representation is as a sequence of TimeSamples structured as follows:","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"julia> struct TimeSample\n time::DateTime\n value::Float64\n end","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"We can look at its structure:","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"julia> dump(TimeSample(now(), 2.6e9))\nTimeSample\n time: DateTime\n instant: Dates.UTInstant{Millisecond}\n periods: Millisecond\n value: Int64 63720043490844\n value: Float64 2.6e9","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Thus we see the that structural tangent would be:","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Tangent{TimeSample}(\n time::Tangent{DateTime}(\n instant::Tangent{UTInstant{Millisecond}}(\n periods::Tangent{Millisecond}(\n value::Int64\n )\n )\n ),\n value::Float64\n)","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"But instead in the custom sensitivity rule we would write a semi-structured tangent type. Since there is not a natural tangent type for TimeSample but there is for DateTime.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Tangent{TimeSample}(\n time::Day,\n value::Float64\n)","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"So the rule author has written a structural tangent with some fields that are natural tangents.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Another related case is for types that overload getproperty such as SVD and QR. In this case the structural tangent will be based on the fields, but those fields do not always have an easy relation to what is actually used in math. For example, the QR type has fields factors and t, but we would more naturally think in terms of the properties Q and R. So most rule authors would want to write semi-structural tangents based on the properties.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"To return to the question of why ChainRules has Tangent{P, <:NamedTuple} whereas Zygote v0.4 just has NamedTuple, it relates to semi-structural derivatives, and being able to overload things more generally. If one knows that one has a semi-structural derivative based on property names, like Tangent{QR}(Q=..., R=...), and one is adding it to the true structural derivative based on field names Tangent{QR}(factors=..., τ=...), then we need to overload the addition operator to perform that correctly. We cannot happily overload similar things for NamedTuple since we don't know the primal type, only the names of the values contained. In fact we can't actually overload addition at all for NamedTuple as that would be type-piracy, so have to use Zygote.accum instead.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Another use of the primal being a type parameter is to catch errors. ChainRules disallows the addition of Tangent{SVD} to Tangent{QR} since in a correctly differentiated program that can never occur.","category":"page"},{"location":"design/many_tangents.html#Tangent-types-for-computational-efficiency","page":"Many Tangent Types","title":"Tangent types for computational efficiency","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"There is another kind of unnatural tangent. One that is for computational efficiency. ChainRules has Thunks and InplaceableThunks, which wrap the computation of a derivative and delays that work until it is needed, either via the derivative being added to something or being unthunked manually, thus saving time if it is never used.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Another tangent type used for efficiency is ZeroTangent which represents the hard zero (in Zygote v0.4 this is nothing). For example the derivative of f(x, y)=2x with respect to y is ZeroTangent(). Add ZeroTangent() to anything, and one gets back the original thing without change. We noted that all tangents need to be a vector space. ZeroTangent() is the trivial vector space. Further, add ZeroTangent() to any primal value (no matter the type) and you get back another value of the same primal type (the same value in fact). So it meets the requirements of a tangent type for all primal types. ZeroTangent can save on memory (since we can avoid allocating anything) and on time (since performing the multiplication ZeroTangent and Thunk are both examples of a tangent type that is valid for multiple primal types.","category":"page"},{"location":"design/many_tangents.html#Conclusion","page":"Many Tangent Types","title":"Conclusion","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"Now, you have seen examples of both tangent types that work for multiple primal types, and primal types that have multiple valid tangent types. Semantically we can handle these very easily in julia. Just put in a few more dispatching on +. Multiple-dispatch is great like that. The down-side is our type-inference becomes hard. If you have exactly 1 tangent type for each primal type, you can very easily work out what all the types on your reverse pass will be - you don't really need type inference - but you lose so much expressiveness.","category":"page"},{"location":"design/many_tangents.html#Appendix:-What-Swift-does","page":"Many Tangent Types","title":"Appendix: What Swift does","text":"","category":"section"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"I don't know how Swift is handling thunks, maybe they are not, maybe they have an optimizing compiler that can just slice out code-paths that don't lead to values that get used; maybe they have a language built in for lazy computation.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"They are, as I understand it, handling ZeroTangent by requiring every tangent type to define a zero method – which it has since it is a vector space. This costs memory and time, but probably not actually all that much. With regards to handling multiple different tangent types for one primal, like natural and structural derivatives, everything needs to be converted to the canonical tangent type of that primal.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"As I understand it, things can be automatically converted by defining conversion protocols or something like that, so rule authors can return anything that has a conversion protocol to the canonical tangent type of the primal.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"However, it seems like this will run into problems. Recall that the natural tangent in the case of getindex on an AbstractArray was a sparse array. But for say the standard dense Array, the only reasonable canonical tangent type is also a dense Array. But if you convert a sparse array into a dense array you do giant allocations to fill in all the other entries with zero.","category":"page"},{"location":"design/many_tangents.html","page":"Many Tangent Types","title":"Many Tangent Types","text":"So this is the story about why we have many-to-many tangent types in ChainRules.","category":"page"},{"location":"index.html#ChainRules","page":"Introduction","title":"ChainRules","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Automatic differentiation (AD) is a set of techniques for obtaining derivatives of arbitrary functions. There are surprisingly many packages for doing AD in Julia. ChainRules isn't one of these packages.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"The AD packages essentially combine derivatives of simple functions into derivatives of more complicated functions. They differ in the way they break down complicated functions into simple ones, but they all require a common set of derivatives of simple functions (rules).","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ChainRules is an AD-independent set of rules, and a system for defining and testing rules.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"note: What is a rule?\nA rule encodes knowledge about propagating derivatives, e.g. that the derivative with respect to x of a*x is a, and the derivative of sin(x) is cos(x), etc.","category":"page"},{"location":"index.html#ChainRules-ecosystem-organisation","page":"Introduction","title":"ChainRules ecosystem organisation","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"The ChainRules ecosystem comprises:","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ChainRulesCore.jl: a system for defining rules, and a collection of tangent types.\nChainRules.jl: a collection of rules for Julia Base and standard libraries.\nChainRulesTestUtils.jl: utilities for testing rules using finite differences.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"AD systems depend on ChainRulesCore.jl to get access to tangent types and the core rule definition functionality (frule and rrule), and on ChainRules.jl to benefit from the collection of rules for Julia Base and the standard libraries.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Packages that just want to define rules only need to depend on ChainRulesCore.jl, which is an exceptionally light dependency. They should also have a test-only dependency on ChainRulesTestUtils.jl to test the rules using finite differences.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Note that the packages with rules do not have to depend on AD systems, and neither do the AD systems have to depend on individual packages.","category":"page"},{"location":"index.html#ChainRules-roll-out-status","page":"Introduction","title":"ChainRules roll-out status","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Numerous packages depend on ChainRulesCore to define rules for their functions.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"6 AD engines currently use ChainRules to get access to rules:","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Zygote.jl is a reverse-mode AD that supports using rrules, calling back into AD, and opting out of rules. However, its own ZygoteRules.jl primitives (@adjoints) take precedence before rrules when both are defined – even if the @adjoint is less specific than the rrule. Internally it uses its own set of tangent types, e.g. nothing instead of NoTangent/ZeroTangent. It also unthunks every tangent.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Diffractor.jl is a forward- and reverse-mode AD that fully supports ChainRules, including calling back into AD, opting out of rules, and uses tangent types internally.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Yota is a reverse-mode AD that fully supports ChainRules, including calling back into AD, opting out of rules, and uses tangent types internally.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ReverseDiff is a reverse-mode AD that supports using rrules, but not calling back into AD and opting out of rules.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Nabla.jl is a reverse-mode AD that supports using rrules, but not opting out of rules, nor calling back into AD.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ReversePropagation.jl is a reverse-mode AD that supports using rrules for scalar functions, but not calling back into AD and opting out of rules.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"On the other hand, ForwardDiff.jl is NOT natively compatible with ChainRules. You can use the package ForwardDiffChainRules.jl to bridge this gap.","category":"page"},{"location":"index.html#Key-functionality","page":"Introduction","title":"Key functionality","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Consider a relationship y = f(x), where f is some function. Computing y from x is the original problem, called the primal computation, in contrast to the problem of computing derivatives. We say that the primal function f takes a primal input x and returns the primal output y.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ChainRules rules are concerned with propagating tangents of primal inputs to tangents of primal outputs (frule, from forwards mode AD), and propagating cotangents of primal outputs to cotangents of primal inputs (rrule, from reverse mode AD). To be able to do that, ChainRules also defines a small number of tangent types to represent tangents and cotangents.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"note: Tangents and cotangents\nStrictly speaking tangents, x = fracdxda, are propagated in frules, and cotangents, x = fracdadx, are propagated in rrules. However, in practice there is rarely a need to distinguish between the two: both are represented by the same tangent types. Thus, except when the detail might clarify, we refer to both as tangents.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"terminology: `frule` and `rrule`\nfrule and rrule are ChainRules specific terms. Their exact functioning is fairly ChainRules specific, though other tools have similar functions. The core notion is sometimes called custom AD primitives, custom adjoints, custom gradients, custom sensitivities. The whole field is a mess for terminology.","category":"page"},{"location":"index.html#Forward-mode-AD-rules-(frules)","page":"Introduction","title":"Forward-mode AD rules (frules)","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"If we know the value of x = fracdxda for some a and we want to know y = fracdyda, the chain rule tells us that y = fracdydx x. Intuitively, we are pushing the derivative forward. This is the basis for forward-mode AD.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"note: frule\nThe frule for f encodes how to propagate the tangent of the primal input (x) to the tangent of the primal output (y).","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"The frule signature for a function foo(args...; kwargs...) is","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"function frule((Δself, Δargs...), ::typeof(foo), args...; kwargs...)\n ...\n return y, ∂Y\nend","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"where y = foo(args; kwargs...) is the primal output, and ∂Y is the result of propagating the input tangents Δself, Δargs... forwards at the point in the domain of foo described by args. This propagation is called the pushforward. Often we will think of the frule as having the primal computation y = foo(args...; kwargs...), and the pushforward ∂Y = pushforward(Δself, Δargs...), even though they are not present in separate forms in the code.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"For example, the frule for sin(x) is:","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"function frule((_, Δx), ::typeof(sin), x)\n return sin(x), cos(x) * Δx\nend","category":"page"},{"location":"index.html#Reverse-mode-AD-rules-(rrules)","page":"Introduction","title":"Reverse-mode AD rules (rrules)","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"If we know the value of y = fracdady for some a and we want to know x = fracdadx, the chain rule tells us that x =y fracdydx. Intuitively, we are pushing the derivative backward. This is the basis for reverse-mode AD.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"note: rrule\nThe rrule for f encodes how to propagate the cotangents of the primal output (y) to the cotangent of the primal input (x).","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"The rrule signature for a function foo(args...; kwargs...) is","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"function rrule(::typeof(foo), args...; kwargs...)\n ...\n return y, pullback\nend","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"where y (the primal output) must be equal to foo(args...; kwargs...). pullback is a function to propagate the derivative information backwards at the point in the domain of foo described by args. That pullback function is used like: ∂self, ∂args... = pullback(Δy) Almost always the pullback will be declared locally within the rrule, and will be a closure over some of the other arguments, and potentially over the primal result too.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"For example, the rrule for sin(x) is:","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"function rrule(::typeof(sin), x)\n sin_pullback(Δy) = (NoTangent(), cos(x)' * Δy)\n return sin(x), sin_pullback\nend","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"note: Why `rrule` returns a pullback but `frule` doesn't return a pushforward\nWhile rrule takes only the arguments to the original function (the primal arguments) and returns a function (the pullback) that operates with the derivative information, the frule does it all at once. This is because the frule fuses the primal computation and the pushforward. This is an optimization that allows frules to contain single large operations that perform both the primal computation and the pushforward at the same time (for example solving an ODE). This operation is only possible in forward mode (where frule is used) because the derivative information needed by the pushforward available with the frule is invoked – it is about the primal function's inputs. In contrast, in reverse mode the derivative information needed by the pullback is about the primal function's output. Thus the reverse mode returns the pullback function which the caller (usually an AD system) keeps hold of until derivative information about the output is available.","category":"page"},{"location":"index.html#Tangent-types","page":"Introduction","title":"Tangent types","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"The types of (co)-tangents depend on the types of the primals. Scalar primals are represented by scalar tangents (e.g. Float64 tangent for a Float64 primal). Vector, matrix, and higher rank tensor primals can be represented by vector, matrix and tensor tangents.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"ChainRules defines a Tangent tangent type to represent tangents of structs, Tuples, NamedTuples, and Dicts.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Additionally, for signalling semantics, we distinguish between two tangent types representing a zero tangent. NoTangent type represent situations in which the tangent space does not exist, e.g. an index into an array can not be perturbed. ZeroTangent is used for cases where the tangent happens to be zero, e.g. because the primal argument is not used in the computation.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"We also define Thunks to allow certain optimisation. Thunks are a wrapper over a computation that can potentially be avoided, depending on the downstream use.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"See the section on tangent types for more details.","category":"page"},{"location":"index.html#Example-of-using-ChainRules-directly","page":"Introduction","title":"Example of using ChainRules directly","text":"","category":"section"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"While ChainRules is largely intended as a backend for autodiff systems, it can be used directly. In fact, this can be very useful if you can constrain the code you need to differentiate to only use things that have rules defined for. This was once how all neural network code worked.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"Using ChainRules directly also helps get a feel for it.","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"using ChainRulesCore\n\nfunction foo(x)\n a = sin(x)\n b = 0.2 + a\n c = asin(b)\n return c\nend\n\n# Define rules (alternatively get them for free via `using ChainRules`)\n@scalar_rule(sin(x), cos(x))\n@scalar_rule(+(x, y), (1.0, 1.0))\n@scalar_rule(asin(x), inv(sqrt(1 - x^2)))","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"#### Find dfoo/dx via rrules\n#### First the forward pass, gathering up the pullbacks\nx = 3;\na, a_pullback = rrule(sin, x);\nb, b_pullback = rrule(+, 0.2, a);\nc, c_pullback = rrule(asin, b)\n\n#### Then the backward pass calculating gradients\nc̄ = 1; # ∂c/∂c\n_, b̄ = c_pullback(c̄); # ∂c/∂b = ∂c/∂b ⋅ ∂c/∂c\n_, _, ā = b_pullback(b̄); # ∂c/∂a = ∂c/∂b ⋅ ∂b/∂a\n_, x̄ = a_pullback(ā); # ∂c/∂x = ∂c/∂a ⋅ ∂a/∂x\nx̄ # ∂c/∂x = ∂foo/∂x\n# output\n-1.0531613736418153","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"#### Find dfoo/dx via frules\nx = 3;\nẋ = 1; # ∂x/∂x\nnofields = ZeroTangent(); # ∂self/∂self\n\na, ȧ = frule((nofields, ẋ), sin, x); # ∂a/∂x = ∂a/∂x ⋅ ∂x/∂x \nb, ḃ = frule((nofields, ZeroTangent(), ȧ), +, 0.2, a); # ∂b/∂x = ∂b/∂a ⋅ ∂a/∂x\nc, ċ = frule((nofields, ḃ), asin, b); # ∂c/∂x = ∂c/∂b ⋅ ∂b/∂x\nċ # ∂c/∂x = ∂foo/∂x\n# output\n-1.0531613736418153","category":"page"},{"location":"index.html","page":"Introduction","title":"Introduction","text":"#### Find dfoo/dx via FiniteDifferences.jl\nusing FiniteDifferences\ncentral_fdm(5, 1)(foo, x)\n# output\n-1.0531613736418257\n\n#### Find dfoo/dx via ForwardDiff.jl\nusing ForwardDiff\nForwardDiff.derivative(foo, x)\n# output\n-1.0531613736418153\n\n#### Find dfoo/dx via Zygote.jl\nusing Zygote\nZygote.gradient(foo, x)\n# output\n(-1.0531613736418153,)","category":"page"},{"location":"ad_author/opt_out.html#Support-opting-out-of-rules","page":"Support opting out of rules","title":"Support opting out of rules","text":"","category":"section"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"We provide two ways to know that a rule has been opted out of.","category":"page"},{"location":"ad_author/opt_out.html#rrule-/-frule-returns-nothing","page":"Support opting out of rules","title":"rrule / frule returns nothing","text":"","category":"section"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"@opt_out defines a frule or rrule matching the signature that returns nothing.","category":"page"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"If you are in a position to generate code, in response to values returned by function calls then you can do something like:","category":"page"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"res = rrule(f, xs)\nif res === nothing\n y, pullback = perform_ad_via_decomposition(r, xs) # do AD without hitting the rrule\nelse\n y, pullback = res\nend","category":"page"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"The Julia compiler will specialize based on inferring the return type of rrule, and so can remove that branch.","category":"page"},{"location":"ad_author/opt_out.html#no_rrule-/-no_frule-has-a-method","page":"Support opting out of rules","title":"no_rrule / no_frule has a method","text":"","category":"section"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"@opt_out also defines a method for ChainRulesCore.no_frule or ChainRulesCore.no_rrule. The body of this method doesn't matter, what matters is that it is a method-table. A simple thing you can do with this is not support opting out. To do this, filter all methods from the rrule/frule method table that also occur in the no_frule/no_rrule table. This will thus avoid ever hitting an rrule/frule that returns nothing (and thus prevents your library from erroring). This is easily done, though it does mean ignoring the user's stated desire to opt out of the rule.","category":"page"},{"location":"ad_author/opt_out.html","page":"Support opting out of rules","title":"Support opting out of rules","text":"More complex you can use this to generate code that triggers your AD. If for a given signature there is a more specific method in the no_rrule/no_frule method-table, than the one that would be hit from the rrule/frule table (Excluding the one that exactly matches which will return nothing) then you know that the rule should not be used. You can, likely by looking at the primal method table, workout which method you would have it if the rule had not been defined, and then invoke it.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html#config","page":"RuleConfig","title":"Rule configurations and calling back into AD","text":"","category":"section"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"RuleConfig is a method for making rules conditionally defined based on the presence of certain features in the AD system. One key such feature is the ability to perform AD either in forwards or reverse mode or both.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"This is done with a trait-like system (not Holy Traits), where the RuleConfig has a union of types as its only type-parameter. Where each type represents a particular special feature of this AD. To indicate that the AD system has a special property, its RuleConfig should be defined as:","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"struct MyADRuleConfig <: RuleConfig{Union{Feature1, Feature2}} end","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"And rules that should only be defined when an AD has a particular special property write:","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"rrule(::RuleConfig{>:Feature1}, f, args...) = # rrule that should only be define for ADs with `Feature1`\n\nfrule(::RuleConfig{>:Union{Feature1,Feature2}}, f, args...) = # frule that should only be define for ADs with both `Feature1` and `Feature2`","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"warning: Rules with Config always take precedence over rules without\nEven if the other arguments are more specific the rule with the config will always take precedence. For example of there is a rule rrule(::RuleConfig, ::typeof(foo), ::Any) and other rrule(foo, ::Float64), the first will always be selected. This is because the AD will always attempt to provide its config when checking for a rule, and only if that doesn't match, will the config-less rule be tried. In practice this doesn't happen often, but when it does the solution is a little ugly – though very similar to resolving method ambiguities. You need to manually add methods that dispatch from a rule with config to the one without. See for example the rule for sum(abs2, xs) in ChainRules.jl.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"A prominent use of this is in declaring that the AD system can, or cannot support being called from within the rule definitions.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html#Writing-rules-that-call-back-into-AD","page":"RuleConfig","title":"Writing rules that call back into AD","text":"","category":"section"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"To define e.g. rules for higher order functions, it is useful to be able to call back into the AD system to get it to do some work for you.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"For example the rule for reverse mode AD for map might like to use forward mode AD if one is available. Particularly for the case where only a single input collection is being mapped over. In that case we know the most efficient way to compute that sub-program is in forwards, as each call with-in the map only takes a single input.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"Note: the following is not the most efficient rule for map via forward, but attempts to be clearer for demonstration purposes.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"function rrule(config::RuleConfig{>:HasForwardsMode}, ::typeof(map), f::Function, x::Array{<:Real})\n # real code would support functors/closures, but in interest of keeping example short we exclude it:\n @assert (fieldcount(typeof(f)) == 0) \"Functors/Closures are not supported\"\n\n y_and_ẏ = map(x) do xi\n frule_via_ad(config, (NoTangent(), one(xi)), f, xi)\n end\n y = first.(y_and_ẏ)\n ẏ = last.(y_and_ẏ)\n\n pullback_map(ȳ) = NoTangent(), NoTangent(), ȳ .* ẏ\n return y, pullback_map\nend","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html#Writing-rules-that-depend-on-other-special-requirements-of-the-AD.","page":"RuleConfig","title":"Writing rules that depend on other special requirements of the AD.","text":"","category":"section"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"The >:HasReverseMode and >:HasForwardsMode are two examples of special properties that a RuleConfig could allow. Others could also exist, but right now they are the only two. It is likely that in the future other such will be provided ","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"Note: you can only depend on the presence of a feature, not its absence. This means we may need to define features and their complements, when one is not the obvious default (as in the case of HasReverseMode/NoReverseMode and HasForwardsMode/NoForwardsMode.).","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"Such special properties generally should only be defined in ChainRulesCore. (Theoretically, they could be defined elsewhere, but the AD and the package containing the rule need to load them, and ChainRulesCore is the place for things like that.)","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html#Writing-rules-that-are-only-for-your-own-AD","page":"RuleConfig","title":"Writing rules that are only for your own AD","text":"","category":"section"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"A special case of the above is writing rules that are defined only for your own AD. Rules which otherwise would be type-piracy, and would affect other AD systems. This could be done via making up a special property type and dispatching on it. But there is no need, as we can dispatch on the RuleConfig subtype directly.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"For example in order to avoid mutation in nested AD situations, Zygote might want to have a rule for add!! that makes it just do +.","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"struct ZygoteConfig <: RuleConfig{Union{}} end\n\nrrule(::ZygoteConfig, typeof(ChainRulesCore.add!!), a, b) = a+b, Δ->(NoTangent(), Δ, Δ)","category":"page"},{"location":"rule_author/superpowers/ruleconfig.html","page":"RuleConfig","title":"RuleConfig","text":"As an alternative to rules only for one AD, would be to add new special property definitions to ChainRulesCore (as described above) which would capture what makes that AD special.","category":"page"}] +} diff --git a/previews/PR663/siteinfo.js b/previews/PR663/siteinfo.js new file mode 100644 index 000000000..7741b41f3 --- /dev/null +++ b/previews/PR663/siteinfo.js @@ -0,0 +1 @@ +var DOCUMENTER_CURRENT_VERSION = "previews/PR663"; diff --git a/previews/PR663/videos.html b/previews/PR663/videos.html new file mode 100644 index 000000000..15a8fba9f --- /dev/null +++ b/previews/PR663/videos.html @@ -0,0 +1,8 @@ + +Videos · ChainRules
GitHub

Videos

For people who learn better by video we have a number of videos of talks we have given about the ChainRules project. Note however, that the videos are frozen in time reflecting the state of the packages at the time they were recorded. This documentation is the continuously updated canonical source. However, we have tried to note below each video notes on its correctness.

The talks that follow are in reverse chronological order (i.e. most recent video is first).

EuroAD 2021: ChainRules.jl: AD system agnostic rules for JuliaLang

Presented by Frames White. Slides

This is the talk to watch if you want to understand why the ChainRules project exists, what its challenges are, and how those have been overcome. It is intended less for users of the package, and more for people working in the field of AD more generally. It does also serve as a nice motivation for those first coming across the package as well though.

+ +

Abstract:

The ChainRules project is a suite of JuliaLang packages that define custom primitives (i.e. rules) for doing AD in JuliaLang. Importantly it is AD system agnostic. It has proved successful in this goal. At present it works with about half a dozen different JuliaLang AD systems. It has been a long journey, but as of August 2021, the core packages have now hit version 1.0.

This talk will go through why this is useful, the particular objectives the project had, and the challenges that had to be solved. This talk is not intended as an educational guide for users (For that see our 2021 JuliaCon talk: > Everything you need to know about ChainRules 1.0 (https://live.juliacon.org/talk/LWVB39)). Rather this talk is to share the insights we have had, and likely (inadvertently) the mistakes we have made, with the wider autodiff community. We believe these insights can be informative and useful to efforts in other languages and ecosystems.

JuliaCon 2021: Everything you need to know about ChainRules 1.0

Presented by Miha Zgubič. Slides

If you are just wanting to watch a video to learn all about ChainRules and how to use it, watch this one.

Slide on opting out is incorrect

Slide 42 is incorrect (@no_rrule sum_array(A::Diagonal)), in the ChainRulesCore 1.0 release the following syntax is used: @opt_out rrule(::typeof(sum_array), A::Diagonal). This syntax allows us to include rule config information.

+ +

Abstract:

ChainRules is an automatic differentiation (AD)-independent ecosystem for forward-, reverse-, and mixed-mode primitives. It comprises ChainRules.jl, a collection of primitives for Julia Base, ChainRulesCore.jl, the utilities for defining custom primitives, and ChainRulesTestUtils.jl, the utilities to test primitives using finite differences. This talk provides brief updates on the ecosystem since last year and focuses on when and how to write and test custom primitives.

JuliaCon 2020: ChainRules.jl

Presented by Frames White. Slides

This talk is primarily of historical interest. This was the first public presentation of ChainRules. Though the project was a few years old by this stage. A lot of things are still the same; conceptually, but a lot has changed. Most people shouldn't watch this talk now.

Outdated Terminology

A lot of terminology has changed since this presentation.

  • DoesNotExist → NoTangent
  • Zero → ZeroTangent
  • Composite{P} → Tangent{T}

The talk also says differential in a lot of places where we now would say tangent.

+ +

Abstract:

The ChainRules project allows package authors to write rules for custom sensitivities (sometimes called custom adjoints) in a way that is not dependent on any particular autodiff (AD) package. It allows authors of AD packages to access a wealth of prewritten custom sensitivities, saving them the effort of writing them all out themselves. ChainRules is the successor to DiffRules.jl and is the native rule system currently used by ForwardDiff2, Zygote and soon ReverseDiff