Constraints engine materializes entire constraint

[wouterwln]

Should instead check all neighbors and resolve constraint locally as a BitSetTuple

[wouterwln]

Documenting this since this problem is way harder than it seems.
When we get a constraint we get the in-context representation of the constraint. Let's look at the following setting.

@model function inner_model(a, b, c)
    x ~ Normal(a, b)
    c := exp(x)
end

@model function outer_model(in1, in2, out)
    x ~ Gamma(in1, in2)
    out ~ inner_model(a = 1, b = x)
end

With the following constraints:

@constraints begin
    for q in inner_model
        q(x, a, b) = q(x)q(a,b)
    end
end

This constraint should push a constraint to the Normal node in the inner_model instance. This means we get te following constraint representation:

FactorizationConstraint(
    (
         IndexedVariable(:x, nothing),
         IndexedVariable(:a, nothing),
         IndexedVariable(:b, nothing),
     ),
     [
        FactorizationConstraintEntry([
             IndexedVariable(:x, nothing),
         ]),
         FactorizationConstraintEntry([
             IndexedVariable(:a, nothing),
             IndexedVariable(:b, nothing),
         ]),
     ],
)

Now the context contains a mapping from these :x, :a and :b, so we are perfectly able to find which nodes in the graph are being referenced by this constraint. However, this materialization of the constraint is too expensive, and this is exactly what we're trying to avoid. (This is what is currently implemented, and it works, but it is slow).

Another approach is that we start from the neighbors of the Normal node, Since we have that :a and :b are actually interfaces of the submodel, they are being created by the context of outer_model, and therefore a call to neighbors(model, normal_node) would result in the following:

[GraphPPL.NodeLabel(:x, 5), GraphPPL.NodeLabel(:x, 9), GraphPPL.NodeLabel(:constvar_9, 10)]

This illustrates the problem. Since the variable we try to reference with x in our constraint and the variable we are trying to reference with a in the constraint are both called :x in the respective contexts that created them. Long story short: the mapping from neighbor labels -> constraint labels is not a mathematically valid mapping as :x points to 2 different labels in the constraint.

Also, if we replace the constraints with the "global" representation of the variables mentioned we get into trouble, as the constraint specified above would translate to:

FactorizationConstraint(
    (
         IndexedVariable(:x, nothing),
         IndexedVariable(:x, nothing),
         IndexedVariable(:constvar_9, nothing),
     ),
     [
        FactorizationConstraintEntry([
             IndexedVariable(:x, nothing),
         ]),
         FactorizationConstraintEntry([
             IndexedVariable(:x, nothing),
             IndexedVariable(:constvar_9, nothing),
         ]),
     ],
)

giving the same problem as above: x is referenced twice here.

We also cannot store the NodeLabels directly in the FactorizationConstraint as this would mean that we would have to materialize every indexed statement.

@bvdmitri I'm curious about your thoughts on this :)

[albertpod]

ping @wouterwln

[wouterwln]

@albertpod this is currently WIP, ETA is somewhere at the end of next week. I'll update the timeline