CalcFactor.cache in Parametric case

[dehann]

Users can now overload:

IIF.preambleCache(dfg::AbstractDFG, vars::AbstractVector{Symbol}, usrfnc::AbstractFactor)

which runs once at factor creation. Cache is stored at ccw.dummyCache::C and CalcFactor.cache::C.

Parametric code needs to be refactored to allow preampleCache(dfg, vars, usrfnc) function to run here:

IncrementalInference.jl/src/ParametricUtils.jl

Lines 116 to 118 in fe40bd8

	# FIXME #1480, cache = preambleCache(dfg,vars,usrfnc)
	cache = nothing
	calcf = CalcFactor(getFactorMechanics(cf), _getFMdThread(fct), 0, 0, (), [], true, cache)

[Affie]

This breaks a lot of things in parametric. I'll need some help understanding this.

[dehann]

Okay, we can do a worksession. First week in April?

[Affie]

@dehann , is preamble cache second parameter supposed to be
AbstractVector{<:DFGVariable} or
AbstractVector{Symbol}

[dehann]

Hi Johan, sorry I don't quite follow your question? I think you mean what is the signature for preambleCache (see below)

So default for a factor is cache::Nothing. That gets put into CalcFactor.cache::Nothing. So getSample and residual functions will just see cf.cache === nothing.

If one goes and overrides IIF.preambleCache(dfg::AbstractDFG, vars::AbstractVector{<:DFGVariable}, fct::MyFactor) and returns an object, say type ::T. Then once the thing is going, the getSample and residual functions will see that as cf.cache::T, and the user can put whatever they want in the cache (links to YouTube if it so turns out).

Here are the docs for preambleCache:

• https://juliarobotics.org/Caesar.jl/latest/concepts/stash_and_cache/#IncrementalInference.preambleCache

I tried to capture most of the headline stuff in docs, perhaps that helps if above does not answer yet?

• https://juliarobotics.org/Caesar.jl/latest/concepts/stash_and_cache/

Happy to answer more?

[Affie]

The second parameter here: IIF.preambleCache(dfg::AbstractDFG, vars::AbstractVector{<:DFGVariable}, fct::MyFactor) is not consistent in source and docs.
It is the vector of symbols in some places (including docs) AbstractVector{Symbol}

[Affie]

From doc used as symbols:
addFactor!(fg, [:a;:b], mfc)

[dehann]

Not following on inconsistency. yes addFactor uses Vector of Symbol. preambleCache is different, it uses Vector of <:DFGVariable

[Affie]

Sorry copied wrong line, see RoME in places:

https://github.com/JuliaRobotics/RoME.jl/blob/1f9795a06994169e946506686171894aabccf714/src/factors/Bearing2D.jl#L18
and
https://github.com/JuliaRobotics/RoME.jl/blob/1f9795a06994169e946506686171894aabccf714/src/factors/DynPose2D.jl#L145

I'm guessing it's just a bug then?

[Affie]

The doc I wanted to reference is the news:

IncrementalInference.jl/NEWS.md

Line 33 in 16e2a90

- Add factor `preambleCache(dfg, vlbls, usrfnc)`, #1462, #1466. Doesn't work for parametric yet (#1480).

[dehann]

oh sorry, let's fix that

[dehann]

is that better?

[dehann]

Oh, that is a bug yes. Should be ::AbstractVector{<:DFGVariable}

[Affie]

It looks like for simple factors such as Pose2Pose2 the cache doesn't make a real difference, here is Pose2Pose2 without a cache:

[dehann]

My sense is to get performance, we need to get allocations down to 0. The only way to do that is with in-place operations everywhere. The only way we can do in-place is to preallocate memory somewhere, and the only way I see to do that is with cf.cache?

[Affie]

Currently, we cannot get allocations down to 0, this example uses SArrays for p,q and MArray for X_hat

julia> @benchmark log!(M, X_hat, p, q)
BenchmarkTools.Trial: 10000 samples with 983 evaluations.
 Range (min … max):   61.297 ns … 38.177 μs  ┊ GC (min … max):  0.00% … 99.67%
 Time  (median):      66.344 ns              ┊ GC (median):     0.00%
 Time  (mean ± σ):   110.464 ns ±  1.131 μs  ┊ GC (mean ± σ):  30.69% ±  2.99%

  ▆██▇▆▅▄▃▂▁▁▁▁▁▁                    ▂▃▂▄▅▂                    ▂
  ██████████████████▇▇▆▆▆▆▇█▇▇█▆▆▆▆▆▆███████▇▇▇▆▅▆▆▆▃▅▅▃▃▅▄▃▃▄ █
  61.3 ns       Histogram: log(frequency) by time       157 ns <

 Memory estimate: 192 bytes, allocs estimate: 4.

[Affie]

Also, remember M = getManifold(Pose2) for example uses constant propagation so it's faster not to fetch out of the cache (as far as I understand).

[Affie]

I might have a workaround for my parametric issues
nope, looks like it was another bug that caused the solver to exit sooner.

[dehann]

we cannot get allocations down to 0, this example uses SArrays for p,q and MArray for X_hat

Right, so first cut is then to get everything on the stack and avoid allocations on the heap?

Also, remember M = getManifold(Pose2) for example uses constant propagation so it's faster not to fetch out of the cache (as far as I understand).

Okay, I'm not knowledgeable enough to easily answer that, but sounds like it can be faster.

the only way I see to do that is with cf.cache?

oops, there is another way i forgot about. The factor struct itself can hold memory or fields, but user needs to be careful on how to use multithreading. Part of cf.cache was that IIF can manage more of the multithreading memory. Separate story.

Another thing, I was thinking we should get cf / cf.cache completely allocated on the stack which would also make it faster.

[Affie]

Right, so first cut is then to get everything on the stack and avoid allocations on the heap?

Matheusz is updating some manifolds to work faster with SArray, so I think it should be on the stack then. Parametric is not a big lift and I tried it as an experiment already.

I think we should consider upgrading IIF to be able to work with SArrays before we do #1010?

[dehann]

I think we should consider upgrading IIF to be able to work with SArrays before we do #1010?

That's a good idea, we should be able to do that relatively quickly.

Matheusz is updating some manifolds to work faster with SArray, so I think it should be on the stack then. Parametric is not a big lift and I tried it as an experiment already.

Ah that's great, sounds good