Backport #2719 to MTK v8

[devmotion]

The PR backports #2719 to a MTK v8 on a new backport_8 branch. I can't use MTK v9 yet due to the incompatibility with Catalyst, so #2719 is not sufficient for my project.

[ChrisRackauckas]

Format so tests run

[devmotion]

I reformatted the code (which blows up the diff). It seems only a subset of CI is run on PRs that do not target the master branch - one problem with the existing version bounds for MTK v8 are the breaking changes in SymbolicUtils 1.6.0 (see e.g. https://github.com/SciML/ModelingToolkit.jl/actions/runs/9150353436/job/25155026031#step:6:1243) and the ADTypes changes (and some incorrect version bounds in NonlinearSolve it seems: https://github.com/SciML/ModelingToolkit.jl/actions/runs/9150353436/job/25155025675?pr=2726#step:6:1903). In my package that's not too problematic since I pin these packages anyway but generally that's annoying for users.

[ChrisRackauckas]

@thazhemadam do you know how to make this run?

[ChrisRackauckas]

Can we just bound SymbolicUtils?

[ChrisRackauckas]

Limit SymbolicIndexingInterface as well?

[devmotion]

Why? Most (all?) problems seem to be caused by ADTypes e.g. https://github.com/SciML/ModelingToolkit.jl/actions/runs/9159389647/job/25179664915?pr=2726#step:6:1826

[gdalle]

The problem here is indeed a package that claimed ADTypes v1 support without ensuring it, namely OptimizationBase. This bug was introduced in OptimizationBase v0.0.6 and only fixed in OptimizationBase v1.0.1, so if we want to amend the general registry we should remove ADTypes v1 compat in OptimizationBase v0.0.6, v0.0.7 and v1.0.0

Cross-ref:

• CompatHelper: bump compat for ADTypes to 1, (keep existing compat) OptimizationBase.jl#37 (introduced bug)
• Current state not compatible with ADTypes v1 despite compat bounds OptimizationBase.jl#44 (noticed bug)
• ADTypes v1.0 support updates OptimizationBase.jl#46 (fixed bug)

Ping @Vaibhavdixit02

[Vaibhavdixit02]

Yeah we can do that, I am not sure how it can be changed in the registry though. Can it just be bounded to v1.0.1 here?

[gdalle]

One option is to yank those versions, aka get rid of them completely. That's what I did in JuliaRegistries/General#107391

https://github.com/JuliaRegistries/General?tab=readme-ov-file#when-is-yanking-a-release-appropriate

But I'm not sure it is necessary if you tag a v0.0.8 patch release removing the faulty compat entry?

[gdalle]

Let's discuss on the General PR JuliaRegistries/General#107391

[devmotion]

@gdalle Tests still fail due to ADTypes it seems.

[gdalle]

It's installing OptimizationBase v0.0.7, which I thought I had yanked?

[ChrisRackauckas]

The package servers can take awhile to get the updated registries.

[ChrisRackauckas]

r