Remove DiffEqCallbacks

[Sbozzolo]

DiffEqCallbacks seems like an innocous package, but it is not. DiffEqCallbacks depends several packages, including on NonLinearSolve, which is a chunky package.

julia> @time_imports import ClimaTimeSteppers
      6.6 ms  Preferences
      0.2 ms  PrecompileTools
      3.0 ms  UnsafeAtomics
      4.3 ms  Atomix
      5.6 ms  MacroTools
               ┌ 0.6 ms SuiteSparse_jll.__init__() 
     22.7 ms  SuiteSparse_jll 95.79% compilation time
               ┌ 4.8 ms SparseArrays.CHOLMOD.__init__() 95.58% compilation time
    162.9 ms  SparseArrays 2.82% compilation time
      0.6 ms  StaticArraysCore
    141.1 ms  StaticArrays
      0.3 ms  Adapt
      0.1 ms  Adapt → AdaptStaticArraysExt
      1.0 ms  CEnum
      0.1 ms  LazyArtifacts
      0.2 ms  JLLWrappers
               ┌ 3.8 ms LLVMExtra_jll.__init__() 71.79% compilation time
      4.2 ms  LLVMExtra_jll 64.65% compilation time
               ┌ 0.1 ms LLVM.__init__() 
     24.5 ms  LLVM
      1.4 ms  UnsafeAtomicsLLVM
      8.3 ms  KernelAbstractions
      0.2 ms  FastClosures
      0.2 ms  ExprTools
               ┌ 0.0 ms TimerOutputs.__init__() 
     11.9 ms  TimerOutputs
      2.9 ms  LinearOperators
      0.7 ms  ClimaComms
      0.4 ms  Statistics
      0.1 ms  StaticArrays → StaticArraysStatisticsExt
     13.7 ms  FixedPointNumbers
      0.2 ms  Reexport
               ┌ 14.3 ms ColorTypes.__init__() 99.94% compilation time (100% recompilation)
     31.6 ms  ColorTypes 45.26% compilation time (100% recompilation)
     28.0 ms  Colors
               ┌ 0.1 ms NVTX_jll.__init__() 
      0.4 ms  NVTX_jll
               ┌ 0.1 ms JuliaNVTXCallbacks_jll.__init__() 
      0.2 ms  JuliaNVTXCallbacks_jll
               ┌ 0.0 ms NVTX.__init__() 
     22.8 ms  NVTX
      0.2 ms  SuiteSparse
      0.9 ms  ArrayInterface
      0.3 ms  ArrayInterface → ArrayInterfaceStaticArraysCoreExt
               ┌ 0.0 ms DocStringExtensions.__init__() 
      0.7 ms  DocStringExtensions
      4.2 ms  FunctionWrappers
      0.2 ms  MuladdMacro
      0.1 ms  UnPack
      0.2 ms  Parameters
      0.1 ms  IfElse
      9.4 ms  Static
      0.3 ms  Compat
      0.1 ms  Compat → CompatLinearAlgebraExt
               ┌ 0.0 ms Requires.__init__() 
      0.2 ms  Requires
     21.1 ms  StaticArrayInterface
      0.9 ms  StaticArrayInterface → StaticArrayInterfaceStaticArraysExt
      0.6 ms  ManualMemory
               ┌ 0.0 ms ThreadingUtilities.__init__() 
      4.6 ms  ThreadingUtilities
      0.2 ms  SIMDTypes
      1.6 ms  LayoutPointers
      1.8 ms  CloseOpenIntervals
      7.0 ms  StrideArraysCore
      0.2 ms  BitTwiddlingConvenienceFunctions
      0.6 ms  CpuId
               ┌ 7.2 ms CPUSummary.__init__() 99.36% compilation time
     10.6 ms  CPUSummary 67.33% compilation time
               ┌ 0.0 ms PolyesterWeave.__init__() 
      4.4 ms  PolyesterWeave
      0.5 ms  Polyester
      0.5 ms  FastBroadcast
      7.0 ms  RecipesBase
      0.5 ms  RuntimeGeneratedFunctions
     18.7 ms  Test
      0.5 ms  InverseFunctions
      0.1 ms  InverseFunctions → DatesExt
      0.3 ms  ConstructionBase
      0.4 ms  ConstructionBase → ConstructionBaseStaticArraysExt
      0.1 ms  CompositionsBase
      0.1 ms  CompositionsBase → CompositionsBaseInverseFunctionsExt
               ┌ 5.1 ms Accessors.__init__() 99.83% compilation time (100% recompilation)
     14.3 ms  Accessors 35.61% compilation time (100% recompilation)
      0.3 ms  Accessors → AccessorsStaticArraysExt
      1.9 ms  SymbolicIndexingInterface
      0.1 ms  DataValueInterfaces
      0.4 ms  DataAPI
      0.1 ms  IteratorInterfaceExtensions
      0.1 ms  TableTraits
      6.7 ms  Tables
      1.4 ms  GPUArraysCore
      0.2 ms  ArrayInterface → ArrayInterfaceGPUArraysCoreExt
     15.0 ms  RecursiveArrayTools
      0.2 ms  RecursiveArrayTools → RecursiveArrayToolsFastBroadcastExt
               ┌ 0.0 ms TruncatedStacktraces.__init__() 
      0.2 ms  TruncatedStacktraces
     31.4 ms  Setfield
      3.3 ms  IrrationalConstants
      0.6 ms  DiffRules
      0.6 ms  DiffResults
               ┌ 0.0 ms OpenLibm_jll.__init__() 
      0.2 ms  OpenLibm_jll
      0.2 ms  NaNMath
      0.3 ms  LogExpFunctions
      0.1 ms  LogExpFunctions → LogExpFunctionsInverseFunctionsExt
               ┌ 4.3 ms CompilerSupportLibraries_jll.__init__() 93.70% compilation time
      4.5 ms  CompilerSupportLibraries_jll 90.24% compilation time
               ┌ 0.1 ms OpenSpecFun_jll.__init__() 
      0.4 ms  OpenSpecFun_jll
      4.9 ms  SpecialFunctions
      0.3 ms  CommonSubexpressions
     25.5 ms  ForwardDiff
      0.2 ms  RecursiveArrayTools → RecursiveArrayToolsForwardDiffExt
      0.4 ms  ForwardDiff → ForwardDiffStaticArraysExt
      0.2 ms  EnumX
      0.1 ms  ConcreteStructs
      0.4 ms  PreallocationTools
      0.2 ms  FunctionWrappersWrappers
      0.2 ms  SciMLStructures
      0.1 ms  CommonSolve
      1.2 ms  ADTypes
      7.4 ms  SciMLOperators
               ┌ 9.0 ms SciMLBase.__init__() 99.89% compilation time (100% recompilation)
     50.6 ms  SciMLBase 17.73% compilation time (100% recompilation)
      0.3 ms  Tricks
     15.4 ms  DiffEqBase
     19.1 ms  DataStructures
      1.0 ms  FiniteDiff
      0.2 ms  FiniteDiff → FiniteDiffStaticArraysExt
     38.6 ms  FillArrays
      1.7 ms  FillArrays → FillArraysSparseArraysExt
      0.2 ms  FillArrays → FillArraysStatisticsExt
    155.1 ms  ArrayLayouts
      0.6 ms  ArrayLayouts → ArrayLayoutsSparseArraysExt
     67.8 ms  LazyArrays
      0.2 ms  LazyArrays → LazyArraysStaticArraysExt
      1.9 ms  NLSolversBase
      1.4 ms  LineSearches
               ┌ 0.0 ms HostCPUFeatures.__init__() 
      3.6 ms  HostCPUFeatures
     66.6 ms  VectorizationBase
      1.9 ms  SLEEFPirates
     14.9 ms  OffsetArrays
      0.7 ms  StaticArrayInterface → StaticArrayInterfaceOffsetArraysExt
      0.1 ms  OffsetArrays → OffsetArraysAdaptExt
     74.6 ms  LoopVectorization
      0.2 ms  LoopVectorization → SpecialFunctionsExt
      0.4 ms  TriangularSolve
    106.8 ms  RecursiveFactorization
      9.0 ms  KLU
      1.0 ms  Sparspak
      2.9 ms  FastLapackInterface
     11.8 ms  ChainRulesCore
      0.3 ms  StaticArrays → StaticArraysChainRulesCoreExt
      0.2 ms  ChainRulesCore → ChainRulesCoreSparseArraysExt
      0.1 ms  LinearOperators → LinearOperatorsChainRulesCoreExt
      0.3 ms  LogExpFunctions → LogExpFunctionsChainRulesCoreExt
      0.5 ms  SpecialFunctions → SpecialFunctionsChainRulesCoreExt
      0.1 ms  ADTypes → ADTypesChainRulesCoreExt
      0.3 ms  SciMLBase → SciMLBaseChainRulesCoreExt
      0.2 ms  DiffEqBase → DiffEqBaseChainRulesCoreExt
      2.2 ms  LoopVectorization → ForwardDiffExt
      5.2 ms  Krylov
               ┌ 0.2 ms IntelOpenMP_jll.__init__() 
      0.5 ms  IntelOpenMP_jll
               ┌ 0.3 ms oneTBB_jll.__init__() 
      0.5 ms  oneTBB_jll
               ┌ 1.2 ms MKL_jll.__init__() 
      1.3 ms  MKL_jll
               ┌ 0.0 ms LinearSolve.__init__() 
    209.9 ms  LinearSolve
      0.3 ms  LinearSolve → LinearSolveRecursiveArrayToolsExt
      0.2 ms  LinearSolve → LinearSolveKernelAbstractionsExt
      0.6 ms  MaybeInplace
      0.4 ms  SimpleTraits
      1.1 ms  ArnoldiMethod
      0.5 ms  Inflate
      2.1 ms  SharedArrays
     10.8 ms  Graphs
      0.4 ms  VertexSafeGraphs
      0.1 ms  PackageExtensionCompat
               ┌ 0.0 ms SparseDiffTools.__init__() 
      1.8 ms  SparseDiffTools
      0.2 ms  SparseDiffTools → SparseDiffToolsPolyesterExt
      0.7 ms  SparseMatrixColorings
               ┌ 0.0 ms DifferentiationInterface.__init__() 
      2.0 ms  DifferentiationInterface
      0.9 ms  DifferentiationInterface → DifferentiationInterfaceForwardDiffExt
      0.8 ms  DifferentiationInterface → DifferentiationInterfaceFiniteDiffExt
      0.2 ms  DifferentiationInterface → DifferentiationInterfaceChainRulesCoreExt
    225.3 ms  SimpleNonlinearSolve
      0.5 ms  SimpleNonlinearSolve → SimpleNonlinearSolveChainRulesCoreExt
    350.7 ms  NonlinearSolve
      2.1 ms  Functors
      1.7 ms  DiffEqCallbacks
      5.7 ms  ClimaTimeSteppers

As we can see, NonlinearSolve and SimpleNonLinearSolve are alone more than 20 % of the load time of the package.

DiffEqCallbacks is used only in one place:

ClimaTimeSteppers.jl/src/integrators.jl

Lines 99 to 100 in d10a7d0

	saving_callback =
	NonInterpolatingSavingCallback(save_func, DiffEqCallbacks.SavedValues(sol.t, sol.u), save_everystep)

to enable the saving callback. I think we can very easily roll our own version and remove this dependency