Roadmap to v1

[avik-pal]

Features

• Move Training utilities out of Experimental. One major recent push has been to clean up the training and ensure it is used everywhere in tutorials. It is about time to guarantee semvar for it.
• Preliminary XLA support. Please have a look at Meta Issue for Reactant Compilation of Lux Models #733 for a detailed discussion
• DispatchDoctor Add type-stability checks via DispatchDoctor.jl #683. Adding this directly to LuxCore. While not strictly breaking, I want to move this to 0.6. Fixed in feat: easy mechanism to set preferences #798
• Automatic Eltype warning using preferences. See Performance Pitfalls and How to Catch them #727

Minor TODOs

• Reexport NNlib from Lux
• Remove xlogy and xlogx in favor of https://github.com/JuliaStats/LogExpFunctions.jl See refactor: use xlogx and xlogy from LogExpFunctions #796 unsafe for GPU broadcasting

Deprecations

• Flattening of Chains Remove Auto-Flattening of Chains #682
• Remove unnecessary overloads of Base.keys
• Deprecate the TrainState construction using rng
• Older versions of preferences (mostly done in Performance Pitfalls and How to Catch them #727)
• cpu/gpu
• disable_stacktrace_truncation!
• Experimental symbols present outside Experimental
• Strings in DebugLayer/layer_map
• Experimental.@compact and Experimental.StatefulLuxLayer
• Not specifying true/false in StatefulLuxLayer.

Underlying packages that need to be updated to 1.0

• LuxCore -- the naming changes is probably too disruptive without any major benefit. We will only do the following 2 name changes: feat!: 1.0 release LuxCore.jl#43
  • AbstractLuxLayer
  • AbstractLuxContainerLayer
• LuxDeviceUtils --> MLDataDevices
  • Change naming of the package once registered
  • Renaming drop Lux "branding" MLDataDevices.jl#52
  • Remove Deprecations fix!: remove deprecations for 1.0 release MLDataDevices.jl#53
• WeightInitializers
  • Test other RNGs WeightInitializers.jl#11
• LuxLib
  • Remove reexport of NNlib
  • Remove Deprecations fix!: remove deprecations for 1.0 release LuxLib.jl#82
• Refactor of LuxTestUtils

Documentation Updates

• Add a migration guide to 1.0
• Update wording on interface specs. Make it sound mandatory.
• Update WeightInitializers.jl documentation to highlight what RNGs are supported

[avik-pal]

Another consideration is whether we should do a 0.6 release or move directly to a 1.0 release.

I have read posts in the wild complaining about the rapid release, which led to the ..57 patches, which might give a (false) notion that we were fixing bugs, but most of them were features 😓

[avik-pal]

General Status Updates:

• Boltz documentation has been separated out to allow for easier update
• LuxCore is ready for an update
  • if I don't hear back on anything, I will move ahead with this
• LuxLib is mostly ready
  • let's land the layernorm default change first
• dropping XLA support as milestone since that is non-breaking