[Feature] Implement DeMo optimizer

[kozistr]

Problem (Why?)

Close #299
Close #300

Solution (What/How?)

• implement DeMo optimizer

Other changes (bug fixes, small refactors)

N/A

Notes

N/A

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (e22d18c) to head (202fd2f).
Report is 15 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #301   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           98        98           
  Lines         7427      7428    +1     
=========================================
+ Hits          7427      7428    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.