Skip to content

Fix PSGD, spring cleaning
Compare
Choose a tag to compare
@ClashLuke ClashLuke released this 01 Jan 15:50
v1.4.0
0519edb
This commit was signed with the committer’s verified signature.
ClashLuke Lucas Nestler
GPG key ID: 0969BFEDB85D91E2
Verified
Learn about vigilant mode.
  • Previously, only the first parameter of PSGD was trained; This is fixed now
  • All PSGDs were PurePSGD - now momentum_into_precond_update and exp_avg_input have their expected effect again
  • preliminary support for external changes of group['lr']
Assets 2
Loading