Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)
deep-learning pytorch lie-groups optimization-algorithms stochastic-gradient-descent preconditioner low-rank-approximation kronecker-factored-approximation second-order-optimization affine-group hessian-vector-product
-
Updated
Dec 14, 2024 - Python