ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
-
Updated
Feb 27, 2023 - Python
#
second-order-optimization
Here are 24 public repositories matching this topic...
Pytorch implementation of preconditioned stochastic gradient descent (Kron and affine preconditioner, low-rank approximation preconditioner and more)
deep-learning pytorch lie-groups optimization-algorithms stochastic-gradient-descent preconditioner low-rank-approximation kronecker-factored-approximation second-order-optimization affine-group hessian-vector-product
-
Updated
Dec 14, 2024 - Python
Distributed K-FAC Preconditioner for PyTorch
-
Updated
Feb 25, 2025 - Python
An implementation of PSGD Kron second-order optimizer for PyTorch
-
Updated
Feb 21, 2025 - Python
FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)
-
Updated
Nov 29, 2020 - Python
This repository implements FEDL using pytorch
-
Updated
Nov 6, 2020 - Python
PyTorch implementation of the Hessian-free optimizer
deep-neural-networks deep-learning conjugate-gradient optimization-algorithms stochastic-optimizers newtons-method matrix-free second-order-optimization hessian-free pytorch-implementation ggn
-
Updated
Jun 14, 2024 - Python
Tensorflow implementation of preconditioned stochastic gradient descent
deep-learning tensorflow lie-groups optimization-algorithms stochastic-gradient-descent preconditioner low-rank-approximation second-order-optimization affine-group hessian-vector-product
-
Updated
Nov 23, 2023 - Python
Hessian-based stochastic optimization in TensorFlow and keras
tensorflow optimization keras optimization-algorithms stochastic-optimization newton-method second-order-optimization
-
Updated
Mar 29, 2022 - Python
Implementation of PSGD optimizer in JAX
-
Updated
Dec 31, 2024 - Python
This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.
-
Updated
Jan 12, 2025 - Python
Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose
-
Updated
Aug 1, 2023 - Python
FOSI library for improving first order optimizers with second order information
-
Updated
Mar 21, 2024 - Python
Newton’s second-order optimization methods in python
optimization-algorithms newtons-method quasi-newton newton-method second-order-optimization quasi-newton-method
-
Updated
Jun 8, 2022 - Python
The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:
computer-vision deep-learning top-k language-model low-rank-approximation second-order-optimization ggt adaptive-gradient-method full-matrix-preconditioning m-fac
-
Updated
Sep 12, 2023 - Python
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
deep-learning fisher optimizer optimization-methods natural-gradients kfac second-order-optimization hessian-free pytorch-implementation
-
Updated
Jul 28, 2023 - Python
Modular optimization library for PyTorch.
optimization pytorch gradient-descent optimization-algorithms newton-raphson second-order-optimization derivative-free-optimization gradient-free-optimization zeroth-order-optimization optimizers grafting gradient-approximations first-order-optimization
-
Updated
Feb 24, 2025 - Python
Stochastic Second-Order Methods in JAX
machine-learning gauss-newton-method quasi-newton stochastic-optimization jax second-order-optimization hessian-free natural-gradient
-
Updated
Sep 23, 2024 - Python
NG+: A new second-order optimizer for deep learning
-
Updated
Nov 13, 2022 - Python
sophia optimizer further projected towards flat areas of loss landscape
-
Updated
Dec 19, 2024 - Python
Improve this page
Add a description, image, and links to the second-order-optimization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the second-order-optimization topic, visit your repo's landing page and select "manage topics."