Pinned Loading

1. zo2 zo2 Public

   ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory

   Python 84 6

2. Tiny-DeepSpeed Tiny-DeepSpeed Public

   Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library

   Python 10 1

3. Flash-Attention-Implementation Flash-Attention-Implementation Public

   Implementation of Flash-Attention (both forward and backward) with PyTorch, CUDA, and Triton

   Python 1

4. Tiny-Megatron Tiny-Megatron Public

   Tiny-Megatron, a minimalistic re-implementation of the Megatron library

   Python 3

5. MetaProfiler MetaProfiler Public

   MetaProfiler is a lightweight, structure-agnostic operator-level profiler for PyTorch models that leverages MetaTensor execution to simulate and benchmark individual ops without loading the full mo…

   Python 1