Releases: VainF/Torch-Pruning
Releases · VainF/Torch-Pruning
v1.4.2
What's Changed
- fixed a bug in attention head pruning
- fixed potentially buggy typo by @Alejandro-Casanova in #405
New Contributors
- @Alejandro-Casanova made their first contribution in #405
Full Changelog: v1.4.1...v1.4.2
v1.4.1
What's Changed
- Add Isomorphic Pruning, an improved algorithm for global pruning.
- Unify local/global/isomorphic pruning with
Scope
for importance ranking - Allow user-defined scope for importance ranking. The following key-value pair
(model.layer1, model.layer2): 0.4
will perform global ranking only within layer1 and layer2, with the pruning ratio of 40%.pruner = tp.pruner.MetaPruner( ... global_pruning=True, pruning_ratio=0.5, # default pruning ratio pruning_ratio_dict = {(model.layer1, model.layer2): 0.4, model.layer3: 0.2}, # Global pruning will be performed on layer1 and layer2 )
- Bugfixing
New Contributors
- @Miocio-nora made their first contribution in #380
Full Changelog: v1.4.0...v1.4.1
v1.4.0: Improved Support for Huggingface Transformers & LLMs
What's Changed
- Add support for Grouped Query Attention (GQA) in Huggingface transformers.
- Include minimal examples for Large Language Models (LLaMA-2 & LLaMA-3).
Full Changelog: v1.3.7...v1.4.0
v1.3.7
- Add more docstrings and comments
- Minor bug fixing
Full Changelog: v1.3.6...v1.3.7
v1.3.6
v1.3.5: bugfixing
v1.3.4
What's Changed
- fix nan and inf bug in sparse learning by @HollyLee2000 in #310
- Fixed a bug in interactive pruning + iterative pruning + sparse training by @VainF in #311
New Contributors
- @HollyLee2000 made their first contribution in #310
Full Changelog: v1.3.3...v1.3.4
v1.3.3: Bugfixing
Full Changelog: v1.3.2...v1.3.3
v1.3.2: A regular release with minor BugFixing.
- Fixed an issue in grouped conv.
- Include more normalization layers in importance estimation
- Add a MACs / FLOPs counter for timm Swin
- Add sentinel normalizer, which normalizes the importance score by utilizing the k-th smallest element.
- ...