Skip to content

Latest commit

 

History

History
159 lines (116 loc) · 8.08 KB

sparse-transfer-experiment-guide.md

File metadata and controls

159 lines (116 loc) · 8.08 KB

Sparsify Sparse-Transfer Experiment Guide

The Sparsify Sparse-Transfer Experiment Guide is a guide for running Sparse-Transfer Experiments with the Sparsify CLI. We also have One-Shot and Training-Aware Experiments, which you can explore in the Next Steps section of this guide.

Table of Contents

  1. Experiment Overview
  2. CLI Quickstart
  3. Examples
  4. Next Steps
  5. Resources

Experiment Overview

Sparsity Sparsification Speed Accuracy
++++ ++++ +++++

Sparse-Transfer Experiments are the second quickest way to create a faster and smaller model for your dataset. Sparse, foundational models that have been pre-sparsified on a large, upstream dataset such as ImageNet are transferred to your dataset through fine-tuning keeping the sparse architecture intact.

Generally, Sparse-Transfer Experiments result in a 5–10x speedup with minimal accuracy loss. They are ideal when a sparse model already exists for your use case, and you want to quickly utilize it for your dataset.

The CLI Quickstart below will walk you through the steps to run a Sparse-Transfer Experiment on your model. To utilize the cloud pathways for Sparse-Transfer Experiments, review the Cloud User Guide.

CLI Quickstart

Now that you understand what a Sparse-Transfer Experiment is and the benefits, including fine-tuning a pre-optimized, sparse model on your data, you're ready to use the CLI to effectively run a Sparse-Transfer Experiment.

Before you run a Sparse-Transfer Experiment, confirm you are logged into the Sparsify CLI. For instructions on Installation and Setup, review the Sparsify Install and Setup Section in the Sparsify README.

Sparse-Transfer Experiments use the following general command:

sparsify.run sparse-transfer --use-case USE_CASE --data DATA --optim-level OPTIM_LEVEL* --model MODEL*

* optional arguments

The description, rules, and possible values for each of the arguments are described below:

USE_CASE

The generally supported use cases for Sparsify are:

  • cv-classification
  • cv-detection
  • cv-segmentation
  • nlp-question_answering
  • nlp-text_classification
  • nlp-sentiment_analysis
  • nlp-token_classification
  • nlp-named_entity_recognition

Note that other aliases are recognized for these use cases, such as image-classification for cv-classification. Sparsify will automatically recognize these aliases and apply the correct use case.

Currently, custom use cases are not supported for Sparse-Transfer Experiments.

DATA

For all Sparsify Experiments, you will need to provide a dataset to create a sparse model. Due to the varied ML pipelines and implementations, Sparsify standardizes on a few popular formats for datasets. Confirm that your data is formatted properly according to the standards listed below.

Different use cases may require different input formats depending on what is considered standard for that use case. Specifically, the following are the supported formats as well as links to specs and guides for creating datasets for each format:

  • cv-classification: Image Folder Format
  • cv-detection - YOLO Format
  • cv-segmentation - YOLO Format
  • nlp-*: Hugging Face CSV or JSONW Format
    • NLP Dataset Guide
    • Example structure: data/{SPLIT}.csv or data/{SPLIT}.jsonl or data/{SPLIT}.json

Currently, custom use cases are not supported for dataset representation and datasets must conform to the definitions above. In the near future, these will be supported through plugin specifications.

For full details on Sparsify datasets, read the Sparsify Datasets Guide.

OPTIM_LEVEL

When using Sparsify, the optim (sparsification) level is one of the top arguments you should decide on. Specifically, it controls how much sparsification is applied to your model with higher values resulting in faster and more compressed models. At the max range, though, you may see a drop in accuracy.

Sparse-Transfer optim_level mappings are unique since they map to models available in the SparseZoo to transfer from. Increasing the optim level will result in smaller and more compressed models. The specific mappings are the following:

  • optim-level == 0.0: the largest model selected from the SparseZoo with no optimizations.
  • optim-level < 0.25: the largest model selected from the SparseZoo with INT8 quantization applied to the model (activations and weights).
  • optim-level < 0.5: the largest model selected from the SparseZoo with both unstructured pruning (sparsity) and INT8 quantization applied to the model.
  • optim-level < 0.75: the medium model selected from the SparseZoo with both unstructured pruning (sparsity) and INT8 quantization applied to the model.
  • optim-level <= 1.0: the smallest model selected from the SparseZoo with both unstructured pruning (sparsity) and INT8 quantization applied to the model.

The default of 0.5 will result in a medium-sized sparse model with INT8 quantization, and is a good default to start with.

MODEL

Models are optional for the Sparse-Transfer pathway. If no model is provided, the best pre-sparsified model and recipe from the SparseZoo for the given optimization level will be used.

If you choose to override the model, it is expected to be a pre-sparsified model and adhere to the following formats depending on the use case:

Currently, custom use cases are not supported for model representation and models must conform to the definitions above. In the near future, these will be supported through plugin specifications.

For full details on Sparsify models, read the Sparsify Models Guide.

Examples

Check back in soon for walkthroughs and examples of One-Shot Experiments applied to various popular models and use cases.

Next Steps

Now that you have successfully run a Sparse-Transfer Experiment, check out the following guides to continue your Sparsify journey:

Resources

To learn more about Sparsify and the available pathways other than Sparse-Transfer Experiments, refer to the Sparsify README.