Please install and setup AIMET before proceeding further. This evaluation was run using AIMET 1.26 for TensorFlow 2.4 i.e. please set release_tag="1.26"
and AIMET_VARIANT="tf_gpu"
in the above instructions.
pip install numpy==1.19.5
The TF2 pretrained resnet50 model is directly imported from package tensorflow.keras.applications
- ImageNet can be downloaded from here:
- The directory where the data is located should contains subdirectories, each containing images for a class
- The ImageNet validation dataset should be organized in the following way
< path to ImageNet validation dataset >
├── n01440764
├── n01443537
├── ...
- To run evaluation with QuantSim in AIMET, use the following:
python aimet_zoo_tensorflow/resnet50_tf2/evaluators/resnet50_tf2_quanteval.py \
--dataset-path <path to imagenet dataset> \
--batch-size <batch size for loading the dataset> \
--model-config <model configuration to test>
Available model configurations are:
-
resnet50_w8a8
-
Example : python aimet_zoo_tensorflow/resnet50_tf2/evaluators/resnet50_tf2_quanteval.py --dataset-path --batch-size 4 --model-config resnet50_w8a8
In the evaluation script included, we have used the default config file, which configures the quantizer ops with the following assumptions:
- Weight quantization: 8 bits, symmetric quantization
- Bias parameters are not quantized
- Activation quantization: 8 bits, asymmetric quantization
- Model inputs are quantized
Below are the top1 accuracy results of the TensorFlow 2.4 resnet50 model for the imagenet dataset:
Model Configuration | Top1 (%) |
---|---|
Resnet50_FP32 | 74.9 |
Resnet50 + simple PTQ(w8a8) | 74.8 |