This repository contains four test suites for the RPP library: image
/voxel
/audio
/miscellaneous
augmentations processing. These test suites can be used to validate the functionality and performance of the RPP image
/voxel
/audio
/miscellaneous
augmentation functionalities.
-
OpenCV Version
3.X
/4.X
sudo apt install libopencv-dev
-
- Source:
https://github.com/libjpeg-turbo/libjpeg-turbo.git
- Tag: 3.0.2
sudo apt-get install nasm sudo apt-get install wget git clone -b 3.0.2 https://github.com/libjpeg-turbo/libjpeg-turbo.git cd libjpeg-turbo mkdir build cd build cmake -DCMAKE_INSTALL_PREFIX=/usr \ -DCMAKE_BUILD_TYPE=RELEASE \ -DENABLE_STATIC=FALSE \ -DCMAKE_INSTALL_DEFAULT_LIBDIR=lib \ -DWITH_JPEG8=TRUE \ .. make -j$nproc sudo make install
- Source:
-
Libsndfile
sudo apt-get install libsndfile1-dev
-
Imagemagick
sudo apt-get install imagemagick
-
Nifti-Imaging - nifti_clib
git clone https://github.com/NIFTI-Imaging/nifti_clib.git cd nifti_clib mkdir build cd build cmake .. sudo make -j$nproc install
-
Python3 and Python3 PIP
sudo apt install python3-dev python3-pip
-
Python: Openpyxl
pip install openpyxl
-
Python: Pandas
pip install pandas
Note
- All package installs are shown with the
apt
package manager. Use the appropriate package manager for your operating system. - Run
sudo ldconfig
to configure dynamic run-time bindings
The image test suite can be executed under 2 backend scenarios - (HOST/HIP):
- HOST backend - (On a CPU with HOST backend)
- HIP backend - (On a GPU with HIP backend)
The image test suite accepts the following command line arguments:
- input_path1: The path to the input folder 1. Default is $cwd/../TEST_IMAGES/three_images_mixed_src1
- input_path2: The path to the input folder 2. Default is $cwd/../TEST_IMAGES/three_images_mixed_src2
- case_start: The starting case number for the test range (0-91). Default is 0
- case_end: The ending case number for the test range (0-91). Default is 91
- test_type: The type of test to run (0 = Unit tests, 1 = Performance tests). Default is 0
- case_list: A list of specific case numbers to run. Must be used in conjunction with --test_type
- profiling: Run the tests with a profiler (YES/NO). Default is NO. This option is only available with HIP backend
- qa_mode: Output images from tests will be compared with golden outputs - (0 / 1). Default is 0
- decoder_type: Type of Decoder to decode the input data - (0 = TurboJPEG / 1 = OpenCV). Default is 0
- num_runs: Specifies the number of runs for running the performance tests
- preserve_output: preserves the output images or performance logs generated from the previous test suite run - (0 = remove output images or performance logs / 1 = preserve output images or performance logs). Default is 1
- batch_size: Specifies the batch size to use for running tests. Default is 1
The test suite can be run with the following command:
python runTests.py --input_path1 <input_path1> --input_path2 <input_path2> --case_start <case_start> --case_end <case_end> --test_type <test_type>
The test suite can be run with the following command:
python runTests.py --input_path1 <input_path1> --input_path2 <input_path2> --case_start <case_start> --case_end <case_end> --test_type <test_type> --profiling <profiling>
- QA mode (Unit tests) - Tolerance based PASS/FAIL tests for RPP HIP/HOST functionalities checking pixelwise match between C/SSE/AVX/HIP versions after comparison to preset golden outputs. Please note that QA mode is only supported with a batch size of 3. Note: QA mode is not supported for case 84 due to run-to-run variation of outputs.
python runTests.py --case_start 0 --case_end 91 --test_type 0 --qa_mode 1 --batch_size 3
- QA mode (Performance tests) - Tolerance based PASS/FAIL tests for RPP HIP/HOST functionalities checking achieved improvement in performance percentage over BatchPD versions after comparison to a threshold percentage of improvement
python runTests.py --case_list 21 36 63 --test_type 1 --qa_mode 1 --batch_size 8 --num_runs 100
- Unit test mode - Unit tests allowing users to pass a path to a folder containing images, to execute the desired functionality and variant once, report RPP execution wall time, save and view output images Note: For testcase 82(RICAP) Please use images of same resolution and Batchsize > 1 RICAP dataset path: rpp/utilities/test_suite/TEST_IMAGES/three_images_150x150_src1
python runTests.py --case_start 0 --case_end 91 --test_type 0 --qa_mode 0
- Performance test mode - Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time, or optionally, AMD rocprof kernel profiler max/min/avg time for HIP backend variants. Note: For testcase 82(RICAP) Please use images of same resolution and Batchsize > 1 RICAP dataset path: rpp/utilities/test_suite/TEST_IMAGES/three_images_150x150_src1
python runTests.py --case_start 0 --case_end 91 --test_type 1
To run the unit tests / performance tests for specific case numbers. please case use case_list parameter. Example as below
- To run unittests for case numbers 0, 2, 4
python runTests.py --case_list 0 2 4 --test_type 0
- To run performance tests for case numbers 0, 2, 4
python runTests.py --case_list 0 2 4 --test_type 1
To run performance tests with AMD rocprof kernel profiler for HIP backend variants. This will generate profiler times for HIP backend variants
python runTests.py --test_type 1 --profiling YES
The image test suite includes:
- Unit tests that execute the desired functionality and variant once, report RPP execution wall time and save output images
- Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time, or optionally, AMD rocprof kernel profiler max/min/avg time for HIP backend variants.
- Unit and Performance tests are included for three layouts - PLN1 (1 channel planar NCHW), PLN3 (3 channel planar NCHW) and PKD3 (3 channel packed/interrleaved NHWC).
- Unit and Performance tests are included for various input/output bitdepths including U8/F32/F16/I8.
- Support for pixelwise output referencing against golden outputs, and functionality validation checking, by tolerance-based pass/fail criterions for each variant. (Current support only for U8 variants)
- Support for TurboJPEG and OpenCV decoder for decoding input images
The 3D Voxel test suite can be executed under 2 backend scenarios - (HOST/HIP):
- HOST backend - (On a CPU with HOST backend)
- HIP backend - (On a GPU with HIP backend)
The voxel test suite accepts the following command line arguments:
- header_path: Path to the nii header
- data_path: Path to the nii data file
- case_start: The starting case number for the test range (0-38). Default is 0
- case_end: The ending case number for the test range (0-38). Default is 4
- test_type: The type of test to run (0 = Unit tests, 1 = Performance tests). Default is 0
- case_list: A list of specific case numbers to run. Must be used in conjunction with --test_type
- profiling: Run the tests with a profiler (YES/NO). Default is NO. This option is only available with HIP backend
- qa_mode: Output images from tests will be compared with golden outputs - (0 / 1). Default is 0
- num_runs: Specifies the number of runs for running the performance tests
- preserve_output: preserves the output images or performance logs generated from the previous test suite run - (0 = remove output images or performance logs / 1 = preserve output images or performance logs). Default is 1
- batch_size: Specifies the batch size to use for running tests. Default is 1
The test suite can be run with the following command:
python runVoxelTests.py --header_path <header_path> --data_path <data_path> --case_start <case_start> --case_end <case_end> --test_type <test_type>
The test suite can be run with the following command:
python runVoxelTests.py --header_path <header_path> --data_path <data_path> --case_start <case_start> --case_end <case_end> --test_type <test_type> --profiling <profiling>
- QA mode - Tolerance based PASS/FAIL tests for RPP HIP/HOST functionalities checking pixelwise match between C/SSE/AVX/HIP versions after comparison to preset golden outputs.
python runVoxelTests.py --case_start 0 --case_end 4 --test_type 0 --qa_mode 1 --batch_size 3
- Unit test mode - Unit tests allowing users to pass a path to a folder containing nii fikes, to execute the desired functionality and variant once, report RPP execution wall time, save and view output images, gifs and nii files
python runVoxelTests.py --case_start 0 --case_end 4 --test_type 0 --qa_mode 0
- Performance test mode - Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time, or optionally, AMD rocprof kernel profiler max/min/avg time for HIP backend variants.
python runVoxelTests.py --case_start 0 --case_end 4 --test_type 1
To run the unit tests / performance tests for specific case numbers. please case use case_list parameter. Example as below
- To run unittests for case numbers 0, 2, 4
python runVoxelTests.py --case_list 0 2 4 --test_type 0
- To run performance tests for case numbers 0, 2, 4
python runVoxelTests.py --case_list 0 2 4 --test_type 1
To run performance tests with AMD rocprof kernel profiler for HIP backend variants. This will generate profiler times for HIP backend variants
python runVoxelTests.py --test_type 1 --profiling YES
The image test suite includes:
- Unit tests that execute the desired functionality and variant once, report RPP execution wall time and save output images, gifs and nii files
- Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time, or optionally, AMD rocprof kernel profiler max/min/avg time for HIP backend variants.
- Unit and Performance tests are included for three layouts - PLN1 (1 channel planar NCHW), PLN3 (3 channel planar NCHW) and PKD3 (3 channel packed/interrleaved NHWC).
- Unit and Performance tests are included for one input/output bitdepth F32.
- Support for pixelwise output referencing against golden outputs, and functionality validation checking, by tolerance-based pass/fail criterions for each variant.
RPP test suite uses a sample ".nii" image usage from the BraTS2020 Dataset at https://www.kaggle.com/code/rastislav/3d-mri-brain-tumor-segmentation-u-net/input as per the following Data Usage Agreement present at the dataset link above.
Data Usage Agreement / Citations: You are free to use and/or refer to the BraTS datasets in your own research, provided that you always cite the following three manuscripts:
- B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al. "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)", IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694
- S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J.S. Kirby, et al., "Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features", Nature Scientific Data, 4:170117 (2017) DOI: 10.1038/sdata.2017.117
- S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, et al., "Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge", arXiv preprint arXiv:1811.02629 (2018)
The audio test suite can be executed to validate the functionality and performance of the AMD ROCm Performance Primitives (RPP) audio library.
- HOST backend - (On a CPU with HOST backend)
- F32 Bit Depth
The audio test suite accepts the following command line arguments:
- input_path: The path to the input folder. Default is $cwd/../TEST_AUDIO_FILES/eight_samples_single_channel_src1
- case_start: The starting case number for the test range (0-0). Default is 0
- case_end: The ending case number for the test range (0-0). Default is 0
- test_type: The type of test to run (0 = QA tests, 1 = Performance tests). Default is 0
- qa_mode: Output audio data from tests will be compared with golden outputs - (0 / 1). Default is 0
- case_list: A list of specific case numbers to run. Must be used in conjunction with --test_type
- num_runs: Specifies the number of runs for running the performance tests
- preserve_output: preserves the output or performance logs generated from the previous test suite run - (0 = remove output or performance logs / 1 = preserve output or performance logs). Default is 1
- batch_size: Specifies the batch size to use for running tests. Default is 1
The test suite can be run with the following command:
python runAudioTests.py --input_path <input_path> --case_start <case_start> --case_end <case_end> --test_type <test_type>
- QA mode - Tolerance based PASS/FAIL tests for RPP AUDIO HOST functionalities checking match between output and preset golden outputs. Please note that QA mode is only supported with a batch size of 3.
python runAudioTests.py --case_start 0 --case_end 0 --qa_mode 1 --batch_size 3
- Performance test mode - Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time.
python runAudioTests.py --case_start 0 --case_end 0 --test_type 1
To run the QA tests / performance tests for specific case numbers. please case use case_list parameter. Example as below
- To run QA tests for case numbers 0, 1, 2
python runAudioTests.py --case_list 0 1 2 --qa_mode 1 --batch_size 3
- To run performance tests for case numbers 0, 1, 2
python runAudioTests.py --case_list 0 1 2 --test_type 1
The audio test suite includes:
- Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time.
- QA and Performance tests are included for one input/output bitdepth F32.
- Support for output referencing against golden outputs, and functionality validation checking, by tolerance-based pass/fail criterions for each variant.
The miscellaneous test suite can be executed to validate the functionality and performance of the AMD ROCm Performance Primitives (RPP) generic libraries that can process N-Dimensional inputs
- HOST backend - (On a CPU with HOST backend)
- HIP backend - (On a GPU with HIP backend)
- F32 Bit Depth
The miscellaneous test suite accepts the following command line arguments:
- case_start: The starting case number for the test range (1-1). Default is 1
- case_end: The ending case number for the test range (1-1). Default is 1
- test_type: The type of test to run (0 = QA tests, 1 = Performance tests). Default is 0
- qa_mode: Output audio data from tests will be compared with golden outputs - (0 / 1). Default is 0
- case_list: A list of specific case numbers to run. Must be used in conjunction with --test_type
- num_dims: The number of dimensions of input. Default is 2
- num_runs: Specifies the number of runs for running the performance tests
- preserve_output: preserves the output or performance logs generated from the previous test suite run - (0 = remove output or performance logs / 1 = preserve output or performance logs). Default is 1
- batch_size: Specifies the batch size to use for running tests. Default is 1
- profiling: Run the tests with a profiler (YES/NO). Default is NO. This option is only available with HIP backend
The test suite can be run with the following command:
python runMiscTests.py --case_start <case_start> --case_end <case_end> --test_type <test_type>
- QA mode - Tolerance based PASS/FAIL tests for RPP MISC HOST functionalities checking match between output and preset golden outputs. Please note that QA mode is only supported with a batch size of 3 and num dims 2 or 3
python runMiscTests.py --case_start 1 --case_end 1 --qa_mode 1 --batch_size 3
- Performance test mode - Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time.
python runMiscTests.py --case_start 1 --case_end 1 --test_type 1
The miscellaneous test suite includes:
- Performance tests that execute the desired functionality and variant 100 times by default, and report max/min/avg RPP execution wall time.
- QA and Performance tests are included for one input/output bitdepth F32.
- Support for output referencing against golden outputs, and functionality validation checking, by tolerance-based pass/fail criterions for each variant.