Docker: PyTorch with MIVisionX

Docker - PyTorch + MIVisionX for Training

Steps to create docker for PyTorch training with MIVisionX RALI:

1. Pull latest rocm/pytorch docker

sudo docker pull rocm/pytorch:rocm3.7_ubuntu18.04_py3.6_pytorch

2. Install numactl

sudo apt-get install numactl

3. Install DLLOGGER

pip3.6 install git+https://github.com/NVIDIA/dllogger

4. Clone this github repo to fetch pre-requisites

git clone https://github.com/LakshmiKumar23/MLPerf-mGPU-dev/
cd single_stage_detector
pip3 install -r requirements.txt
python3 -m pip install pycocotools==2.0.0
pip3.6 install numpy==1.16

5. Follow the production ready installation step from this confluence page

pip3.6 install -v --install-option="--cpp_ext" --install-option="--cuda_ext" 'git+https://github.com/ROCmSoftwarePlatform/apex.git'

6. Clone MIVisionX. Run the setup script with '--neural_net off' option.

git clone https://github.com/GPUOpen-ProfessionalCompute-Libraries/MIVisionX
cd MIVisionX
python MIVisionX-setup.py --neural_net off

7. Build and install MIVisionX. Comment out lines for find_package - miopen and miopengemm - in MIVisionX/amd_openvx_extensions/CMakeLists.txt and MIVisionX/apps/CMakeLists.txt

cd MIVisionX
mkdir build
cd build
cmake -DNEURAL_NET=OFF ../
make -j
sudo make install

export PATH=$PATH:/opt/rocm/mivisionx/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/mivisionx/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/rpp/lib

8. Setup rali_pybind

sudo ./run.sh

9. Commit and save the new docker state

sudo docker commit <container ID> <new_image_name:tag_name>
sudo docker push <new_image_name:tag_name>