Skip to content

Commit

Permalink
PyBind - Packaging and CTests (#221)
Browse files Browse the repository at this point in the history
* rocAL PyBind - Updates

* Cleanup

* Py Test - Import test

* PyTests

* Tests - Updates

* PyTests - Updates

* PyTest - Pipelines

* PyTest - Fix

* rocAL Pybind - Tests

* CMakeLists - Updates

* Tests - CTests for pybind

* rocAL PyBind - Test Install

* rocal pybind - fix cmakelists

* Test - Updates

* Test - Updates

* Test - Updates

* PyBind Readme - updates

* CHANGELOG.md - Updates
  • Loading branch information
kiritigowda authored Oct 12, 2024
1 parent a0fb3b9 commit cc619f5
Show file tree
Hide file tree
Showing 13 changed files with 320 additions and 22 deletions.
14 changes: 8 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,26 @@

[rocAL Documentation](https://github.com/ROCm/rocAL)

## rocAL 2.1.0 (unreleased)
## rocAL 2.1.0 for ROCm 6.3.0

### Changes
* Setup: rocdecode install disabled
* Package: rocdecode dependency removed
* rocAL Pybind support for package install - rocAL Python module: To use python module, you can set PYTHONPATH:
+ `export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH`

### Removals
* TBA
* rocdecode dependencies for package install removed

### Optimizations
* TBA
* CTest updates

### Resolved issues
* TBA
* Test failures fixed

### Known issues
* Package install requires `OpenCV` manual install
* CentOS/RedHat/SLES requires `FFMPEG Dev` package manual install
* Package install requires `TurboJPEG`, and `RapidJSON` manual install
* `CentOS`/`RedHat`/`SLES` requires additional `FFMPEG Dev` package manual install
* Hardware decode requires rocm usecase `graphics`

### Upcoming changes
Expand Down
40 changes: 32 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ rocAL can be currently used to perform the following operations either with rand
sudo apt install rpp-dev
```

* MIVisionX Components: [AMD OpenVX™](https://github.com/ROCm/MIVisionX/tree/master/amd_openvx) and AMD OpenVX™ Extensions: `VX_RPP` and `AMD Media`
* [MIVisionX](https://github.com/ROCm/MIVisionX) Components: [AMD OpenVX™](https://github.com/ROCm/MIVisionX/tree/master/amd_openvx) and AMD OpenVX™ Extensions: `VX_RPP` and `AMD Media`
```shell
sudo apt install mivisionx-dev
```
Expand Down Expand Up @@ -165,7 +165,8 @@ The installation process uses the following steps:
* Install ROCm `6.1.0` or later with [amdgpu-install](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/amdgpu-install.html) with `--usecase=rocm`
* Use **either** [package install](#package-install) **or** [source install](#source-install) as described below.
>[!IMPORTANT]
> Use **either** [package install](#package-install) **or** [source install](#source-install) as described below.
### Package install
Expand Down Expand Up @@ -193,9 +194,12 @@ Install rocAL runtime, development, and test packages.
sudo zypper install rocal rocal-devel rocal-test
```
>[!NOTE]
> * Package install requires `TurboJPEG` and `RapidJSON` manual install
>[!IMPORTANT]
> * Package install requires `TurboJPEG`, and `RapidJSON` manual install
> * `CentOS`/`RedHat`/`SLES` requires additional `FFMPEG Dev` package manual install
> * rocAL Python module: To use python module, you can set PYTHONPATH:
> + `export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH`
### Source install
Expand All @@ -207,7 +211,8 @@ To build rocAL from source and install, follow the steps below:
git clone https://github.com/ROCm/rocAL.git
```
**Note:** rocAL has support for two GPU backends: **OPENCL** and **HIP**:
>[!NOTE]
> rocAL has support for two GPU backends: **OPENCL** and **HIP**:
#### HIP Backend
Expand All @@ -229,7 +234,7 @@ git clone https://github.com/ROCm/rocAL.git
```
>[!NOTE]
> * `PyPackageInstall` used for rocal_pybind installation
> * `sudo` required for pybind installation
>[!IMPORTANT]
> * Use `-D PYTHON_VERSION_SUGGESTED=3.x` with `cmake` for using a specific Python3 version if required.
Expand All @@ -256,13 +261,14 @@ git clone https://github.com/ROCm/rocAL.git
* The installer will copy
* Executables into `/opt/rocm/bin`
* Libraries into `/opt/rocm/lib`
* rocal_pybind into `/opt/rocm/lib`
* Header files into `/opt/rocm/include/rocal`
* Apps, & Samples folder into `/opt/rocm/share/rocal`
* Documents folder into `/opt/rocm/share/doc/rocal`
### Verify with rocal-test package
Test package will install ctest module to test rocAL. Follow below steps to test packge install
Test package will install ctest module to test rocAL. Follow below steps to test package install
```shell
mkdir rocAL-test && cd rocAL-test
Expand All @@ -272,11 +278,29 @@ ctest -VV
>[!NOTE]
> * Make sure all rocAL required libraries are in your PATH
> * `RHEL`/`SLES` - Export FFMPEG libraries into your PATH
> + `export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib64/:/usr/local/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH`
> + `export LD_LIBRARY_PATH=/usr/local/lib:/usr/local/lib64/:/usr/local/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH`
> ```shell
> export PATH=$PATH:/opt/rocm/bin
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib
> ```
### Verify rocAL PyBind with rocal-test package
Test package will install ctest module to test rocAL PyBindings. Follow below steps to test package install
```shell
mkdir rocal-pybind-test && cd rocal-pybind-test
cmake /opt/rocm/share/rocal/test/pybind
ctest -VV
```
>[!NOTE]
> * Make sure all rocAL required libraries are in your PATH
> ```shell
> export PATH=$PATH:/opt/rocm/bin
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/lib
> export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH
> ```
## Documentation
Run the steps below to build documentation locally.
Expand Down
1 change: 0 additions & 1 deletion rocAL/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
# SOFTWARE.

cmake_minimum_required(VERSION 3.5)
set(VERSION "2.0.0")

# Set Project Version and Language
project(rocal VERSION ${VERSION} LANGUAGES CXX)
Expand Down
11 changes: 7 additions & 4 deletions rocAL_pybind/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,11 @@
# THE SOFTWARE.

cmake_minimum_required(VERSION 3.5)
set(VERSION "2.0.0")

# Set Project Version and Language
project(rocAL_pybind VERSION ${VERSION} LANGUAGES CXX)
project(rocal_pybind VERSION ${VERSION} LANGUAGES CXX)

set(TARGET_NAME rocAL_pybind)
set(TARGET_NAME rocal_pybind)

set(CMAKE_INSTALL_LIBDIR "lib" CACHE STRING "Library install directory")
include(GNUInstallDirs)
Expand Down Expand Up @@ -280,8 +279,12 @@ if(${BUILD_ROCAL_PYBIND})
add_dependencies(PyPackageInstall PyPackageBuild)

#install rocAL pybind libs
install(FILES ${CMAKE_BINARY_DIR}/${TARGET_NAME}/${CMAKE_INSTALL_LIBDIR}/${TARGET_PYMODULE_NAME} DESTINATION ${CMAKE_INSTALL_PREFIX_PYTHON} COMPONENT runtime)
install(FILES ${CMAKE_BINARY_DIR}/${TARGET_NAME}/${CMAKE_INSTALL_LIBDIR}/${TARGET_PYMODULE_NAME} DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT runtime)
install(DIRECTORY amd DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT runtime)
message("-- ${Green}Radeon Augmentation Library Python Binding - rocal_pybind module added ${ColourReset}")

# PyBind Test - Installed
install(DIRECTORY ${CMAKE_SOURCE_DIR}/tests/pybind DESTINATION ${CMAKE_INSTALL_DATADIR}/rocal/test COMPONENT test)
else()
message("-- ${Red}WARNING: rocAL PyBind Module excluded - Dependency Failure${ColourReset}")
endif()
6 changes: 4 additions & 2 deletions rocAL_pybind/getrocALwheelname.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@ def wheel_name(**kwargs):
tag = '-'.join(bdist_wheel_cmd.get_tag())
return f'{distname}-{tag}.whl'


print(wheel_name(name='amd-rocal', version='2.0.0'), end='')
# Version updates - IMP: Change in version requires to match top level CMakeLists.txt
## * getrocALWheelname.py
## * setup.py
print(wheel_name(name='amd-rocal', version='2.1.0'), end='')
5 changes: 4 additions & 1 deletion rocAL_pybind/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,14 @@ class BinaryDistribution(Distribution):
def has_ext_modules(self):
return True

# Version updates - IMP: Change in version requires to match top level CMakeLists.txt
## * getrocALWheelname.py
## * setup.py
setup(
name='amd-rocal',
description='AMD ROCm Augmentation Library Python Bindings',
url='https://github.com/ROCm/rocAL',
version='2.0.0',
version='2.1.0',
author='AMD',
license='MIT',
packages=find_packages(where='@TARGET_NAME@'),
Expand Down
97 changes: 97 additions & 0 deletions tests/pybind/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
################################################################################
#
# MIT License
#
# Copyright (c) 2023 Advanced Micro Devices, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
################################################################################
cmake_minimum_required (VERSION 3.5)

project(rocal-pybind-test)

# make test with CTest
enable_testing()
include(CTest)

# ROCM Path
if(DEFINED ENV{ROCM_PATH})
set(ROCM_PATH $ENV{ROCM_PATH} CACHE PATH "Default ROCm installation path")
elseif(ROCM_PATH)
message("-- ${PROJECT_NAME} INFO:ROCM_PATH Set -- ${ROCM_PATH}")
else()
set(ROCM_PATH /opt/rocm CACHE PATH "Default ROCm installation path")
endif()

set(Python3_FIND_VIRTUALENV FIRST)
find_package(Python3 QUIET)

set(ROCAL_PYBIND_SCRIPTS OFF)
if(EXISTS "${ROCM_PATH}/lib/amd/rocal")
set(ROCAL_PYBIND_SCRIPTS ON)
endif()

# OpenVX Tests
if(Python3_FOUND AND ROCAL_PYBIND_SCRIPTS)
message("-- ${PROJECT_NAME}: rocAL Pybind found at ${ROCM_PATH}/lib/amd/rocal")
# 1 - decoders test
add_test(NAME rocal_pybind_test_decoders
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/decoders_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_decoders PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
# 2 - functions test
add_test(NAME rocal_pybind_test_functions
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/functions_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_functions PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
# 3 - pipelines test
add_test(NAME rocal_pybind_test_pipelines
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/pipelines_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_pipelines PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
# 4 - randoms test
add_test(NAME rocal_pybind_test_randoms
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/randoms_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_randoms PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
# 5 - readers test
add_test(NAME rocal_pybind_test_readers
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/readers_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_readers PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
# 6 - types test
add_test(NAME rocal_pybind_test_types
COMMAND ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/types_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
)
set_property(TEST rocal_pybind_test_types PROPERTY ENVIRONMENT "PYTHONPATH=${ROCM_PATH}/lib:$PYTHONPATH")
else()
if(NOT Python3_FOUND)
message("-- NOTE: rocAL pybind requires Python3 - NOT FOUND")
endif()
if(NOT ROCAL_PYBIND_SCRIPTS)
message("-- NOTE: rocAL pybind requires rocal_pybind project built and installed - NOT FOUND")
endif()
endif(Python3_FOUND AND ROCAL_PYBIND_SCRIPTS)
28 changes: 28 additions & 0 deletions tests/pybind/decoders_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Copyright (c) 2018 - 2023 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.

import amd.rocal.decoders as decoders

from inspect import getmembers, isfunction

print('rocAL PyBind Decoders')
rocalDecoders = getmembers(decoders, isfunction)
for i in range(len(rocalDecoders)):
print(rocalDecoders[i])
28 changes: 28 additions & 0 deletions tests/pybind/functions_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Copyright (c) 2018 - 2023 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.

import amd.rocal.fn as fn

from inspect import getmembers, isfunction

print('rocAL PyBind Functions')
rocalFunctions = getmembers(fn, isfunction)
for i in range(len(rocalFunctions)):
print(rocalFunctions[i])
28 changes: 28 additions & 0 deletions tests/pybind/pipelines_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Copyright (c) 2018 - 2023 Advanced Micro Devices, Inc. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.

from amd.rocal.pipeline import Pipeline

from inspect import getmembers, isfunction

print('rocAL PyBind Pipelines')
Pipelines = getmembers(Pipeline, isfunction)
for i in range(len(Pipelines)):
print(Pipelines[i])
Loading

0 comments on commit cc619f5

Please sign in to comment.