Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adsk Contrib - Improve the OSL unit test framework #1514

Conversation

hodoulp
Copy link
Member

@hodoulp hodoulp commented Oct 7, 2021

Signed-off-by: Patrick Hodoul Patrick.Hodoul@autodesk.com

The pull request finalizes the OSL unit test framework to compile and run the generated OSL shaders. In order to ease the unit test coverage, the OSL unit tests have exactly the same name & behavior than the GPU ones except that there are no LUT and dynamic property support.

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
@hodoulp
Copy link
Member Author

hodoulp commented Oct 8, 2021

When using -DOCIO_INSTALL_EXT_PACKAGES=ALL the custom FindImath() uses a custom detection (otherwise, it uses the usual find_package(), etc.) declaring the target Imath::Imath for a static library; however, it seems that the find_package(OpenShadingLanguage 1.11.15) then fails to correctly find another Imath target. So, the cmake configuration step succeeds but the generation step fails.

cmake -DCMAKE_CXX_STANDARD=17 -DOCIO_INSTALL_EXT_PACKAGES=ALL ../.

-- Setting SOVERSION to '2.2' as none was specified.
-- Installing expat: /home/aswfuser/ocio_1/build_rls/ext/dist/lib64/libexpat.a (version "2.2.8")
-- Installing yaml-cpp: /home/aswfuser/ocio_1/build_rls/ext/dist/lib/libyaml-cpp.a (version "0.6.3")
-- Installing pystring: /home/aswfuser/ocio_1/build_rls/ext/dist/lib/libpystring.a (version "1.1.3")
-- Installing Imath: /home/aswfuser/ocio_1/build_rls/ext/dist/lib64/libImath-3_1.a (version "3.1.2")
-- Installing lcms2: /home/aswfuser/ocio_1/build_rls/ext/dist/lib/liblcms2.a (version "2.2")
-- Installing pybind11: /home/aswfuser/ocio_1/build_rls/ext/dist/include (version "2.6.1")
-- Found Python: /usr/local/bin/python3.9 (found version "3.9.7") found components: Interpreter Development.Module 
-- Found OpenImageIO: /usr/local/include (found suitable version "2.3.7.2", minimum required is "2.1.9") 
-- OpenImageIO includes     = /usr/local/include
-- OpenImageIO libraries    = /usr/local/lib64/libOpenImageIO.so
-- OpenImageIO library_dirs = /usr/local/lib64
-- Found OpenImageIO: /usr/local/include (found suitable version "2.3.7.2", minimum required is "2.3.7.2") 
-- OpenImageIO includes     = /usr/local/include
-- OpenImageIO libraries    = /usr/local/lib64/libOpenImageIO.so
-- OpenImageIO library_dirs = /usr/local/lib64
-- OpenShadingLanguage includes    = /usr/local/include
-- OpenShadingLanguage shaders     = /usr/local/include/../share/OSL/shaders
-- OpenShadingLanguage library dir = /usr/local/lib64
CMake Warning at src/bindings/python/CMakeLists.txt:44 (message):
  Building PyOpenColorIO with OCIO_BUILD_DOCS disabled will result in
  incomplete Python docstrings.

-- Configuring done
CMake Error at tests/osl/CMakeLists.txt:19 (add_executable):
  Target "test_osl_exec" links to target "Imath::ImathConfig" but the target
  was not found.  Perhaps a find_package() call is missing for an IMPORTED
  target, or an ALIAS target is missing?

CMake Error at tests/osl/CMakeLists.txt:19 (add_executable):
  Target "test_osl_exec" links to target "Imath::ImathConfig" but the target
  was not found.  Perhaps a find_package() call is missing for an IMPORTED
  target, or an ALIAS target is missing?

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
@hodoulp
Copy link
Member Author

hodoulp commented Oct 8, 2021

The above commit finally completes the work on the Imath and OSL support for the OSL unit test framework.

When the locally compiled version of Imath is used (e.g. -DOCIO_INSTALL_EXT_PACKAGES=ALL), the OSL test make generation failed because of a missing target Imath::ImathConfig. It appears that some Imath versions now have two targets i.e. Imath::Imath and Imath::ImathConfig but share/cmake/module/findImath.cmake was only defining the first one in that case.

When using the system installed version of Imath, share/cmake/module/findImath.cmake finds the library using the find_package() method which relies on Imath to define the right targets. So, it was successfully generating the make files.

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
@remia
Copy link
Collaborator

remia commented Oct 17, 2021

Not commenting on the OSL side directly, even though the code looks good to me:

  • Looks like the additional Imath target was removed here via this issue but hasn't been released yet,
  • Do we want to add the required dependencies (OIIO and OSL are the only 2 not installed by Find-*.cmake scripts I believe) in the CI build to test the OSL layer ? Unless I'm mistaking none of it is currently run. We have code to install some them in the analysis workflow, except for OSL itself that is new obviously.

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
@hodoulp
Copy link
Member Author

hodoulp commented Oct 18, 2021

The previous commit improves the OSL library find and the OSL version to enhance the validation against the C++ version.

hodoulp and others added 3 commits October 18, 2021 11:07
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
@hodoulp
Copy link
Member Author

hodoulp commented Oct 19, 2021

Friendly reminder that the pull request can now be merged this week.

@hodoulp
Copy link
Member Author

hodoulp commented Oct 25, 2021

@remia @michdolan The pull request is now older enough to be merged (i.e. following the two-weeks rule). Could you have a look some so I can merge it this week?

@hodoulp
Copy link
Member Author

hodoulp commented Oct 25, 2021

Do we want to add the required dependencies (OIIO and OSL are the only 2 not installed by Find-*.cmake scripts I believe) in the CI build to test the OSL layer ?

The two libraries are part of the CI Linux containers, and the OSL unit tests successfully run. I did not plan to do anything for the Windows & macOS platforms because the OSL unit tests validate the OSL translation which is platform independent.

@remia @michdolan Do you think the Windows and macOS CI builds must now include OpenImageIO and OpenShadingLanguage ?

@remia
Copy link
Collaborator

remia commented Oct 26, 2021

The two libraries are part of the CI Linux containers, and the OSL unit tests successfully run. I did not plan to do anything for the Windows & macOS platforms because the OSL unit tests validate the OSL translation which is platform independent.

Good point, I missed that, only 3 out of 12 Linux CI jobs appears to have the dependencies to run the OSL tests but this is probably fine. I don't think we need to run these on Windows or macOS as you said it's platform independent, unless @lgritz thinks it's important.

@lgritz
Copy link
Collaborator

lgritz commented Oct 26, 2021

@remia No, as you said, the osl is platform independent.

@remia remia mentioned this pull request Oct 26, 2021
@hodoulp hodoulp merged commit 56ef7f5 into AcademySoftwareFoundation:master Oct 26, 2021
@hodoulp hodoulp deleted the adsk_contrib/improve_osl_support branch October 26, 2021 20:52
hodoulp added a commit that referenced this pull request Oct 26, 2021
* Adsk Contrib - Improve the OSL unit test framework

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix a Linux build break

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix a 'git rebase' break

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Some improvements

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Workaround to sucesfully link with OSL

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Found the right fix

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Add more tests for Grading ops

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Improve OSL library find

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix with OpenImageIO version requires C++14

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Another OSL cmake improvement

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
hodoulp added a commit that referenced this pull request Oct 28, 2021
* Adsk Contrib - Improve the OSL unit test framework

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix a Linux build break

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix a 'git rebase' break

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Some improvements

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Workaround to sucesfully link with OSL

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Found the right fix

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Add more tests for Grading ops

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Improve OSL library find

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Fix with OpenImageIO version requires C++14

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>

* Another OSL cmake improvement

Signed-off-by: Patrick Hodoul <Patrick.Hodoul@autodesk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants