Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a generator of C interfaces #378

Merged
merged 9 commits into from
Jan 2, 2024

Conversation

amontoison
Copy link
Member

@maleadt @pengtu @michel2323
I worked on a generator of C interfaces today.
It takes as input a C++ header file and generate the C interfaces with a C header.
You have an example with onemkl_lapack.cpp and one_lapack.h.

Even if it's not perfect, it will highly help to interface the libraries of oneAPI.
Do we have a Julia artifact that contains all hxx / hpp such that I can test it a different library?

@maleadt
Copy link
Member

maleadt commented Nov 7, 2023

Great stuff! I wondered if we could consider calling the C++ libraries directly (given that we only need to support a well-defined subset of types, maybe it's possible to do the ABI conversions ourselves and treat the entry-points as C functions), but a generator is pretty great as well.

Do we have a Julia artifact that contains all hxx / hpp such that I can test it a different library?

No, I attempted packaging the oneAPI Base Toolkit in Yggdrasil at some point, but the installer is really redistribution-unfriendly. This is also why we're using Conda right now, much to my dismay.
Maybe it would be possible to create a simple oneAPI_Support_Headers JLL though, repackaging just the bits you need?

@pengtu
Copy link
Contributor

pengtu commented Nov 7, 2023

@amontoison : This is very cool!

@amontoison
Copy link
Member Author

amontoison commented Nov 27, 2023

@maleadt @pengtu @michel2323 @kballeda

I finalized the script generate_interfaces.jl.
When we run julia --project generate_interfaces.jl, the files onemkl.h and onemkl.cpp are updated in the folder deps/src.

The script uses the headers of the new artifact oneAPI_Support_Headers_jll.jl, which means that we can easily update the C interfaces in the future.

Before that generate_interfaces.jl updates the two files, it generates the following files in the current folder:

  • onemkl_blas.h
  • onemkl_blas.cpp
  • onemkl_lapack.h
  • onemkl_lapack.cpp
  • onemkl_sparse.h
  • onemkl_sparse.cpp

At the end of the process, these files are concatenated with onemkl_prologue.{h,cpp} and onemkl_epilogue.{h,cpp} to form onemkl.cpp and onemkl.h.

In the current state, we have a C interface for almost all BLAS and LAPACK routines. For SPARSE MKL, I don't know how to interface the type matrix_handle_t.
Can one of you take a look? It shouldn't be complicated.

If the sparse part works, we'll be able to use the sparse_gemv and sparse_gemm routines to solve sparse linear systems with Krylov.jl, and if possible, test this on Aurora. 🚀

@kballeda
Copy link
Contributor

kballeda commented Nov 27, 2023

@amontoison we are looking into enabling sparse primitives - sparse_gemv and sparse_gemm.

@pengtu
Copy link
Contributor

pengtu commented Dec 11, 2023

@maleadt : Could you merge this PR to the oneAPI.jl repo or need additional reviews/tests?

@maleadt maleadt merged commit 59f5981 into JuliaGPU:master Jan 2, 2024
1 check passed
@maleadt
Copy link
Member

maleadt commented Jan 2, 2024

Thanks @amontoison!

@amontoison amontoison deleted the generator_c_interfaces branch January 2, 2024 18:27
@maleadt
Copy link
Member

maleadt commented Jan 23, 2024

@amontoison I didn't realize you didn't use Clang.jl to parse the headers. Doing this string-based seems really fragile; did you consider using Clang.jl?

@amontoison
Copy link
Member Author

@maleadt Using Clang.jl is the best solution but I don't know how we can parse *.hxx herders with it to generate Julia interfaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants