javacpp-presets llvm bitcode example ?

[eix128]

Hi ,
There is binary lifter project :
https://github.com/lifting-bits/mcsema
This project generates bitcode from binary.

I want to call bitcode function inside my java app.
I can use bitcode on C++ Like this:

#include <string>
#include <memory>

#include <llvm/Bitcode/ReaderWriter.h>
#include <llvm/ExecutionEngine/ExecutionEngine.h>
#include <llvm/ModuleProvider.h>
#include <llvm/Support/MemoryBuffer.h>
#include <llvm/ExecutionEngine/JIT.h>

using namespace std;
using namespace llvm;

int main()
{
InitializeNativeTarget();
llvm_start_multithreaded();
LLVMContext context;

string error;
auto_ptr<MemoryBuffer> buffer(MemoryBuffer::getFile("bitcode.bc"));
auto_ptr<Module> module(ParseBitcodeFile(buffer.get(), context, &error));
auto_ptr<ModuleProvider> mp(new ExistingModuleProvider(module));
module.release();

auto_ptr<ExecutionEngine> ee(ExecutionEngine::createJIT(mp.get(), &error));
mp.release();

Function* func = ee->getFunction("foo");

typedef void (*PFN)();
PFN pfn = reinterpret_cast<PFN>(ee->getPointerToFunction(func));
pfn();

}

How can i use same on javacpp-presets llvm Java ?
file : "bitcode.bc"
method: "foo"
with some params and get return value.

I need example use case

[saudet]

Looks a lot like the following sample. Did you give it at try?
https://github.com/bytedeco/javacpp-presets/tree/master/llvm/samples/polly

[eix128]

is these methods removed ?
optimizeModule(module, cpu, 3, 0);
createOptimizedJITCompilerForModule(engine, module, cpu, 3);

[saudet]

No, they were added by @yukoba.

[yukoba]

@eix128

is these methods removed ?

It is not released yet. You can use them from 1.5.4.

[eix128]

1.5.4 yes
I tried to compile llvm myself but it requires Visual Studio 2017
I ve edit some codes for Visual Studio 2019

bash cppbuild.sh -platform windows-x86_64 install llvm

/d/repo/javacpp-presets /d/repo/javacpp-presets/llvm
Detected platform "windows-x86_64"
Building platform "windows-x86_64"
Installing "llvm"
/d/repo/javacpp-presets/llvm/cppbuild /d/repo/javacpp-presets
Decompressing archives... (ignore any symlink errors)
Total bytes read: 416645120 (398MiB, 1.3MiB/s)
tar: clang-10.0.0.src/test/Driver/Inputs/multilib_32bit_linux_tree/usr/bin/ld: Cannot create symlink to ‘i386-unknown-linux-gnu-ld’: No such file or directory
tar: clang-10.0.0.src/test/Driver/Inputs/multilib_64bit_linux_tree/usr/bin/ld: Cannot create symlink to ‘x86_64-unknown-linux-gnu-ld’: No such file or directory
Total bytes read: 135649280 (130MiB, 1.2MiB/s)
tar: Exiting with failure status due to previous errors
Total bytes read: 135649280 (130MiB, 1.1MiB/s)
Total bytes read: 27238400 (26MiB, 1.4MiB/s)
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.19041.
CMake Error at CMakeLists.txt:49 (project):
Failed to run MSBuild command:

C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/MSBuild/Current/Bin/MSBuild.exe

to get the value of VCTargetsPath:

Microsoft (R) Build Engine version 16.6.0+5ff7b0c9e for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 6/16/2020 6:36:52 PM.
Project "D:\repo\javacpp-presets\llvm\cppbuild\windows-x86_64\llvm-10.0.0.src\build\CMakeFiles\3.17.3\VCTargetsPath.vcxproj" on node 1 (default targets).
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(777,5): error : The OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform='Win64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [D:\repo\javacpp-presets\llvm\cppbuild\windows-x86_64\llvm-10.0.0.src\build\CMakeFiles\3.17.3\VCTargetsPath.vcxproj]
Done Building Project "D:\repo\javacpp-presets\llvm\cppbuild\windows-x86_64\llvm-10.0.0.src\build\CMakeFiles\3.17.3\VCTargetsPath.vcxproj" (default targets) -- FAILED.

Build FAILED.

"D:\repo\javacpp-presets\llvm\cppbuild\windows-x86_64\llvm-10.0.0.src\build\CMakeFiles\3.17.3\VCTargetsPath.vcxproj" (default target) (1) ->
(_CheckForInvalidConfigurationAndPlatform target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(777,5): error : The OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform='Win64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [D:\repo\javacpp-presets\llvm\cppbuild\windows-x86_64\llvm-10.0.0.src\build\CMakeFiles\3.17.3\VCTargetsPath.vcxproj]

    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.18

Exit code: 1

-- Configuring incomplete, errors occurred!
See also "D:/repo/javacpp-presets/llvm/cppbuild/windows-x86_64/llvm-10.0.0.src/build/CMakeFiles/CMakeOutput.log".
/d/repo/javacpp-presets/llvm

[saudet]

No need to build it from source, you can use the snapshots: http://bytedeco.org/builds/

[eix128]

Okay nice ,
it worked now.
This example is generating bitcode itself.
I want to load bitcode from file that is generated from mcsema
https://github.com/lifting-bits/mcsema

and run the function with params and get return values , including pass by pointer .
Also it would be good example call java callback from bitcode

[Neiko2002]

@eix128 I've have created a small library to load, compile and run LLVM IR code. You might wanna take a look. Interesting for you should be the LLVMStoredModuleBuilder class.

[eix128]

@Neiko2002 Thanks , very nice , what about running function ?

[Neiko2002]

@eix128 Using this library, you create a Java interface containing methods that have the same name and signature as the functions in the LLVM IR code. I have no idea how the generated code form mcsema looks like and how they manage the naming. But you can build an LLVM module and iterate thru all functions it has. Then you can ask for the function name and signature similar to what I did in the LLVMProgram class.

[eix128]

@Neiko2002 Mcsema is generating LLVM native bitcode.
So it can be run directly with LLVM
Look at top for C++ example:
#894 (comment)

But there is no java equvalient

[Neiko2002]

@eix128 Sorry I do not get it. Of course, LLVM native bitcode can be run directly with LLVM, that's exactly what my library does. It just adds a layer of type safety. You were asking how to run a native function and I described how to do it with llvm-jnr.

But back to you initial topic question. You want to run a "bitcode function inside my java app". Are you looking for an equivalent of your code snipped? Getting a function object from LLVM? Those functions are quite limited that's why saudet suggested using polly.

[saudet]

But back to you initial topic question. You want to run a "bitcode function inside my java app". Are you looking for an equivalent of your code snipped? Getting a function object from LLVM? Those functions are quite limited that's why saudet suggested using polly.

Not "polly", maybe you mean "clang" here? BTW, why is it that we cannot always use LLVMRunFunction() to execute functions generated with LLVMAddFunction()? @yukoba Do you know?

[saudet]

Ah, ok, LLVMRunFunction() is the one that only supports main() like functions. Here's some history about that:
https://www.mail-archive.com/llvm-bugs@lists.llvm.org/msg07603.html

Well, once we get libc++ in there, we can start using Clang for maximum performance with this :)

[Neiko2002]

Not "polly", maybe you mean "clang" here?

I just said polly, because you pointed him to the polly example in your initial answer of this topic. What he is looking for is more the JNA part of this example or the JNR part in my library, where he can call arbitrary functions with no parameter restrictions like LLVMRunFunction().

[saudet]

BTW, you guys should checkout TVM: https://github.com/bytedeco/javacpp-presets/tree/master/tvm

It's pretty awesome. It uses LLVM as its main backend, but it can link with MKL, etc as well, so we get everything in the same framework to get great overall performance like this:
https://medium.com/apache-mxnet/speed-up-your-bert-inference-by-3x-on-cpus-using-apache-tvm-9cf7776cd7f8

Although it's made to compile "deep learning models", it's IR is actually pretty general and we can easily create custom graphs with it's high-level API like this:
https://tvm.apache.org/docs/dev/relay_intro.html

It's mainly usable from Python (callable from Java with the presets for CPython), but since it's getting pretty stable, they've also started to offer a C++ API, that we can wrap with JavaCPP to use it in Java with callbacks and everything without any Python at all.

[saudet]

/cc @oneengineer

[oneengineer]

@eix128
You can use java JNA to run the LLVM JIT compiled function, like the code below.
It provide you the ability to prepare parameter data purely in Java. LLVMRunFunction has some limitations when you pass the parameters.

I remeber @Neiko2002 has several good examples.

https://github.com/Neiko2002/llvm-orc/blob/master/src/main/java/net/wpm/llvm/LLVMOrcEagerlyCompiler.java#L99

        long addr = LLVMGetFunctionAddress(engine, "aplusb");
        var pointer = com.sun.jna.Pointer.createConstant( addr );
        var function = com.sun.jna.Function.getFunction(pointer);
        Object r = function.invoke( int.class, new Object[] { 10, 23} );
        int temp = (Integer)r;
        System.out.println("result is " + temp); // 33

[saudet]

BTW, you guys should checkout TVM: https://github.com/bytedeco/javacpp-presets/tree/master/tvm

It's pretty awesome. It uses LLVM as its main backend, but it can link with MKL, etc as well, so we get everything in the same framework to get great overall performance like this:
https://medium.com/apache-mxnet/speed-up-your-bert-inference-by-3x-on-cpus-using-apache-tvm-9cf7776cd7f8

Although it's made to compile "deep learning models", it's IR is actually pretty general and we can easily create custom graphs with it's high-level API like this:
https://tvm.apache.org/docs/dev/relay_intro.html

It's mainly usable from Python (callable from Java with the presets for CPython), but since it's getting pretty stable, they've also started to offer a C++ API, that we can wrap with JavaCPP to use it in Java with callbacks and everything without any Python at all.

And here's a good step in the right direction for the generalization of a framework like TVM:
https://medium.com/octoml/compiling-classical-ml-for-up-to-30x-performance-gains-and-hardware-portability-2aef760af694

[saudet]

/cc @supergrecko

[junlarsen]

If you'd like I can include a sample that calls a function at a given address using JNA in #1016

I should also add an example that does this for the libgccjit preset.

[saudet]

@supergrecko No need to, there's already a sample from @yukoba using JNA here:
https://github.com/bytedeco/javacpp-presets/blob/master/llvm/samples/polly/MatMulBenchmark.java

@eix128 Does the new sample code from @supergrecko at pull #1016 answer your original question?
Rereading this thread though, the question was actually about calling function pointers, same as issue #833.

BTW, the presets for LLVM now bundle Clang and LLD, and I've also tried to bundle libc++, but it's a bit trickier as it still depends on the native C header files and libraries (glibc on Linux, UCRT on Windows, etc). It seems like it would be hard to come up with something that is fully self-contained as with libffi... :(

[saudet]

Duplicate of #833

[saudet]

FYI, I've added presets for libffi in commit 3075f99, so we can now use that directly at least to avoid any overhead from JNA or JNR.