1.4. Bindings FAQ

What are the different components of a native binding?

In order to invoke a native function, LWJGL needs both Java and native code. Lets use glShaderSource, a function that was introduced in version 2.0 of the OpenGL specification:

void glShaderSource(GLuint shader, GLsizei count, const GLchar **strings, const GLint *length)

It's a void function that accepts an unsigned integer, a signed integer size, a pointer to an array of strings and a pointer to an array of signed integers. This is translated to the following native method in LWJGL:

public static native void nglShaderSource(int shader, int count, long strings, long length)

Note the 'n' prefix and the type mappings: Both the signed and unsigned integer parameters were mapped to a Java int and both pointers mapped to a Java long. We'll discuss these mappings later.

The 'n' prefix is not strictly necessary, but is added so that auto-completion in IDEs keeps native and non-native methods separate.

In addition to the native method, a few normal Java methods are included:

public static void glShaderSource(int shader, PointerBuffer strings, IntBuffer length)
public static void glShaderSource(int shader, CharSequence... strings)
public static void glShaderSource(int shader, CharSequence string)

These are the Java friendly methods that developers will normally use. The first specializes the pointer types and drops the count argument, the second replaces everything with a CharSequence vararg parameter and the third with a single CharSequence.

At the native side, everything is extremely simple. This is the C definition of the function pointer:

typedef void (APIENTRY *glShaderSourcePROC) (jint, jint, const intptr_t, const intptr_t);

and the JNI function implementation:

JNIEXPORT void JNICALL Java_org_lwjgl_opengl_GL20_nglShaderSource(
    JNIEnv *__env, jclass clazz, // unsed
    jint shader, jint count, jlong stringsAddress, jlong lengthAddress
) {
	glShaderSourcePROC glShaderSource = /* retrieve the function pointer */;
	const intptr_t strings = (const intptr_t)stringsAddress;
	const intptr_t length = (const intptr_t)lengthAddress;
	UNUSED_PARAM(clazz)
	glShaderSource(shader, count, strings, length);
}

The C types used do not match the original function, but they are binary compatible.

LWJGL makes a very important implementation decision here; the native code does nothing but cast parameters to the appropriate types and call the native function. All the complexity and mapping from user-friendly parameters to the correct native data is handled in Java code. All pointers are passed as primitive longs, primitive values are passed unchanged and that's it. The JNIEnv and jclass parameters are never touched and there's no interaction with the JVM in any way; no jobject parameters are ever passed and there are no C-to-Java calls. Native methods are also public, which allows users to build their own convenience methods on top of the raw ones and they can be certain that nothing fancy happens when a JNI function is called.

Moving forward, the current design will make it easier for LWJGL to adopt Project Panama that will be available in Java 10.

The above applies to all LWJGL bindings. Most users will never have to deal with the native methods directly and they shouldn't unless there's a good reason; using them is inherently unsafe. More experienced developers can do interesting things with raw pointer arithmetic and org.lwjgl.system.MemoryUtil utilities.

What is the __functionAddress parameter that many native methods require?

There are two problems with dynamically linked libraries: a) function pointers must be dynamically discovered at run-time and b) certain libraries might use different function pointers in different contexts. On the other hand the JNI code is static, which means that the only way to invoke such functions is by passing function addresses down to native code.

The '__' prefix is there to make it more obvious that it's a synthetic parameter.

Unless the native methods are used directly, users rarely have to pass a function address explicitly. LWJGL uses special mechanisms to do it automatically. For example, OpenGL methods use thread-local storage for the current context and Vulkan methods use the dispatchable handle parameter to retrieve the correct function pointer.

How are primitive values mapped?

The mapping is generally straightforward, except for unsigned types and opaque pointers:

• Unsigned integer types are mapped to the corresponding signed type in Java. This might sound error-prone, but in most cases it isn't. They can easily be converted to and from higher precision integers using simple binary operations. The basic arithmetic operations, except division, work the same for both signed and unsigned integers. Java 8 also provides useful utilities for handling them.

• Opaque pointers are mapped to long in Java.

Opaque pointers are pointers to data that has unspecified structure and usually correspond to internal state managed by a 3rd-party API. The values of such pointers are simply passed in and out of the API and are never used to directly access memory.

Lets see some examples. The following function declarations:

void glDepthMask(GLboolean mask)
void glAlphaFunc(GLenum func, GLfloat ref)
void glClear(GLbitfield mask)
void glCopyPixels(GLint x, GLint y, GLsizei width, GLsizei height, GLenum type)
void glVertexAttrib4s(GLuint index, GLshort v0, GLshort v1, GLshort v2, GLshort v3)
void glVertexAttrib4Nub(GLuint dinex, GLubyte x, GLubyte y, GLubyte z, GLubyte w)
void glfwSwapBuffers(GLFWwindow* window)

are mapped to the following Java methods:

public static void glDepthMask(boolean flag)
public static void glAlphaFunc(int func, float ref)
public static void glClear(int mask)
public static void glCopyPixels(int x, int y, int width, int height, int type)
public static void glVertexAttrib4s(int index, short v0, short v1, short v2, short v3)
public static void glVertexAttrib4Nub(int index, byte x, byte y, byte z, byte w)
public static void glfwSwapBuffers(long window)

How are pointers to primitive values mapped?

Given a native function declaration like:

void glUniform4fv(GLint location, GLsizei count, const GLfloat *value);

we have the following Java methods:

public static native void nglUniform4fv(int location, int count, long value) // A
public static void glUniform4fv(int location, FloatBuffer value) // B

In this case, we have a float pointer and an explicit count parameter that specifies how many vec4 vectors should be read from that pointer.

In GLSL, a vec4 is a vector of 4 float values. So the pointer must point to an array of count * 4 * 4 bytes.

In method A, which is the raw JNI method, the count parameter and the pointer address are explicit.

In method B, two transformations have taken place; a) the value parameter is now a FloatBuffer, matching the native pointer type and b) the count parameter is now implicit, replaced by value.remaining() / 4. The 4 is there because each count is a vector of 4 values.

The current Buffer.position() affects the pointer address that will be passed to the native function. If a FloatBuffer wraps a pointer with address x and the current position is 2, the address x + 8 will be used.

The current Buffer.limit() controls how many values will be read from or written to a buffer. Combined with the note on Buffer.position() above, this matches how java.nio buffers are normally used in the JDK. The only difference is that LWJGL never modifies the current buffer position or limit. This reduces the need to flip() or rewind() buffers.

How are output parameters handled?

Output parameters are standard data pointer parameters. The memory to which they point will be used by the function to return data.

A classic example is glGetIntegerv, a very useful OpenGL function:

void glGetIntegerv(GLenum pname, GLint *data)

LWJGL includes the following methods for this function:

public static native void nglGetIntegerv(int pname, long params) // A
public static void glGetIntegerv(int pname, IntBuffer params) // B
public static int glGetInteger(int pname) // C

Methods A and B are similar to the ones above, except there's no explicit count parameter. If there was a count parameter, the same transformation would have been applied in the case of B (params.remaining() would be used implicitly). For this function, the user must make sure there's enough space for the returned data, based on the queried parameter.

This is admittedly bad design, but it's a very old function. Modern OpenGL functions do a much better job with validation and most of them require explicit sizes.

Method C is the interesting one. It can be used when the query returns a single value. Without it, the user would have to allocate a single value buffer, call the method, then read the value from the buffer, a verbose procedure. Instead, LWJGL drops the params parameter and changes the method return type from void to int. This makes it much more natural and convenient to use.

LWJGL uses org.lwjgl.system.MemoryStack to implement methods like C. Such methods can safely be used from multiple contexts concurrently.

That's cool. Can the same be done for single-value input parameters?

Of course, if it's useful for a particular function. For example:

void glDeleteTextures(GLsizei n, const GLuint *textures)

can be used with any of these:

public static native void nglDeleteTextures(int n, long textures)
public static void glDeleteTextures(IntBuffer textures)
public static void glDeleteTextures(int texture) // single value!

Is there a convenient way to use String or CharSequence?

Yes, LWJGL makes handling text data very easy. The convention for text is that input parameters are mapped to CharSequence and output (returned) values are mapped to String. LWJGL handles character encodings correctly, it supports ASCII, UTF-8 and UTF-16 encoding/decoding and will use the right one depending on the function used. Finally, it can just as easily handle null-terminated strings and strings with explicit lengths. Some examples:

GLint glGetAttribLocation(GLuint program, const GLchar *name) // A) null-terminated input
GLubyte *glGetString(GLenum name) // B) null-terminated output
void glGetProgramInfoLog(GLuint program, GLsizei bufSize, GLsizei *length, GLchar *infoLog) // C) output w/ explicit length

and the Java methods:

// A)
public static native int nglGetAttribLocation(int program, long name)
public static int glGetAttribLocation(int program, ByteBuffer name)
public static int glGetAttribLocation(int program, CharSequence name)
// B)
public static native long nglGetString(int name)
public static String glGetString(int name)
// C)
public static native void nglGetProgramInfoLog(int program, int maxLength, long length, long infoLog)
public static void glGetProgramInfoLog(int program, IntBuffer length, ByteBuffer infoLog)
public static String glGetProgramInfoLog(int program, int maxLength)
public static String glGetProgramInfoLog(int program)

Bonus feature: the last glGetProgramInfoLog will use glGetProgrami(program, GL_INFO_LOG_LENGTH) internally to automatically allocate enough storage for the returned log text!

How can callbacks from native code be configured?

Each native callback type is mapped to an interface/abstract class pair. Similar to CharSequence and String, when the callback type is an input parameter it is mapped to the interface and when an output (return value) to the abstract class. This is important for two reasons:

• Lambda SAM conversions are only applicable to interfaces. All callback interfaces are functional interfaces by definition.
• Native callback functions are generated at runtime. This means that there are native resources that must be stored as state and be explicitly deallocated when no longer used. Only the abstract class can have state and a free() method.

An example of a callback function that can be registered in an OpenGL context is:

// Function signature:
typedef void (APIENTRY *GLDEBUGPROC)(GLenum source, GLenum type, GLuint id, GLenum severity, GLsizei length, const GLchar* message, GLvoid* userParam)
// Callback registration:
void glDebugMessageCallback(GLDEBUGPROC callback, const void *userParam)

These are mapped to:

public static native void nglDebugMessageCallback(long callback, long userParam)
public static void glDebugMessageCallback(GLDebugMessageCallbackI callback, long userParam)

and the org.lwjgl.opengl.GLDebugMessageCallbackI interface, which defines a single method:

void invoke(int source, int type, int id, int severity, int length, long message, long userParam);

Note that glDebugMessageCallback accepts a GLDebugMessageCallbackI. The native callback function is generated when glDebugMessageCallback is called and the pointer to that function is passed to LWJGL. This means that simply passing a lambda to glDebugMessageCallback would cause a memory leak, because there is nothing returned that the developer can use to free the generated native callback function. There are two approaches to solve this problem:

// Method 1: store this reference in a Java variable.
GLDebugMessageCallback cb = GLDebugMessageCallback.create((source, type, id, severity, length, message, userParam) -> {
    // print message
});
glDebugMessageCallback(cb, NULL);
// later...
cb.free();

// Method 2: use glGetPointer to retrieve the callback
glDebugMessageCallback((source, type, id, severity, length, message, userParam) -> {
    // print message
}, NULL);
// later...
GLDebugMessageCallback.create(glGetPointer(GL_DEBUG_CALLBACK_FUNCTION)).free();

The second method is more convenient, but not all APIs provide a way to retrieve a callback function pointer that was previously set. In such cases, the LWJGL user is responsible for storing the callback reference in Java code.

Is there a special mapping for native structs?

Yes, each struct type is mapped to a class. The class contains creation/allocation methods and getters/setters for the struct members. The struct layout (field offsets, sizeof, alignof) is available as static final int fields.

LWJGL supports structs, unions and any combination of those nested inside one another. The member offsets and alignment/size characteristics are calculated at runtime, to handle pointer size differences and dynamic padding.

Each struct class also has an inner Buffer class that represents an array of those structs. The Buffer class has an API that is compatible with java.nio buffers and also getters/setters that can be used to access the array of structs with the flyweight pattern.

Example:

typedef struct GLFWimage
{
    int width;
    int height;
    unsigned char* pixels;
} GLFWimage;
GLFWcursor* glfwCreateCursor(const GLFWimage* image, int xhot, int yhot);
GLFWAPI void glfwSetWindowIcon(GLFWwindow* window, int count, const GLFWimage* images);

The above are mapped to:

public class GLFWImage extends Struct implements NativeResource

public static long nglfwCreateCursor(long image, int xhot, int yhot)
public static long glfwCreateCursor(GLFWImage image, int xhot, int yhot) // A

public static void nglfwSetWindowIcon(long window, int count, long images)
public static void glfwSetWindowIcon(long window, GLFWImage.Buffer images) // B

The GLFWImage has the following static fields:

GLFWImage.SIZEOF // sizeof(GLFWimage)
GLFWImage.ALIGNOF // alignof(GLFWimage)
GLFWImage.WIDTH // offsetof(GLFWimage, width)
GLFWImage.HEIGHT // offsetof(GLFWimage, height)
GLFWImage.PIXELS // offsetof(GLFWimage, pixels)

and the following getters and setters:

img.width()
img.height()
img.pixels()

There are also "unsafe" static getters and setters that can be used to access memory locations that are not wrapped in a struct or struct buffer class.

In the above methods, A is straightforward. The result of image.address() is passed to the native code. In B images is an array of GLFWimage structs. Note that it's been mapped to GLFWImage.Buffer and the count parameter has been removed, just like in other methods that accept auto-sized java.nio buffers.

The memory backing structs and struct buffers can be any off-heap memory. MemoryStack, MemoryUtil and BufferUtil can all be used to allocate struct instances, with the corresponding usability/performance characteristics.