glslang shader processing #285

syntheticmagus · 2020-06-05T10:02:25Z

Preliminary work to process shaders using glslang instead of other custom processing mechanisms.

…tored implementation needs.

…g yet, hasn't worked but problem might be coming from elsewhere.

… linker object. A robust revision of ALL logic for linker objects is necessary to get the ASTs to match because replacements in the linker object aggregate should be symbol-only, without struct indexers or aggregates. So this is little more than a hack proving to show that, with the a little more logic work, we can render glTFs. On a related note, though, we can render (some) glTFs!

…lution.

syntheticmagus

Current State
What's done:

Updated Babylon.js to latest master (with modification to remove NativeShaderProcessor.ts).
On Windows, correctly renders cube, glTF cube, glTF textured cube, Brain Stem, and Flight Helmet.
Flight helmet tested both with and without IBL, works both ways.

What's not done.

100% untested cross-plat, almost guaranteed to not work considering everything I've done so far has been inside ShaderCompilerD3D.cpp.
Have not changed names on attributes/varying variables (a_position, a_rotation, etc.).
Have not added swizzle logic (appending .xyz to vec4s being assigned to vec3s, for example.).

…ing (if unfinished) states.

…ing.

CedricGuillemet · 2020-06-15T10:03:34Z

marble tower on Windows tested ok.
Metal/MacOS: brainstem, cesium man tested ok.

Todo:

the default cube lighting is wrong
error at metal shader link time for marble tower, avocado
test on simulator/device

CedricGuillemet · 2020-06-15T14:05:56Z

The 2 issues I have seem to be linked to the same root: location in varying between VS and FS.
VS Output:

struct xlatMtlMain_out
{
    float3 vNormalW [[user(locn0)]];
    float3 vPositionW [[user(locn1)]];
    float4 gl_Position [[position]];
};

FS Input:

struct xlatMtlMain_in
{
    float3 vPositionW [[user(locn0)]];
    float3 vNormalW [[user(locn1)]];
};

I'm adding a new traverser to support that. @syntheticmagus

CedricGuillemet · 2020-06-15T17:57:46Z

marble tower, avocado render good with my latest chances. I will test iOS tomorrow.

CedricGuillemet · 2020-06-16T09:00:24Z

Validation tests passing on Linux. I had to change the version to 330.

CedricGuillemet · 2020-06-16T10:22:00Z

on iOS, tested OK: cesiumman, avocado and marble tower.

…cmagus/BabylonNative into glslangShaderProcessing

…ming from glslang.

bghgary

Looks good overall. My biggest concern the opengl one which still goes to SPIR-V. Is that necessary?

bghgary · 2020-06-18T21:26:21Z

Plugins/NativeEngine/CMakeLists.txt

+target_compile_definitions(NativeEngine
+    PRIVATE NOMINMAX)


nit: Is this a WIN32 only thing?

I don't know if it's only meaningful for Windows or not, but we haven't been treating it as platform-specific so far. What's here matches the CMakeLists.txt files from NAPI, AppRuntime, NativeEngine, and I think everywhere else we use NOMINMAX.

I'm pretty sure this is WIN32 only, but we can fix that later if necessary.

bghgary · 2020-06-18T22:03:36Z

Plugins/NativeEngine/Source/ShaderCompilerOpenGL.cpp

            }

            program.addShader(&shader);
        }

-        std::unique_ptr<spirv_cross::Compiler> CompileShader(glslang::TProgram& program, EShLanguage stage, std::string& glsl)
+        std::pair<std::unique_ptr<spirv_cross::Parser>, std::unique_ptr<spirv_cross::Compiler>> CompileShader(glslang::TProgram& program, EShLanguage stage, std::string& glsl)
        {
            std::vector<uint32_t> spirv;
            glslang::GlslangToSpv(*program.getIntermediate(stage), spirv);


Do we still need to go to SPIRV for OpenGL?

Right now, the logic in NativeEngine still uses the SPIRV types to build the bgfx representation of the shader. Ultimately that logic should probably be moved out of NativeEngine and adapted to use glslang features instead of SPIRV. However, that looks like a fairly substantial change with only a tangential connection to this one, so my opinion is that we should leave that to its own PR. I've made an issue to track this: #300.

syntheticmagus added 18 commits June 3, 2020 14:38

VERY preliminary shader manipulation using glslang.

5552aad

Got the cube rendering.

eee7239

"Fixed" the position and normal remapping problem, for now.

8889cbd

Added pointer scope.

d5d2e16

Continuing cleanup, both in code and memory management.

ca99cd0

Still refactoring.

1d004a9

Simplifying and reducing.

82a7cf7

Removed generic scope type that was much more powerful than the refac…

e2714f0

…tored implementation needs.

WIP support for texture/sampler splitting. Hard to tell if it's worki…

9edfc41

…g yet, hasn't worked but problem might be coming from elsewhere.

Changed index type to match constant union precision precedent.

5647128

Still WIP, slowly narrowing the gap between what works and what doesn't.

630aa30

Still WIP, trying to understand.

69f59cc

Think I fixed an ID issue? Hard to be sure, still not working.

e34e6b6

Replaced linker object hack with (seemingly) more long-term viable so…

f793e62

…lution.

Removed debug code and created struct for ID generation.

19ee22a

Added support for array sizes in uniforms.

7c6f63e

Updated Babylon components to slightly-modified master.

1d3c72d

syntheticmagus commented Jun 8, 2020

View reviewed changes

syntheticmagus added 3 commits June 7, 2020 21:56

Merge from master.

dab17c9

Fixed warnings treated as errors.

24735fb

Fixed warning that popped up in CI but not locally for some reason.

0092697

syntheticmagus marked this pull request as ready for review June 8, 2020 06:57

syntheticmagus requested review from bghgary and CedricGuillemet June 8, 2020 06:57

syntheticmagus added 5 commits June 8, 2020 18:43

Attribute renaming.

d8a476a

Refactor moving traversers into their own file.

7bc8608

Half way to Android.

99bf91e

Back to working state for Windows.

c1362b7

Brain stem working on Android simulator.

7c3b47c

syntheticmagus and others added 3 commits June 12, 2020 18:43

Removed some debug code, Android and Windows now appear to be in work…

5e27383

…ing (if unfinished) states.

Changes to OpenGL's bgfx code required for new Android shader process…

b94d8e2

…ing.

Metal shader generation

99f2148

Disable location index for varying stage in/out

e8995ff

syntheticmagus added 3 commits June 18, 2020 11:12

Merge from master.

d026e7e

Merge branch 'glslangShaderProcessing' of https://github.com/syntheti…

2dbc66b

…cmagus/BabylonNative into glslangShaderProcessing

Disabled warning in ShaderCompilerTraversers.cpp to ignore warning co…

336001f

…ming from glslang.

bghgary approved these changes Jun 18, 2020

View reviewed changes

syntheticmagus merged commit d22508e into BabylonJS:master Jun 18, 2020

syntheticmagus deleted the glslangShaderProcessing branch June 18, 2020 22:38

syntheticmagus mentioned this pull request Jun 18, 2020

Gslang shader processing #286

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

glslang shader processing #285

glslang shader processing #285

syntheticmagus commented Jun 5, 2020

syntheticmagus left a comment

CedricGuillemet commented Jun 15, 2020

CedricGuillemet commented Jun 15, 2020 •

edited

Loading

CedricGuillemet commented Jun 15, 2020

CedricGuillemet commented Jun 16, 2020

CedricGuillemet commented Jun 16, 2020

bghgary left a comment

bghgary Jun 18, 2020

syntheticmagus Jun 18, 2020

bghgary Jun 18, 2020

bghgary Jun 18, 2020

syntheticmagus Jun 18, 2020

glslang shader processing #285

glslang shader processing #285

Conversation

syntheticmagus commented Jun 5, 2020

syntheticmagus left a comment

Choose a reason for hiding this comment

CedricGuillemet commented Jun 15, 2020

CedricGuillemet commented Jun 15, 2020 • edited Loading

CedricGuillemet commented Jun 15, 2020

CedricGuillemet commented Jun 16, 2020

CedricGuillemet commented Jun 16, 2020

bghgary left a comment

Choose a reason for hiding this comment

bghgary Jun 18, 2020

Choose a reason for hiding this comment

syntheticmagus Jun 18, 2020

Choose a reason for hiding this comment

bghgary Jun 18, 2020

Choose a reason for hiding this comment

bghgary Jun 18, 2020

Choose a reason for hiding this comment

syntheticmagus Jun 18, 2020

Choose a reason for hiding this comment

CedricGuillemet commented Jun 15, 2020 •

edited

Loading