feat(build): build ngx_wasm_module as a dynamic Nginx module

[locao]

Summary

Build ngx_wasm_module as a dynamic Nginx module.

$ bazel build //build:venv --//:wasmx=true --//:wasm_runtime=v8 --//:dyn_ngx_wasm_module=true

[locao]

Let's merge #10371 before merging this one.

[bungle]

Build ngx_wasm_module as a dynamic Nginx module.

Do we build anything else as dynamic? Is there a particular reason for making it dynamic (to be able to disable it if it causes issues?).

[locao]

@bungle We don't build any other module as dynamic. But this one is still a tech preview and people may want to fully disable it.

[hbagdi]

But this one is still a tech preview and people may want to fully disable it.

Could you please better explain what "disable" means here and how users can "fully disable" this module from a Kong package?

I'm not familiar with dynamic nginx modules, so please bear with me.

[locao]

Could you please better explain what "disable" means here and how users can "fully disable" this module from a Kong package?

@hbagdi Sorry, I was not clear. By "disable" I mean users can set wasm = off in kong.conf and WasmX will not be executed. And "fully disable" means users can omit the load_module directive in nginx_kong.conf and the WasmX module will not be loaded at all.

[hbagdi]

By "disable" I mean users can set wasm = off in kong.conf and WasmX will not be executed. And "fully disable" means users can omit the load_module directive in nginx_kong.conf and the WasmX module will not be loaded at all.

I'm more confused so here we go.

1. Why is it that kong doesn't omit 'load_module' directive when wasm=off? Meaning, why is it that disabled and "fully disable" are not the same thing?
2. How is that related to building this module as dynamic or static?
3. If this is statically built, what is the binary size?
4. If this is statically built, what is the impact on RSS/VSZ memory profiles?

[locao]

• Why is it that kong doesn't omit 'load_module' directive when wasm=off? Meaning, why is it that disabled and "fully disable" are not the same thing?
• How is that related to building this module as dynamic or static?

load_module is not available when the module is statically built, the module is part of the nginx binary. So we cannot "remove" the module after building nginx.

But you are right, when the module is built as dynamic we can remove the load_module directive when wasm = off.

• If this is statically built, what is the binary size?
• If this is statically built, what is the impact on RSS/VSZ memory profiles?

I didn't check this.

[hbagdi]

load_module is not available when the module is statically built, the module is part of the nginx binary. So we cannot "remove" the module after building nginx.

Ok, that makes more sense. I should also read nginx documentation more.

I didn't check this.

Do answers to these questions influence the decision of dynamic/static?
I'm still trying to figure out the motivation for this and I don't think I understand yet.

[flrgh]

I wonder if it's possible to de-dupe this shell command string so that it can be referenced only once for each flag in the select() call.

Can you give this a shot and see if it works? Honestly it's not that much prettier, but w/ the state of bazel I'll take any kind of win we can get.

install_wasm_lua_libs_cmd = """
    for fname in $(locations @ngx_wasm_module//:lua_libs); do
        base=${fname##*/ngx_wasm_module/lib/}
        dest="${BUILD_DESTDIR}/openresty/lualib/$base"
        mkdir -p "$(dirname "$dest")"
        cp "$fname" "$dest"
    done
"""

wasmx_lua_libs_cmd = select({
    "@kong//:static_wasmx_flag": install_wasm_lua_libs_cmd,
    "@kong//:dyn_wasmx_flag": install_wasm_lua_libs_cmd,
    "//conditions:default": "\n",
})

I wouldn't consider the duplicate string a blocker though. Don't spend too much time trying to make it work if bazel fights back.

[fffonion]

We can also use a config_settings_group like https://github.com/Kong/kong/blob/master/BUILD.bazel#L231
then you can have a new condition on say @kong//:any_wasmx_flag

[bungle]

• If this is statically built, what is the binary size?
• If this is statically built, what is the impact on RSS/VSZ memory profiles?

Nginx is in general really really small. Something like 1.7 MB on my machine (with all modules build in). I recently removed bunch of modules from our build. It did not have any effect to RSS, the binary size went down perhaps 100kB or so.

The problem I see with dynamic is that it is something that we have never shipped, aka Kong has never tried to load modules dynamically. I am sure it works as expected, but there is always some unknowns that are hard to catch. If that module is done right, I guess it will not run anything if not asked to (?)

[hbagdi]

ping @locao

[locao]

The problem I see with dynamic is that it is something that we have never shipped, aka Kong has never tried to load modules dynamically. I am sure it works as expected, but there is always some unknowns that are hard to catch. If that module is done right, I guess it will not run anything if not asked to (?)

I share this opinion. I will convert this PR to draft and check in the next sync meeting if this is something we really should pursue.

[locao]

Superseded by #11100