mpv crashes with scripts using luasocket

[cript0nauta]

Describe the bug

When mpv is compiled with samba support, running mpv with a script that uses luasocket will trigger a segmentation fault. This happens because both luasocket and samba export symbols with the same name (such as socket_listen and socket_create). When the luasocket C code attempts to call its own functions, it will end up calling samba functions. This will obviously crash.

If I override mpv with { sambaSupport = false; } it works ok, but I'm not sure if disabling samba support is desired.

To Reproduce
Steps to reproduce the behavior:

1. Create the following script and save it into /tmp/script.lua:

local socket = require("socket")
local server = socket.bind("0.0.0.0", 44444)
print("bind successful")

2. Run nix-shell -p mpv
3. Run mpv /dev/null --script=/tmp/script.lua

Expected behavior
A clear and concise description of what you expected to happen.

Running the previous command should produce the following output:

[script] bind successful
Failed to recognize file format.


Exiting... (Errors when loading file)

instead of crashing with a segmentation fault.

Additional context

• To test this bug it is required to use a nixpkgs version that has the commit a6dfaad. Otherwise, mpv won't be able to import luasocket.
• Samba support for mpv is enabled in Linux systems. Therefore, this issue doesn't exist in darwin systems.

Notify maintainers
@AndersonTorres @fpletz @globin @Ma27 @tadeokondrak

Metadata

 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.33, NixOS, 20.09pre222244.22a3bf9fb9e (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
- mpv
- luasocket
- samba
# a list of nixos modules affected by the problem
module:

[AndersonTorres]

Looks like an upstream bug: using libraries with conflicting symbols.

In fact, I have Googled it and got this:

mpv-player/mpv#2009

[doronbehar]

This happens because both luasocket and samba export symbols with the same name (such as socket_listen and socket_create). When the luasocket C code attempts to call its own functions, it will end up calling samba functions.

Interesting suspicion, but I I don't understand how you reached this conclusion - the wrapper has in the environment LUA_CPATH and LUA_PATH while samba's symbols are supposed to be linked "hardly" as reported by ldd - without any relation to the environment.

The wrapper makes mpv run with LUA_CPATH and LUA_PATH containing both the user's (if already set) search directories and the luasocket and lua search paths that were used in the build of mpv.

Initially, I had the suspicion that if I'll unset my own LUA_CPATH and LUA_PATH environment, and let only the wrapper's environment influence it won't segfault, but I was wrong, it still segfaults.

I think that the original intention behind the wrapper's adding this to the environment:

export LUA_CPATH='/nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/lib/lua/5.2/?.so'${LUA_CPATH:+';'}$LUA_CPATH
export LUA_PATH='/nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/share/lua/5.2/?.lua'${LUA_PATH:+';'}$LUA_PATH

Was to make everyone be able to write Lua scripts that load luasocket, and indeed without this environment (and without it being preset in the environment), loading that example script fails due to unfound luasocket.lua, as expected.

Anyway, I tend to think this is indeed an upstream issue. I tried to investigate it with gdb but I didn't manage to figure out what's going on after I run:

Reading symbols from /nix/store/8qfz80qadzind5rxr38mgmdjb9y5c1f2-mpv-0.32.0/bin/mpv...

warning: Loadable section ".dynstr" outside of ELF segments
(ins)(gdb) run
Starting program: /nix/store/8qfz80qadzind5rxr38mgmdjb9y5c1f2-mpv-0.32.0/bin/mpv ./result/bin/mpv
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/nix/store/nwsn18fysga1n5s0bj4jp4wfwvlbx8b1-glibc-2.30/lib/libthread_db.so.1".
[New Thread 0x7ffff0b32700 (LWP 24180)]
[New Thread 0x7ffff0331700 (LWP 24181)]
[New Thread 0x7fffefb30700 (LWP 24182)]
[New Thread 0x7fffef32f700 (LWP 24183)]
[New Thread 0x7fffeeb2e700 (LWP 24184)]
[New Thread 0x7fffedf6b700 (LWP 24185)]
[Thread 0x7fffedf6b700 (LWP 24185) exited]
Failed to recognize file format.


Exiting... (Errors when loading file)
[Thread 0x7fffefb30700 (LWP 24182) exited]
[Thread 0x7fffef32f700 (LWP 24183) exited]
[Thread 0x7ffff0331700 (LWP 24181) exited]
[Thread 0x7fffeeb2e700 (LWP 24184) exited]
[Thread 0x7ffff0b32700 (LWP 24180) exited]
[Inferior 1 (process 24153) exited with code 02]
(ins)(gdb) bt
No stack.

[cript0nauta]

Anyway, I tend to think this is indeed an upstream issue. I tried to investigate it with gdb but I didn't manage to figure out what's going on after I run

I had the same problem with gdb. The problem was that the mpv executable is not an ELF binary, but a shell script wrapping it. I set the wrapped environment variables inside my shell:

export LUA_CPATH='/nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/lib/lua/5.2/?.so'${LUA_CPATH:+';'}$LUA_CPATH
export LUA_PATH='/nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/share/lua/5.2/?.lua'${LUA_PATH:+';'}$LUA_PATH

and then ran gdb to the .mpv-wrapped, that is the actual ELF binary:

gdb /nix/store/xgz19jvhjhlrnqnrxlv9ap3w2c0qyfhr-mpv-0.32.0/bin/.mpv-wrapped
(gdb) r /dev/null --script=/tmp/script.lua
Starting program: /nix/store/xgz19jvhjhlrnqnrxlv9ap3w2c0qyfhr-mpv-0.32.0/bin/.mpv-wrapped /dev/null --script=/tmp/script.lua
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/nix/store/nwsn18fysga1n5s0bj4jp4wfwvlbx8b1-glibc-2.30/lib/libthread_db.so.1".
[New Thread 0x7ffff0b34700 (LWP 21676)]
[New Thread 0x7ffff0333700 (LWP 21677)]
[New Thread 0x7fffefb32700 (LWP 21678)]
[New Thread 0x7fffef331700 (LWP 21679)]
[New Thread 0x7fffeeb30700 (LWP 21680)]
[New Thread 0x7fffee32f700 (LWP 21681)]

Thread 7 "mpv/lua script " received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffee32f700 (LWP 21681)]
0x00007ffff15792d0 in socket_getops_byname () from /nix/store/xa0wiy8296y7hi6a45f3hzz13placplp-samba-4.12.2/lib/samba/libsamba-sockets-samba4.so
(gdb) bt
#0  0x00007ffff15792d0 in socket_getops_byname () from /nix/store/xa0wiy8296y7hi6a45f3hzz13placplp-samba-4.12.2/lib/samba/libsamba-sockets-samba4.so
#1  0x00007ffff1579390 in socket_create () from /nix/store/xa0wiy8296y7hi6a45f3hzz13placplp-samba-4.12.2/lib/samba/libsamba-sockets-samba4.so
#2  0x00007fffedb2573b in inet_trycreate () from /nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/lib/lua/5.2/socket/core.so
#3  0x00007fffedb264ff in tcp_create () from /nix/store/h39qvnwi99vc78c0dzv02ldr9lxa7nay-lua-5.2.4-env/lib/lua/5.2/socket/core.so
#4  0x00007ffff4f74f6e in luaD_precall () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#5  0x00007ffff4f80c07 in luaV_execute () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#6  0x00007ffff4f75328 in luaD_call () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#7  0x00007ffff4f713db in lua_callk () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#8  0x00000000004a0391 in load_scripts ()
#9  0x00007ffff4f74f6e in luaD_precall () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#10 0x00007ffff4f752f2 in luaD_call () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#11 0x00007ffff4f7489c in luaD_rawrunprotected () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#12 0x00007ffff4f7557f in luaD_pcall () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#13 0x00007ffff4f714a7 in lua_pcallk () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#14 0x00000000004a1980 in run_lua ()
#15 0x00007ffff4f74f6e in luaD_precall () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#16 0x00007ffff4f752f2 in luaD_call () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#17 0x00007ffff4f7489c in luaD_rawrunprotected () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#18 0x00007ffff4f7557f in luaD_pcall () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#19 0x00007ffff4f714a7 in lua_pcallk () from /nix/store/w2bz5agszxvwcsq8qps07w4nd9vd8s6j-lua-5.2.4/lib/liblua.so.5.2
#20 0x00000000004a1df3 in load_lua ()
#21 0x00000000004a8cd6 in script_thread ()
#22 0x00007ffff5ed1edd in start_thread () from /nix/store/nwsn18fysga1n5s0bj4jp4wfwvlbx8b1-glibc-2.30/lib/libpthread.so.0
#23 0x00007ffff4844a4f in clone () from /nix/store/nwsn18fysga1n5s0bj4jp4wfwvlbx8b1-glibc-2.30/lib/libc.so.6

The backtrace indicates that luasocket's inet_trycreate function is called, and then it calls libsamba's socket_create. I guess that inet_trycreate tried to call luasocket's socket_create but libsmba's socket_create was called instead.

I don't have any idea why this might be happening.

[doronbehar]

Well done on getting that stack trace 👍, I thought I managed to run gdb on the unwrapped executable but I guess I was wrong...

@cript0nauta I think this should be reported upstream.

[cript0nauta]

I thought in reporting it to upstream, but it seems only nixpkgs' mpv is affected. I tested this in Debian Buster, Xubuntu 20.04 and Arch Linux. All of them use libsamba and show the [script] bind successful message, so I suspect the problem is in nixpkgs itself and not in the upstream version.

the wrapper has in the environment LUA_CPATH and LUA_PATH while samba's symbols are supposed to be linked "hardly" as reported by ldd - without any relation to the environment.

This is a good observation and could give a hint on the bug. It is strange that luasocket, which doesn't depend on libsamba, confuses its own functions with libsamba's functions.

[doronbehar]

I tested this in Debian Buster, Xubuntu 20.04 and Arch Linux.

Can you try and test this when the luasocket rock is installed in a non default location and you use LUA_PATH and LUA_CPATH pointing to that location?

[AndersonTorres]

I thought in reporting it to upstream, but it seems only nixpkgs' mpv is affected. I tested this in Debian Buster, Xubuntu 20.04 and Arch Linux. All of them use libsamba and show the [script] bind successful message, so I suspect the problem is in nixpkgs itself and not in the upstream version.

Maybe when calling the binaries the link resolution is solved in a certain order while the ordering in NixOS is not always the same.

[cript0nauta]

Ok, I've been able to reproduce the problem using Arch Linux and luarocks by running:

pacman -S mpv luarocks base-devel
luarocks install luasocket --version=5.2
mpv /dev/null --script=/tmp/script.lua

I didn't have to set LUA_PATH nor LUA_CPATH because luasocket it is installed in the default location.

If I install the lua52-socket package instead of using luarocks, the script runs successfully. Now I'm going to compare both versions to see why is the luarocks versions failing.

[cript0nauta]

I can now confirm that the problem only exists when using the luarocks version of luasocket. Using the GitHub version of luasocket, as we did before commit 274715c, fixed the issue. I'll prepare a PR reverting part of that commit so we can have mpv with both libsamba and luasocket working properly