[leveldb] Find and use snappy in the system

[Gab-Menezes]

Host Environment

• OS: Windows/WSL2 (Ubuntu 20.04)
• Compiler: g++-11
• Latest version of vcpkg and up-to-date with master

To Reproduce
Steps to reproduce the behavior:
./vcpkg install brpc

Failure logs
dbg-out.log file:

[328/338] : && /usr/bin/c++ -DBRPC_WITH_GLOG=1 -DGFLAGS_NS=google -DBTHREAD_USE_FAST_PTHREAD_MUTEX -D__const__= -D_GNU_SOURCE -DUSE_SYMBOLIZE -DNO_TCMALLOC -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DBRPC_REVISION=\"0.9.7\|master\|7e7dad5fe\|2022-02-04T18:00:26-08:00\" -D__STRICT_ANSI__ -g -DENABLE_THRIFT_FRAMED_PROTOCOL -O2 -pipe -Wall -W -fPIC -fstrict-aliasing -Wno-invalid-offsetof -Wno-unused-parameter -fno-omit-frame-pointer -msse4 -msse4.2 -Wno-aligned-new -DNDEBUG -g -rdynamic src/CMakeFiles/protoc-gen-mcpack.dir/mcpack2pb/generator.cpp.o -o output/bin/protoc-gen-mcpack  -Wl,-rpath,/mnt/c/dev/vcpkg-dev/vcpkg/buildtrees/brpc/x64-linux-dbg/output/lib  output/lib/libbrpc.so  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libgflags.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libprotobufd.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libleveldb.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libprotoc.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libcrypto.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libthriftd.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libssl.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libcrypto.a  -lpthread  -lpthread  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libssl.a  -lz  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libssl.a  -lz  -ldl  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libglog.a  -lrt  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libglog.a  -lrt && :
FAILED: output/bin/protoc-gen-mcpack 
: && /usr/bin/c++ -DBRPC_WITH_GLOG=1 -DGFLAGS_NS=google -DBTHREAD_USE_FAST_PTHREAD_MUTEX -D__const__= -D_GNU_SOURCE -DUSE_SYMBOLIZE -DNO_TCMALLOC -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DBRPC_REVISION=\"0.9.7\|master\|7e7dad5fe\|2022-02-04T18:00:26-08:00\" -D__STRICT_ANSI__ -g -DENABLE_THRIFT_FRAMED_PROTOCOL -O2 -pipe -Wall -W -fPIC -fstrict-aliasing -Wno-invalid-offsetof -Wno-unused-parameter -fno-omit-frame-pointer -msse4 -msse4.2 -Wno-aligned-new -DNDEBUG -g -rdynamic src/CMakeFiles/protoc-gen-mcpack.dir/mcpack2pb/generator.cpp.o -o output/bin/protoc-gen-mcpack  -Wl,-rpath,/mnt/c/dev/vcpkg-dev/vcpkg/buildtrees/brpc/x64-linux-dbg/output/lib  output/lib/libbrpc.so  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libgflags.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libprotobufd.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libleveldb.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libprotoc.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libcrypto.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libthriftd.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libssl.a  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/lib/libcrypto.a  -lpthread  -lpthread  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libssl.a  -lz  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libssl.a  -lz  -ldl  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libglog.a  -lrt  /mnt/c/dev/vcpkg-dev/vcpkg/installed/x64-linux/debug/lib/libglog.a  -lrt && :
/usr/bin/ld: output/lib/libbrpc.so: undefined reference to `snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)'
/usr/bin/ld: output/lib/libbrpc.so: undefined reference to `snappy::RawUncompress(char const*, unsigned long, char*)'
/usr/bin/ld: output/lib/libbrpc.so: undefined reference to `snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)'
/usr/bin/ld: output/lib/libbrpc.so: undefined reference to `snappy::MaxCompressedLength(unsigned long)'
collect2: error: ld returned 1 exit status

install-x64-linux-dbg-out.log

Additional context
I'm trying to build brpc to be able to create braft port mentioned here #22763
I have found another issue talking about this problem here: apache/doris#5253

[JackBoosY]

After reading the cmake configurations, I confirm that the executable target protoc-gen-mcpack would link to libbrpc.so which also exists in the log content.
So please run the following command and provide the output:
nm packages/brpc_x64-linux/debug/lib/libbrpc.so | grep RawCompress

Thanks.

[Gab-Menezes]

There is no debug folder inside packages/brpc_x64-linux only /share/brpc/vcpkg_abi_info.txt.
The libbrpc.so can be found at buildtrees/brpc/x64-linux-dbg/output/lib/libbrpc.so

[Gab-Menezes]

@JackBoosY are you able to reproduce it ? Because if #22685 got merged people are/were able to install it. Maybe I'm doing something wrong

[JackBoosY]

@Gab-Menezes No, I can't repro this issue locally.

[Gab-Menezes]

So I tried in my another computer that runs Ubuntu 20.04 with g++-10 and went fine. In my windows computer I run g++-11 so can be this.
I run abidiff between the 2 libbrpc.so files (from the windows computer to the ubuntu computer) (I know
changing compilers, optimizations, platform specific code ... can change abi) but I got no reference to snappy.
diff.txt

[Gab-Menezes]

Changed to g++-10 and didn't work. Did you use wsl or native linux ?
Because if run native linux it can be something related to wsl

[JackBoosY]

@Gab-Menezes native linux.

[Gab-Menezes]

@JackBoosY I tried in another windows computer with a fresh WSL2 install and got the same error
This makes me think that maybe there is a missing package in WSL installation, because openssl for example would not build without build-essentials (and it didn't mention it, I had to discover by myself)

[JackBoosY]

Maybe related to the link order.

[Gab-Menezes]

@JackBoosY I tried compiling in fresh Ubuntu 20.04 install using a VM, and wasn't able to compile either. So I decided to list all the packages installed in my Ubuntu computer and the VM, wrote a python script to take the diferrence between the 2 lists. With this I had all the possible packages that could help. So I decided to only install the ones that start with lib*, went through one by one searching what the package was for, and end up with this list:

libjq1
libjsoncpp1
libminizip1
libncurses-dev
libnvpair1linux
libonig5
libpfm4
librhash0
libstb0
libtinfo-dev
libubsan0
libuutil1linux
libuv1-dev
libz3-4
libz3-dev
libzpool2linux
zlib1g-dev

After this installed all of them in the VM and tried to build brpc and was successful, but after installing all of this packages in my WSL I still wasn't able to build.

What makes me think that is one of those packages + another one that come with native Ubuntu and doesn't come with Ubuntu in WSL2

[Gab-Menezes]

Dude this doesn't make any sense. I installed this packages in another windows computer and was able to compile, but not in my personal one and every single package installed in this computer is installed in mine also. I tried to replicate everything g++ version, cmake version.

[Gab-Menezes]

@JackBoosY Found it !!!!!!!!!!!!!!!! The problem occurs when you have libsnappy-dev installed. The leveldb package will try to link with snappy, but things will go wrong haha. So to install brpc you need one of those packages and can't have libsnappy-dev installed.

Can I close this issue ?

[JackBoosY]

@Gab-Menezes Thanks for digger into this, please do not close this issue unless we fix that.

[JackBoosY]

Finally I can reproduce this issue and I will fix this later.