[CRASH] Keydb crashes with redisearch in KeyDB image created from main branch #708
Comments
|
Just tried, though using this Dockerfile FROM ubuntu:20.04 AS builder
WORKDIR /keydb
RUN apt-get update
RUN apt-get install -y git make python3 build-essential make
RUN git clone -b v2.8.1 --recursive https://github.com/RediSearch/RediSearch.git
RUN cd RediSearch && make setup && make build
FROM eqalpha/keydb:latest AS prod
ADD keydb.conf /etc/keydb.conf
COPY --from=builder /keydb/RediSearch/bin/linux-x64-release/search/redisearch.so /etc/libs/redisearch.so
CMD ["keydb-server", "/etc/keydb.conf"]and following keydb.conf most of the config comes from my currently running cluster, I just remembered that I read somewhere that Though I'm getting the following error, I think I got this in former trials as well also commented in the forum, though seems to be down or having certificate issues currently or sthg Really would love to get RediSearch and RedisJSON working, that'd be 🔥 I think it would be really nice to be able to get the Redis modules from the redis-stack-server Dockerfile, instead of manually building them from source, though not sure that's going to work, I tried that once but failed as well, can share my Dockerfile later. Just realized that I also commented here #406 I understand that KeyDB is open source, which I highly appreciate and value, though it's a little worrying how little official reactions issues here get, same feels on Slack and the forums 🥲 |
|
Just now saw your comments on Slack and that you're using that Dockerfile because of the newer Ubuntu so tried as well, all went well until I started a container which resulted in basically the same as OP I think Bug reportDockerfileFROM ubuntu:20.04 AS builder
WORKDIR /keydb
RUN apt-get update
RUN apt-get install -y git make python3 build-essential make
RUN git clone -b v2.8.1 --recursive https://github.com/RediSearch/RediSearch.git
RUN cd RediSearch && make setup && make build
FROM eqalpha/keydb:latest AS prod
ADD keydb.conf /etc/keydb.conf
COPY --from=builder /keydb/RediSearch/bin/linux-x64-release/search/redisearch.so /etc/libs/redisearch.so
CMD ["keydb-server", "/etc/keydb.conf"]keydb.confTried again this time removed the Did it work for you @sapvs when loading redisearch later? |
|
@CanRau , Yes it works for me when loading redisearch later. I am loading them lazily, using wrapper entrypoint script, which loads both modules after keydb has started. Redis JSON has no issues loading at startup, only RediSearch module causes the crash. Setup repo with same here. root@46154a85d68f:/data# keydb-cli
127.0.0.1:6379> FT.CREATE idx:movie ON hash PREFIX 1 "movie:" SCHEMA title TEXT SORTABLE release_year NUMERIC SORTABLE rating NUMERIC SORTABLE genre TAG SORTABLE
OKCrash issue on load with RediSearch still exists, and also KeyDB image created from main branch shows redis_version:255.255.255 |
|
As long as the project is not abandoned that's fine with me, though curious what about Redis 7 |
|
Unfortunately we don't have a timeline for when we will merge Redis 7, the team is mostly busy with internal work for Snap. |
|
Having the same problem with redisearch in keydb v6.3.4, rejson v2.6.0, redisearch v2.8.1 |
|
@sotnikovse there is a "debug reload" command that will reload the rdb that you can try |
|
"debug reload" with loaded redisearch module failed with the same error. And with flash enabled, recreating of FT.CREATE works unexpectedly. |
|
Hi @sotnikovse Good to see you have reached the same point as me in January, which I posted the issue here: Like you, I got redisearch working with flash enabled, but unfortunately, the indexes are not created upon server restart. When I investigated further, I observed that with flash enabled, there is no "loading" of an RDB on startup, as the data resides on disk. This means the RDB loading is not performed, rather keydb accesses data directly from disk on demand. My next step was to see if I can create the search indexes on startup, but did not get far. We basically need to read all data from disk on startup and "index" them in the redisearch module. If anyone ever gets a chance to look into this, please let me know, keen to see redisearch working on keydb with flash. |
|
Looks like there have been 2 version upgrades since I tested in January. I can confirm the this PR introduced in v6.3.3 is the cause of the crash in redisearch.so: The line that causes the crash is: Unfortunately, to fix this, you will need to patch redisearch in 2 places: src/redismodule.h and src/spec.c After this, you will not get the crash on startup. |
|
Hi @keithchew, thank you very much for this solution, keydb v6.3.4, rejson v2.6.0 and patched redisearch v2.8.1 startup without crash. |
|
hi @sotnikovse not a problem, it is my pleasure. Please note that if you start with an empty flash folder and a non-empty RDB (eg backup from another instance), the above solution is not complete/robust. This is because the current code will fire both RDB_START and FLASH_START events, which is not ideal when you add code to create/load indexes at startup. You will need to update the startup process slightly to accommodate this scenario. |
bobbintb
added a commit
to bobbintb/RediSearch
that referenced
this issue
Sep 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Keydb crashes with redisearch in KeyDB image created from main branch
Crash report
Aditional information
Created KeyDB image using pkg/docker/Dockerfile
docker build --build-arg BRANCH=main -t keydb_local .Compiled RediSearch, copied binary to keydb image in above and started the container.
Dockerfile
Running this image causes failure. However when
CMD loadmoduleis removed from the Dockerfile. And loading module from keydb-cli causes no crash.Module is loaded and commands are available
The text was updated successfully, but these errors were encountered: