Support KeyDB

[Spartee]

Description

KeyDB is a multi-threaded fork of Redis that we've provably seen to have better overall QPS and latency compared to vanilla Redis. With the introduction of configurable builds in #122 we should easily be able to support KeyDB.

Justification

Users looking to get maximum performance in terms of throughput should use and benefit from using KeyDB as the orchestrator backend. Since KeyDB is fully compatible with RedisAI, all the SmartRedis commands should work.

Implementation Strategy

Users should be able to switch between redis and keydb in the cli build of the dependency chain. They should also be able to specify this in the versioner (smartsim/_core/_install).

There are three places that need to be added to to support keydb directly.

Versioner

Versioner lists out the versions for each of the third-party dependencies. Here users should be able to use env vars to change the build to a KeyDB branch.

    # Redis
    REDIS = Version_(get_env("SMARTSIM_REDIS", "6.0.8"))
    REDIS_URL = get_env("SMARTSIM_REDIS_URL", "https://github.com/redis/redis.git/")
    REDIS_BRANCH = get_env("SMARTSIM_REDIS_BRANCH", REDIS)

This shouldn't be changed but we should be sure this route works.

builder.py

The RedisBuilder class build_from_git method needs to be changed to be able to move the binaries for KeyDB instead of Redis below. this should use glob for anything ending in server or cli

    @property
    def is_built(self):
        """Check if Redis is built"""
        server = self.bin_path.joinpath("redis-server").is_file()
        cli = self.bin_path.joinpath("redis-cli").is_file()
        return server and cli

# in RedisBuilder.build_from_git
        # move redis binaries to smartsim/smartsim/_core/bin
        redis_src_dir = redis_build_path / "src"
        self.copy_file(
            redis_src_dir / "redis-server", self.bin_path / "redis-server", set_exe=True
        )
        self.copy_file(
            redis_src_dir / "redis-cli", self.bin_path / "redis-cli", set_exe=True
        )

build.py

So in addition to the env var option, the CLI should have an option to build KeyDB in the build.py module which governs the smart build command

The change should add a --keydb flag as follows

        parser.add_argument(
            "--keydb",
            action="store_true",
            default=False,
            help="Build KeyDB instead of Redis",
        )

Open Design Questions

• Should we allow a Redis and KeyDB build to coexist?
• How do we specify that we need to siwtch configuration files in the orchestrator class?