feat: launch module from docker image #3
Conversation
crates/common/src/config.rs
Outdated
| pub extra: T, | ||
| } | ||
|
|
||
| impl<'de, T> Deserialize<'de> for ModuleConfig<T> |
There was a problem hiding this comment.
This custom deserializer is the quick approach I took to scratch my itch for modelling a strict config structure: exactly one of path or docker_image must be specified in the config.example.toml for a given module - marking the module as either launchable from a raw binary (like it used to be) OR from a docker image (the new addition).
We could choose another approach for modelling the config (i.e. an additional tag specifying the variant, and then the necessary source data (path or docker image id); or similar...)
crates/cli/src/lib.rs
Outdated
| let full_config_path = std::fs::canonicalize(&config_path).unwrap().to_string_lossy().to_string(); | ||
| Some(vec![format!("{}:{}", full_config_path, "/config.toml")]) | ||
| }, | ||
| network_mode: Some(String::from("host")), // Use the host network |
There was a problem hiding this comment.
It's worth noting here that I'm making substantial use of the host network mode in order to share the networking namespace of the host with the module itself.
This is sacrificing sandboxing for easier configuration.
NOTE: This is a beta feature for Docker desktop (if you're using that), and must be enabled explicitly:
The host networking driver only works on Linux hosts, but is available as a Beta feature, on Docker Desktop version 4.29 and later.
Might not be the best long-term decision, but for simplicity's sake it's closer to native; just like the alternative (previous) approach of launching modules as subprocesses.
|
|
||
| [[modules]] | ||
| id = "DA_COMMIT" | ||
| docker_image="da_commit" |
There was a problem hiding this comment.
That's the example image that must be available in the local registry before starting commit-boost.
You can find the example source code in this repo.
NOTE: Whilst we're still in development, you will have to modify the Cargo.toml of the example module to point to a correct source for the cb-... dependencies.
| let config = CommitBoostConfig::from_file(&config_path); | ||
|
|
||
| // Initialize Docker client | ||
| let docker = bollard::Docker::connect_with_local_defaults().expect("Failed to connect to Docker"); |
There was a problem hiding this comment.
Here, we currently expect that CB and the docker engine run on the same machine.
However, the way we connect to the docker daemon is modifiable if more complex needs arise: ref.
|
Looks good ! Now that we have docker images I would if we should keep the two different sources (Docker and binary)? Would rather remove the binary one and add it back once we're ready to support it again. |
|
There are currently no new issues introduced with the support of binary sourced modules. In fact, the current You might be right, though, that we might run into difficulties at some point... so I guess we can then remove the binary option. Does that change perspectives or do we stick to removing binary sources right away? |
|
Yep I would remove it for now, would rather keep the codebase simple and lean and add it back once we can support it smoothly |
Adds the ability to also launch a module by creating and starting a container from a docker image provided for that module.
The
config.example.tomlis modified to launch an example replica ofda_commitwhich has been dockerized. From the viewpoint of CB, it simply expects a docker imageda_commitin the local registry. However, the dockerized version of the example moduleda_commitis not defined in this repo, but is available here, to more clearly showcase the boundaries between the two different perspectives: CB operator vs. module author.Nothing has been changed in the business logic of
da_commit, only theCargo.tomlhas been modified to decouple dependencies forcommit-boost's workspace and aDockerfilehas been added.Before you launch CB, you need to:
da_commitimage locally.IMPORTANT: While this feature of CB is in development, you might have to modify
Cargo.tomlof the example module to point to the accurate current version of CB before you build the image. At the time of writing, the referred branch is a working version, but things might become incompatible when commits are actively added to the branch.NOTE: #3 (comment)