Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support running Bacalhau in Docker compose #4596

Merged
merged 6 commits into from
Oct 9, 2024

Conversation

jamlo
Copy link
Contributor

@jamlo jamlo commented Oct 7, 2024

This pull requests addresses issue #4595

jamlo added 3 commits October 7, 2024 09:07
This initial setup addresses the need to run Bacalhau inside containers
while also supporting Docker workloads within these containers
(using Docker-in-Docker, or DinD).

Notes:
1- This setup supports minio and running S3 compatible jobs. Minio is
deployed in it own container

2- This supports local image registry to eliminate the need to use
DockerHub for testing. We can experiment with images freely in our local
environments and CI.

3- This is a full and read deployment of Bacalhau using Docker-compose.
It will allows us to replicate production setup easily.

4- It can be used in our CIs when needed

5- It can act as an entry point for new users to experiment with
Bacalhau

TODO;
1- Use https://github.com/nestybox/sysbox
2- Think about Fuzz Tests
3- IPFS support
4- Think about how to increase the number or requester nodes and compute
nodes when needed to test different scenarios.
5- Check the DevStack and see how can we integrate with it.
@jamlo jamlo linked an issue Oct 7, 2024 that may be closed by this pull request
@jamlo jamlo requested review from wdbaruni, frrist and udsamani October 7, 2024 18:52
Copy link
Member

@wdbaruni wdbaruni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comments

test-integration/Dockerfile-ClientNode Show resolved Hide resolved
test-integration/compute_node_image_setup.sh Outdated Show resolved Hide resolved
test-integration/compute_node_image_setup.sh Outdated Show resolved Hide resolved
@jamlo jamlo requested a review from udsamani October 8, 2024 15:58
test-integration/docker-compose.yml Show resolved Hide resolved
test-integration/docker-compose.yml Show resolved Hide resolved
test-integration/docker-compose.yml Show resolved Hide resolved
test-integration/compute_node_image_setup.sh Show resolved Hide resolved
@frrist
Copy link
Member

frrist commented Oct 8, 2024

One more thing that comes to mind: https://github.com/bacalhau-project/bacalhau/tree/main/ops/metrics
We have a docker-compose based metrics suite for bacalhau nodes - it spins up grafana, prometheus, jaeger, and an otel collector. In the future it may be worth unifying these together.

@jamlo jamlo merged commit 54e8e07 into main Oct 9, 2024
3 of 4 checks passed
@jamlo jamlo deleted the jamlo/bacalhau-in-docker-compose branch October 9, 2024 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Deploy Bacalhau using Docker Compose
4 participants