Building a system that resembles Kafka. Producers shall be able to send data to the cluster, while consumers can read it. It is for educational purposes, but who knows?
- Log
- Partition / Topic / Replication
- Partition connectivity / Raft Group
- Broker / Cluster Raft Group
- Broker GRPC Server
Came to the conclusion that relying purely on gossip for metadata management can lead to inconsistencies and failures.
let's follow the traditional controller-based kafka architecture.
- Trivial to install
- Ease to use
- Ease to operate (need metrics)