IBM/sarama Go client library for Apache Kafka support#68

Merged

matskramer merged 51 commits intomainfrom mk_sarama

Oct 8, 2024

+77,956-499

Commits on Oct 2, 2024

UI: Added UI html and js for upload. Endpoint is not yet there.
matskramer
committed
UI: First version of backend to upload using UI
matskramer
committed
UI: Using gin to bind to JSON
matskramer
committed
UI: Added a TODO to remove code if custom bind functions don't gets fixed
matskramer
committed
Added debug ability for a service in GoLand running in docker started using docker-compose (the only service currently supported is UI, but more services can be added later)
matskramer
committed
UI: renamed an import alias
matskramer
committed
Kafka: first simple config of Kafka with kRaft for docker-compose
matskramer
committed
Kafka: git ignores files in volumes kraft-data
matskramer
committed
Kafka: First sending of message to topic-mock-next in kafka (prototype to make it work locally)
matskramer
committed
Kafka: added som debug printf values
matskramer
committed
Kafka: improved kafka config
matskramer
committed
Kafka: gitignore
matskramer
committed
Kafka: gitignore
matskramer
committed
Kafka: finally a correct gitignore pattern
matskramer
committed
Kafka: First working prototype of consuming from a Kafka topic
matskramer
committed
Kafka: Improved prototype of consuming from a Kafka topic
matskramer
committed
Kafka: First working prototype flow: web ->UI->Kafka->Mockas->Upload
matskramer
committed
Kafka: Two kafka brokers for same topic (first working prototype)
matskramer
committed
Kafka: added some prototype usage of headers and metadata in a Kafka message
matskramer
committed
Kafka: improved some logging and error messages
matskramer
committed
Kafka: messaging consumption using throttling with exponential backoff. UI for kafka now requires a username and password to access.
matskramer
committed
Kafka: just added some TODOs
matskramer
committed
Kafka: Configuration now controls if Kafka should be used or not
matskramer
committed
Kafka: Improved logging and some smaller refactoring (names)
matskramer
committed
Kafka: Added endpoint health (status) to MockAS API
matskramer
committed
Kafka: Two brokers with replication and two mockas service instances consuming from same topic. Brokers is set by configuration to producers and consumers. Rename of default kafka cluster.
matskramer
committed
Kafka: Common kafka message producer to be used by services who needs to publish to one or more kafka topics
matskramer
committed
Kafka: First prototype version of common kafka message consumer to be used by services who needs to consume from one or more kafka topics
matskramer
committed
Kafka: Common kafka message producer
matskramer
committed
Kafka: Concept for: Produce and consume messages in chains. Producer and consumer in same service. Two different consumer groups for same topic in different services.
matskramer
committed
Kafka: Improved logging and error handling
matskramer
committed
Kafka: Improved logging and error handling
matskramer
committed
Kafka: kafka client created in main for ui
matskramer
committed
Kafka: user UID and GID that is running Kafka is set using a ".env" file. The ".env" file is excluded from git but there is a ".env.example" file commited
matskramer
committed
Removed .env from version controll - NO SENSITIVE DATA EXISTED IN THE FILE
matskramer
committed
Kafka: More .env properties possible to set for kafka. Updated docker-compose file for debug in Goland.
matskramer
committed
Kafka: EventPublisher is the generic interface used to publish events (implementation can then be different frameworks: Kafka, Pulsar, etc)
matskramer
committed
Kafka: EventConsumer is the generic interface used to consume events (implementation can then be different frameworks: Kafka, Pulsar, etc)
matskramer
committed
Kafka: Newer versions of sarama and grpc and their dependencies
matskramer
committed
Kafka: removed .env file, all config values are now back in docker-compose.yaml
matskramer
committed
Kafka: commit after rebase, services starts but api not refactored to 2.8 for ui and kafka message handling.
matskramer
committed
UI: apiv28 portal->document/list
matskramer
committed
Kafka and UI: supports apiv28
matskramer
committed
Kafka: fixed some minor TODOs
matskramer
committed
Diagram over sync and async upload flow.

masv3971
authored and
matskramer
committed
fmt.

masv3971
authored and
matskramer
committed
Fixed some of the review comments -starting in docker and works after some minor tests
matskramer
committed
Fixed some of the review comments - inbound/outbound for kafka
matskramer
committed
Go Remote debugging of service ui in JetBrains Goland now works againg, se docker-compose.yaml and "ui" for what ip to use (172.16.50.50:40000 is currently used)
matskramer
committed

Commits on Oct 4, 2024

Refactorings after review nr 2
matskramer
committed

Commits on Oct 8, 2024

Refactorings after review nr 3
matskramer
committed