ex_aws_msk_iam_auth
is an authentication plugin for broadway_kafka. It enables Broadway Kafka clients to authenticate with Amazon's Managed Streaming for Apache Kafka(Amazon MSK) via AWS_MSK_IAM SASL mechanism.
Add the following dependency to your mix.exs
def deps do
[
{:ex_aws_msk_iam_auth, git: "https://github.com/BigThinkcode/ex_aws_msk_iam_auth"}
]
end
Broadway Kafka supports connecting to Kafka broker via SASL authentication. The following sample configuration shows how ex_aws_msk_iam_auth
plugin can be used with it.
Ref: https://hexdocs.pm/broadway_kafka/BroadwayKafka.Producer.html#module-client-config-options
client_config: [
sasl:
{
:callback,
ExAwsMskIamAuth,
{:AWS_MSK_IAM, "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY"}
},
ssl: true
]
Broadway Kafka is a Kafka Connector for Broadway - an Elixir library to build concurrent, multi-stage data ingestion/processing pipelines with Elixir.
Broadway Kafka is an amalgamation of awesome features from Broadway with Kafka as a producer. Internally, it uses brod as its Kafka client acting as a wrapper. Brod supports SASL PLAIN
, SCRAM-SHA-256
and SCRAM-SHA-512
authentication mechanisms out of the box and also offers extension points to support custom authentication plugins.
MSK supports two variants - MSK Fully Managed and MSK Serverless. In both the variants, Kafka service can be protected via SASL, in particular, AWS's custom SASL mechanism AWS_MSK_IAM(https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html). At the time of writing this library, MSK's Serverless variant's only supported authentication was AWS_MSK_IAM SASL mechanism.
This library takes inspiration from its Java counterpart aws-msk-iam-auth