From 5adcac925ab798583235ac003cd7a8c3d5a9b2a3 Mon Sep 17 00:00:00 2001 From: Amit Mor Date: Sun, 12 Nov 2023 21:48:17 +0200 Subject: [PATCH 1/3] SNS/SQS contention Signed-off-by: Amit Mor Signed-off-by: Amit Mor --- .../components-reference/supported-pubsub/setup-aws-snssqs.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md index 61b68290196..0d316d5aed2 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md @@ -133,6 +133,9 @@ When configuring the PubSub component with SQS dead-letter queues, the metadata When running the Dapr sidecar (`daprd`) with your application on EKS (AWS Kubernetes) node/pod already attached to an IAM policy defining access to AWS resources, you **must not** provide AWS access-key, secret-key, and tokens in the definition of the component spec. {{% /alert %}} +#### SNS/SQS Contention with Dapr + +Fundamentally, SNS aggregates messages from multiple publisher topics into a single SQS queue by creating SQS subscriptions to those topics. As a subscriber, the SNS/SQS PubSub component consumes messages from that sole SQS queue. However, like any SQS consumer, the component cannot selectively retrieve only messages published to specific SNS topics it is configured to subscribe to. This can result in the component receiving messages originating from topics without associated handlers. Typically this occurs during component initialization, if infrastructure subscriptions are ready before component subscription handlers, or during shutdown, if component handlers are removed before infrastructure subscriptions. Since this issue affects any SQS consumer of multiple SNS topics, the component cannot prevent consuming messages from topics lacking handlers. When this happens, the component logs an error indicating such messages were erroneously retrieved. ## Create an SNS/SQS instance From 58570f137ce1c5aecfb0e49fd1100cf4cfdfc185 Mon Sep 17 00:00:00 2001 From: Amit Mor Date: Mon, 13 Nov 2023 10:45:28 +0200 Subject: [PATCH 2/3] added recommendation Signed-off-by: Amit Mor Signed-off-by: Amit Mor --- .../supported-pubsub/setup-aws-snssqs.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md index 0d316d5aed2..207b42fdaca 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md @@ -9,15 +9,16 @@ aliases: ## Component format -To set up AWS SNS/SQS pub/sub, create a component of type `pubsub.aws.snssqs`. +To set up AWS SNS/SQS pub/sub, create a component of type `pubsub.aws.snssqs`. By default, the AWS SNS/SQS component: + - Generates the SNS topics - Provisions the SQS queues - Configures a subscription of the queues to the topics {{% alert title="Note" color="primary" %}} -If you only have a publisher and no subscriber, only the SNS topics are created. +If you only have a publisher and no subscriber, only the SNS topics are created. However, if you have a subscriber, SNS, SQS, and the dynamic or static subscription thereof are generated. {{% /alert %}} @@ -136,6 +137,11 @@ When running the Dapr sidecar (`daprd`) with your application on EKS (AWS Kubern #### SNS/SQS Contention with Dapr Fundamentally, SNS aggregates messages from multiple publisher topics into a single SQS queue by creating SQS subscriptions to those topics. As a subscriber, the SNS/SQS PubSub component consumes messages from that sole SQS queue. However, like any SQS consumer, the component cannot selectively retrieve only messages published to specific SNS topics it is configured to subscribe to. This can result in the component receiving messages originating from topics without associated handlers. Typically this occurs during component initialization, if infrastructure subscriptions are ready before component subscription handlers, or during shutdown, if component handlers are removed before infrastructure subscriptions. Since this issue affects any SQS consumer of multiple SNS topics, the component cannot prevent consuming messages from topics lacking handlers. When this happens, the component logs an error indicating such messages were erroneously retrieved. +In these situations, the unhandled messages would reappear in SQS with their [receive count](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html#sqs-receive-count) decremented after each pull. Thus, there is a risk that an unhandled message could exceed its `messageReceiveLimit` and be lost. + +{{% alert title="Important" color="warning" %}} +It is therefore advisable to consider potential contention scenarios when using SNS/SQS with Dapr, and configure `messageReceiveLimit` appropriately. It is also highly recommended to use SQS dead-letter queues by setting `sqsDeadLettersQueueName` to prevent losing messages. +{{% /alert %}} ## Create an SNS/SQS instance From 6a9ccf472911e19fa294772040207eeafee0b471 Mon Sep 17 00:00:00 2001 From: Amit Mor Date: Tue, 14 Nov 2023 23:45:15 +0200 Subject: [PATCH 3/3] suggested edits Signed-off-by: Amit Mor Signed-off-by: Amit Mor --- .../supported-pubsub/setup-aws-snssqs.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md index 207b42fdaca..6ccddb05234 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-aws-snssqs.md @@ -136,11 +136,19 @@ When running the Dapr sidecar (`daprd`) with your application on EKS (AWS Kubern #### SNS/SQS Contention with Dapr -Fundamentally, SNS aggregates messages from multiple publisher topics into a single SQS queue by creating SQS subscriptions to those topics. As a subscriber, the SNS/SQS PubSub component consumes messages from that sole SQS queue. However, like any SQS consumer, the component cannot selectively retrieve only messages published to specific SNS topics it is configured to subscribe to. This can result in the component receiving messages originating from topics without associated handlers. Typically this occurs during component initialization, if infrastructure subscriptions are ready before component subscription handlers, or during shutdown, if component handlers are removed before infrastructure subscriptions. Since this issue affects any SQS consumer of multiple SNS topics, the component cannot prevent consuming messages from topics lacking handlers. When this happens, the component logs an error indicating such messages were erroneously retrieved. +Fundamentally, SNS aggregates messages from multiple publisher topics into a single SQS queue by creating SQS subscriptions to those topics. As a subscriber, the SNS/SQS pub/sub component consumes messages from that sole SQS queue. + +However, like any SQS consumer, the component cannot selectively retrieve the messages published to the SNS topics to which it is specifically subscribed. This can result in the component receiving messages originating from topics without associated handlers. Typically, this occurs during: + +- **Component initialization:** If infrastructure subscriptions are ready before component subscription handlers, or +- **Shutdown:** If component handlers are removed before infrastructure subscriptions. + +Since this issue affects any SQS consumer of multiple SNS topics, the component cannot prevent consuming messages from topics lacking handlers. When this happens, the component logs an error indicating such messages were erroneously retrieved. + In these situations, the unhandled messages would reappear in SQS with their [receive count](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html#sqs-receive-count) decremented after each pull. Thus, there is a risk that an unhandled message could exceed its `messageReceiveLimit` and be lost. {{% alert title="Important" color="warning" %}} -It is therefore advisable to consider potential contention scenarios when using SNS/SQS with Dapr, and configure `messageReceiveLimit` appropriately. It is also highly recommended to use SQS dead-letter queues by setting `sqsDeadLettersQueueName` to prevent losing messages. +Consider potential contention scenarios when using SNS/SQS with Dapr, and configure `messageReceiveLimit` appropriately. It is highly recommended to use SQS dead-letter queues by setting `sqsDeadLettersQueueName` to prevent losing messages. {{% /alert %}} ## Create an SNS/SQS instance