Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNS/SQS contention #3863

Merged
merged 3 commits into from
Nov 21, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,16 @@ aliases:

## Component format

To set up AWS SNS/SQS pub/sub, create a component of type `pubsub.aws.snssqs`.
To set up AWS SNS/SQS pub/sub, create a component of type `pubsub.aws.snssqs`.

By default, the AWS SNS/SQS component:

- Generates the SNS topics
- Provisions the SQS queues
- Configures a subscription of the queues to the topics

{{% alert title="Note" color="primary" %}}
If you only have a publisher and no subscriber, only the SNS topics are created.
If you only have a publisher and no subscriber, only the SNS topics are created.

However, if you have a subscriber, SNS, SQS, and the dynamic or static subscription thereof are generated.
{{% /alert %}}
Expand Down Expand Up @@ -133,6 +134,22 @@ When configuring the PubSub component with SQS dead-letter queues, the metadata
When running the Dapr sidecar (`daprd`) with your application on EKS (AWS Kubernetes) node/pod already attached to an IAM policy defining access to AWS resources, you **must not** provide AWS access-key, secret-key, and tokens in the definition of the component spec.
{{% /alert %}}

#### SNS/SQS Contention with Dapr

Fundamentally, SNS aggregates messages from multiple publisher topics into a single SQS queue by creating SQS subscriptions to those topics. As a subscriber, the SNS/SQS pub/sub component consumes messages from that sole SQS queue.

However, like any SQS consumer, the component cannot selectively retrieve the messages published to the SNS topics to which it is specifically subscribed. This can result in the component receiving messages originating from topics without associated handlers. Typically, this occurs during:

- **Component initialization:** If infrastructure subscriptions are ready before component subscription handlers, or
- **Shutdown:** If component handlers are removed before infrastructure subscriptions.

Since this issue affects any SQS consumer of multiple SNS topics, the component cannot prevent consuming messages from topics lacking handlers. When this happens, the component logs an error indicating such messages were erroneously retrieved.

In these situations, the unhandled messages would reappear in SQS with their [receive count](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html#sqs-receive-count) decremented after each pull. Thus, there is a risk that an unhandled message could exceed its `messageReceiveLimit` and be lost.

{{% alert title="Important" color="warning" %}}
Consider potential contention scenarios when using SNS/SQS with Dapr, and configure `messageReceiveLimit` appropriately. It is highly recommended to use SQS dead-letter queues by setting `sqsDeadLettersQueueName` to prevent losing messages.
{{% /alert %}}

## Create an SNS/SQS instance

Expand Down
Loading