DP-408: Create a dead-letter queue for each regular queue

[jakzal]

Once the message was delivered 10 times being explicitly removed, it will be move to the dead-letter queue.

Example log from when the queues are created:

co-cdp-localstack  | Creating the ev-outbound queue...
co-cdp-localstack  | 2024-08-02T08:03:36.157 DEBUG --- [et.reactor-0] l.services.sqs.provider    : creating queue key=ev-outbound-deadletter attributes=None tags=None
co-cdp-localstack  | 2024-08-02T08:03:36.157  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.CreateQueue => 200
co-cdp-localstack  | 2024-08-02T08:03:36.354  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.GetQueueAttributes => 200
co-cdp-localstack  | 2024-08-02T08:03:36.547 DEBUG --- [et.reactor-0] l.services.sqs.provider    : creating queue key=ev-outbound attributes={'RedrivePolicy': '{"deadLetterTargetArn":"arn:aws:sqs:us-east-1:000000000000:ev-outbound-deadl
etter","maxReceiveCount":"10"}'} tags=None
co-cdp-localstack  | 2024-08-02T08:03:36.547  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.CreateQueue => 200
co-cdp-localstack  | 2024-08-02T08:03:36.742  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.GetQueueAttributes => 200
co-cdp-localstack  | Created the ev-outbound queue
co-cdp-localstack  | {
co-cdp-localstack  |   "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/ev-outbound",
co-cdp-localstack  |   "QueueArn": "arn:aws:sqs:us-east-1:000000000000:ev-outbound",
co-cdp-localstack  |   "DeadLetterQueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/ev-outbound-deadletter"
co-cdp-localstack  |   "DeadLetterQueueArn": "arn:aws:sqs:us-east-1:000000000000:ev-outbound-deadletter"
co-cdp-localstack  | }
co-cdp-localstack  | 2024-08-02T08:03:36.942  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.GetQueueAttributes => 200
co-cdp-localstack  | {
co-cdp-localstack  |     "Attributes": {
co-cdp-localstack  |         "ApproximateNumberOfMessages": "0",
co-cdp-localstack  |         "ApproximateNumberOfMessagesNotVisible": "0",
co-cdp-localstack  |         "ApproximateNumberOfMessagesDelayed": "0",
co-cdp-localstack  |         "CreatedTimestamp": "1722585816",
co-cdp-localstack  |         "DelaySeconds": "0",
co-cdp-localstack  |         "LastModifiedTimestamp": "1722585816",
co-cdp-localstack  |         "MaximumMessageSize": "262144",
co-cdp-localstack  |         "MessageRetentionPeriod": "345600",
co-cdp-localstack  |         "QueueArn": "arn:aws:sqs:us-east-1:000000000000:ev-outbound",
co-cdp-localstack  |         "ReceiveMessageWaitTimeSeconds": "0",
co-cdp-localstack  |         "VisibilityTimeout": "30",
co-cdp-localstack  |         "SqsManagedSseEnabled": "true",
co-cdp-localstack  |         "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:000000000000:ev-outbound-deadletter\",\"maxReceiveCount\":\"10\"}"
co-cdp-localstack  |     }
co-cdp-localstack  | }
co-cdp-localstack  | 2024-08-02T08:03:37.137  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.GetQueueAttributes => 200
co-cdp-localstack  | {
co-cdp-localstack  |     "Attributes": {
co-cdp-localstack  |         "ApproximateNumberOfMessages": "0",
co-cdp-localstack  |         "ApproximateNumberOfMessagesNotVisible": "0",
co-cdp-localstack  |         "ApproximateNumberOfMessagesDelayed": "0",
co-cdp-localstack  |         "CreatedTimestamp": "1722585816",
co-cdp-localstack  |         "DelaySeconds": "0",
co-cdp-localstack  |         "LastModifiedTimestamp": "1722585816",
co-cdp-localstack  |         "MaximumMessageSize": "262144",
co-cdp-localstack  |         "MessageRetentionPeriod": "345600",
co-cdp-localstack  |         "QueueArn": "arn:aws:sqs:us-east-1:000000000000:ev-outbound-deadletter",
co-cdp-localstack  |         "ReceiveMessageWaitTimeSeconds": "0",
co-cdp-localstack  |         "VisibilityTimeout": "30",
co-cdp-localstack  |         "SqsManagedSseEnabled": "true"
co-cdp-localstack  |     }
co-cdp-localstack  | }

Example log from when the message is moved to DLQ:

co-cdp-localstack  | 2024-08-02T08:04:11.271 DEBUG --- [et.reactor-0] l.services.sqs.models      : de-queued message SqsMessage(id=b38c3a6c-2d24-425e-b166-cff68b3d6f18,group=None) from arn:aws:sqs:us-east-1:000000000000:ev-inbound
co-cdp-localstack  | 2024-08-02T08:04:11.272  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.ReceiveMessage => 200
co-cdp-localstack  | 2024-08-02T08:04:12.504 DEBUG --- [et.reactor-0] l.services.sqs.models      : de-queued message SqsMessage(id=b38c3a6c-2d24-425e-b166-cff68b3d6f18,group=None) from arn:aws:sqs:us-east-1:000000000000:ev-inbound
co-cdp-localstack  | 2024-08-02T08:04:12.505  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.ReceiveMessage => 200
co-cdp-localstack  | 2024-08-02T08:04:13.586 DEBUG --- [et.reactor-0] l.services.sqs.models      : de-queued message SqsMessage(id=b38c3a6c-2d24-425e-b166-cff68b3d6f18,group=None) from arn:aws:sqs:us-east-1:000000000000:ev-inbound
co-cdp-localstack  | 2024-08-02T08:04:13.587  INFO --- [et.reactor-0] localstack.request.aws     : AWS sqs.ReceiveMessage => 200
co-cdp-localstack  | 2024-08-02T08:04:14.780 DEBUG --- [et.reactor-0] l.services.sqs.models      : de-queued message SqsMessage(id=b38c3a6c-2d24-425e-b166-cff68b3d6f18,group=None) from arn:aws:sqs:us-east-1:000000000000:ev-inbound
co-cdp-localstack  | 2024-08-02T08:04:14.780 DEBUG --- [et.reactor-0] l.services.sqs.models      : message SqsMessage(id=b38c3a6c-2d24-425e-b166-cff68b3d6f18,group=None) has been received 11 times, marking it for DLQ