Outbox cleaning using Time-To-Live does not work

[DavidBoike]

Symptoms

Outbox records for dispatched messages are not evicted.

Who's affected

Anyone using Cosmos DB persistence with outbox feature enabled.

Root cause

CosmosDB doesn't have a normal "outbox cleaner" but uses a TTL feature so that the record is automatically deleted after the TTL is complete.

Except according to the docs linked above, if the container TTL is set to null then it doesn't matter what the per-item TTL is set to—the item will never expire.

The container is created here and does not include setting a TTL, so outbox records will never expire.

Containers should be created with a default TTL of -1 which still means "no expiry" but will allow per-item TTLs to work correctly.

Workaround

Manually update the TTL of the container to On (no default).

Plan of action

• Fix the issue
• Release the fix

[danielmarbach]

The problem is we cannot really take the outbox TTL into account because the setting might be changed which then would change the table TTL which is fairly problematic. In addition we are only creating the default shared table which also contains other data. The outbox TTL is supposed to influence outbox records added the time of setting as dispatched. So the sensible way to address this would be to set the default TTL on the created table to -1 instead of null, like @DavidBoike mentioned. The item-based TTLs actually have an effect

If TTL on a container is set to -1, an item in this container that has the time to live set to n, will expire after n seconds, and remaining items will not expire.

https://docs.microsoft.com/en-us/azure/cosmos-db/time-to-live#time-to-live-configurations

[danielmarbach]

Additionally, we are currently using

CreateContainerIfNotExistsAsync which only uses the container ID to check if it exists but does not compare the container properties. This approach was chosen to allow container properties to diverge after creation. It doesn't really make sense to query the properties and update it. I would suggest making sure new versions create the container if it doesn't exist with the container TTL set to -1 but not attempt to query and conditionally update containers because that could lead into disasters when customers deliberately override certain table values after creation.

[danielmarbach]

Here is a suggestion #128

[SeanFeldman]

Version https://github.com/Particular/NServiceBus.Persistence.CosmosDB/releases/tag/0.2.1 with the fix was released