pip-350.md

PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis

Background knowledge

https://pulsar.apache.org/docs/3.2.x/tiered-storage-overview/ Pulsar provides the ability to offload the data from bookkeeper to the cloud storage with the tiered storage. Once the data is offloaded to the cloud storage, the data in the bookkeeper can be deleted after a certain period of time. We use the managedLedgerOffloadDeletionLagInMillis to control the deletion lag time for the offloaded data. The default value of managedLedgerOffloadDeletionLagInMillis is 4 hours. It means the offloaded data will be deleted after 4 hours by default.

Motivation

In some test scenarios, we want to disable the deletionLag and never delete the data from the bookkeeper. Then when the tiered storage data is broken, we can still read the data from the bookkeeper.

Goals

In Scope

Never deletes the bookkeeper data when the managedLedgerOffloadDeletionLagInMillis is set to -1.

Detailed Design

Design & Implementation Details

Only need to check the value of managedLedgerOffloadDeletionLagInMillis in the ManagedLedgerImpl when it is going to delete the bookkeeper data.

pulsar/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java

Line 2579 in 774a5d4

return offloadPolicies.filter(policies -> offload.getComplete() && !offload.getBookkeeperDeleted()

Backward & Forward Compatibility

Fully compatible.

Links

• Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9
• Mailing List voting thread: https://lists.apache.org/thread/c3rh530dlwo6nhrdflpw0mjck85hhfbx