Data Retention Regulator Change Log

0.10.0 (2024-11-01)

• Fix a bug where RecordingMetadata would be incorrectly populated (introduced in 0.7.0):
  • maxRecordedPosition would be correct only for stopped recordings, -1 for active ones
  • startTimestamp would be the maxRecordedPosition
  • stopTimestamp would be the startTimestamp Which would have the following effects:
  • RetainLengthPolicy would always retain all active recordings
  • DeleteSegmentsAfterPeriodPolicy would not delete any segments for active recordings
  • DeleteRecordingsAfterPeriodPolicy would delete after period calculated from recording start instead of stop, unless recording is active, then it would fail to delete
• Upgrade Aeron to 1.46.7.
• Upgrade Agrona to 1.23.1.

0.9.0 (2024-08-30)

Breaking changes

• Java 17 or newer is now required.

Changelog

• JavaDoc jars are now published.
• Upgrade to Aeron 1.46.0
• Upgrade to Agrona 1.23.0

0.8.0 (2024-08-30)

• Upgrade to Aeron 1.45.0
• Upgrade to Agrona 1.22.0

0.7.0 (2024-08-29)

Non-breaking changes

• Make testing policy trees easier in downstream projects without extensive mocking:
  • Allow creation of SegmentFileBoundary
  • Allow creation of RecordingMetadata
• Make dry run output more precise by avoiding the calculation of new start positions before the existing start position in the built-in policies.

0.6.0 (2024-08-14)

Non-breaking changes

• Introduced a new composite policy:
  • RetainMinOfPolicy: combines sub-policies by retaining the minimum of its inner policies.
• Fixed a bug where DeleteRecordingsAfterPeriodPolicy gave the wrong retention instruction for live recordings. Thankfully, the purge would always fail, as the recording was live. A test has been added to prevent regressions.

0.5.0 (2024-08-01)

Non-breaking changes

• Introduce a "dry run" mode, where the tool does not delete any data, but logs what it would have done. This can be useful for testing policies.

• Change the default controlResponseStreamId we use from 20 to 20240712 to reduce the chances of collisions with other AeronArchive clients.

0.4.0 (2024-07-24)

Non-breaking changes

• Make it easier to configure the controlReponseStreamId on the AeronArchive.Context that the DRR uses.
• Avoid attempts to detach partial (i.e., not complete) segment files in DeleteSegmentsAfterPeriodPolicy if they have "expired".

0.3.0 (2024-07-19)

Breaking changes

• Renamed DeleteAfterPeriodPolicy to the more-specific DeleteRecordingsAfterPeriodPolicy

Non-breaking changes

• Introduced new policies:
  • DeleteSegmentsAfterPeriodPolicy: detaches and deletes recording segment files that have not been modified for some period of time.
  • MatchingLogRecordingPolicy: applies an inner policy to the Consensus Module's log recording.

0.2.0 (2024-07-19)

Noteworthy

Initial release with the following built-in data retention policies:

• Leaf policies:
  • RetainEnoughDataForRecoveryPolicy: keeps the last N snapshots and the log following the earliest of these snapshots available for cluster node recovery.
  • RetainLengthPolicy: keeps at least the last N bytes of data. It will detach and delete recording segments before the last N bytes.
  • DeleteAfterPeriodPolicy: deletes recordings that stopped some period of time ago.
• Composite policies:
  • MatchingStreamsPolicy: applies an inner policy to recordings that match a channel URI and/or stream identifier.
  • RetainMaxOfPolicy: safely combines policies, e.g., to retain X snapshots worth of cluster data and Y bytes of data per recording, by retaining the maximum of its inner policies.
  • CascadePolicy: accepts a sequence of policies. It will try each policy in turn and apply the first policy that "covers" a recording, i.e., that has an opinion on what the extent of the recording should be.