Access SST full file checksum via RocksDB#getLiveFilesMetadata

[npepinpe]

Description

This PR passes along the native LiveFileMetaData#file_checksum field from the C++ class to the Java API as a copied byte array. If there is no file checksum generator factory set beforehand, then the array will empty. Please advise if you'd rather it be null - an empty array means one extra allocation, but it avoids possible null pointer exceptions.

Note
This functionality complements but does not supersede #11736

It's outside the scope here to add support for Java based FileChecksumGenFactory implementations. As a workaround, users can already use the built-in one by creating their initial DBOptions via properties:

final Properties props = new Properties();
props.put("file_checksum_gen_factory", "FileChecksumGenCrc32cFactory");

try (final DBOptions dbOptions = DBOptions.getDBOptionsFromProps(props);
     final ColumnFamilyOptions cfOptions = new ColumnFamilyOptions();
     final Options options = new Options(dbOptions, cfOptions).setCreateIfMissing(true)) {
// do stuff
}

I wanted to add a better test, but unfortunately there's no available CRC32C implementation available in Java 8 without adding a dependency or adding a JNI helper for RocksDB's own implementation (or bumping the minimum version for tests to Java 9). That said, I understand the test is rather poor, so happy to change it to whatever you'd like.

Context

To give some context, we replicate RocksDB checkpoints to other nodes. Part of this is verifying the integrity of each file during replication. With a large enough RocksDB, computing the checksum ourselves is prohibitively expensive. Since SST files comprise the bulk of the data, we'd much rather delegate this to RocksDB on file write, and read it back after to compare.

It's likely we will provide a follow up to read the file checksum list directly from the manifest without having to open the DB, but this was the easiest first step to get it working for us.

[npepinpe]

A ldb test failed; it's unclear to me how this is related to my changes. Let me know if there's anything I can do!

[ChrisKujawa]

@adamretter or @ajkr due you have time to review or give feedback on this and #11772?

[facebook-github-bot]

@ajkr has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[ajkr]

LGTM

[facebook-github-bot]

@ajkr merged this pull request in 5b073a7.