Add compaction to lmdb_benchmark

[cberner]

Note: compact() is currently too slow to practically run with the default 1M elements. It takes 43secs on my computer when set to 1000 elements

[Kerollmops]

Hey 👋

As I understand correctly, you are compacting the redb database but not the other databases just before measuring the disk size?

It feel unfair to me. Doesn't RocksDB and Sled support compaction, too?

This kind of benchmark is only ran once every now and then, right? Can't we display the very long compaction time of LMDB (and others) with the associated size. Another solution could be to display both, before and after compaction.

Have a nice end of the year 🎄

EDIT: Only redb, RocksDB and LMDB support compactions.

[cberner]

Can you point me to the method to call to compact the other dbs? I looked but didn't find one

…

Sent from my phone

On Thu, Dec 26, 2024, 3:37 AM Clément Renault ***@***.***> wrote: Hey 👋 As I understand correctly, you are compacting the redb database but not the other databases just before measuring the disk size? It feel unfair to me. Doesn't RocksDB and Sled support compaction, too? This kind of benchmark is only ran once every now and then, right? Can't we display the very long compaction time of LMDB (and others) with the associated size. Another solution could be to display both, before and after compaction. Have a nice end of the year 🎄 — Reply to this email directly, view it on GitHub <#853 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGNXQF6WTX7N7GSEEFZHS32HPS7TAVCNFSM6AAAAABNHLL62KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRSGUZDIMRUGM> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[Kerollmops]

For sure 😌

You can use the heed Env::copy_to_file/fd method to write the content of the database into another file or fd. Note that you then need to reopen the destination file to read the content and, probably open it with the NO_SUB_DIR option.

When it comes to RocksDB, calling the DBCommon::compact_range/_opt method on every database should be enough to compact everything. I don't know if it is blocking but you should be able to decide about that with some options.