Add count function

[mocchira]

In order to make record counting as fast as possible by reducing the number of interactions between erlang runtime and NIF for leo-project/leofs#974.

[mocchira]

Note: set_verify_prop test must fail like below

module 'bucket_expiry'
  bucket_expiry: set_remove_pid...ok
  bucket_expiry: set_verify_prop...*failed*
in function bucket_expiry:'-set_verify_prop/1-fun-5-'/0 (test/bucket_expiry.erl, line 106)
**error:{badmatch,{error,{einval,"eleveldb::property_cache cache.Insert() failed"}}}
  output:<<"">>

  [done in 0.007 s]
module 'iterators'
  iterators: prev_test_case...ok
  iterators: seek_and_next_test_case...ok
  iterators: basic_prefetch_test_case...[0.006 s] ok
  iterators: seek_and_prefetch_test_case...ok
  iterators: aae_prefetch1...[0.002 s] ok
  iterators: aae_prefetch2...[0.002 s] ok
  iterators: aae_prefetch3...[0.002 s] ok
  [done in 0.035 s]
module 'eleveldb'
  eleveldb: eleveldb_test_ (test_open)...[0.008 s] ok
  eleveldb: eleveldb_test_ (test_close)...[0.008 s] ok
  eleveldb: eleveldb_test_ (test_destroy)...[0.015 s] ok
  eleveldb: eleveldb_test_ (test_fold)...[0.010 s] ok
  eleveldb: eleveldb_test_ (test_fold_keys)...[0.008 s] ok
  eleveldb: eleveldb_test_ (test_fold_from_key)...[0.010 s] ok
  eleveldb: eleveldb_test_ (test_close_fold)...[0.013 s] ok
  eleveldb: eleveldb_test_ (test_compression)...[0.031 s] ok
  eleveldb: eleveldb_test_ (test_open_many(9))...[0.072 s] ok
  eleveldb: parallel_test_ (load proc 1)...[0.033 s] ok
  eleveldb: parallel_test_ (load proc 2)...[0.034 s] ok
  eleveldb: parallel_test_ (load proc 3)...[0.033 s] ok
  eleveldb: parallel_test_ (load proc 4)...[0.027 s] ok
  eleveldb: parallel_test_ (load proc 5)...[0.035 s] ok
  [done in 0.331 s]
module 'eleveldb_schema_tests'
=======================================================
  Failed: 1.  Skipped: 0.  Passed: 29.

This is expected as mentioned in test/bucket_expiry.erl

%% Currently this test ALWAYS FAILS in open source build.  This is
%%  due to the fact that property cache is not available in open source.
%%

So all tests that are supposed to pass have passed.

[mocchira]

The result of microbench

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

Eshell V8.3  (abort with ^G)
1> m(eleveldb).
Module: eleveldb
MD5: 6e07a07773a666ca409406c579ade222
Compiled: No compile time info available
Object file: /home/leofs/dev/leofs.1.3.5/deps/eleveldb/ebin/eleveldb.beam
Compiler options:  [{outdir,"ebin"},
                    debug_info,warnings_as_errors,debug_info,
                    {i,"include"}]
Exports:
async_put/5                   module_info/0
close/1                       module_info/1
count/1                       open/2
delete/3                      option_types/1
destroy/2                     property_cache/2
fold/4                        property_cache_flush/0
fold_keys/4                   property_cache_get/1
get/3                         put/4
get_metadata_pid/1            remove_metadata_pid/2
is_empty/1                    repair/2
iterator/2                    set_metadata_pid/2
iterator/3                    status/2
iterator_close/1              validate_options/2
iterator_move/2               write/3
ok
2> {ok, DB} = eleveldb:open("test.db", [{create_if_missing, true}]).
{ok,<<>>}
3> eleveldb:put(DB, <<"key">>, <<"value">>, []).
ok
4> eleveldb:get(DB, <<"key">>, []).
{ok,<<"value">>}
5> eleveldb:count(DB).
{ok,1}
8> CountFun = fun({_, _}, Acc) -> Acc + 1 end.
#Fun<erl_eval.12.118419387>
9> eleveldb:fold(DB, CountFun, 0, []).
1
13> [eleveldb:put(DB, integer_to_binary(K), <<"value">>, []) || K <- lists:seq(1, 10000)].
[ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,
 ok,ok,ok,ok,ok,ok,ok,ok,ok,ok|...]
14> eleveldb:fold(DB, CountFun, 0, []).
10001
15> eleveldb:count(DB).
{ok,10001}
18> timer:tc(eleveldb, count, [DB]).
{3560,{ok,10001}}
19> timer:tc(eleveldb, fold, [DB, CountFun, 0, []]).
{855794,10001}
21> [eleveldb:put(DB, integer_to_binary(K), <<"value">>, []) || K <- lists:seq(1, 100000)].
[ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,ok,
 ok,ok,ok,ok,ok,ok,ok,ok,ok,ok|...]
22> timer:tc(eleveldb, count, [DB]).
{32411,{ok,100001}}
23> timer:tc(eleveldb, fold, [DB, CountFun, 0, []]).
{2971137,100001}
25> eleveldb:close(DB).
ok
27> {ok, DB2} = eleveldb:open("test.db", []).
{ok,<<>>}
28> timer:tc(eleveldb, fold, [DB2, CountFun, 0, []]).
{2256287,100001}
29> timer:tc(eleveldb, count, [DB2]).
{48403,{ok,100001}}

Around 100x times (3sec vs 30ms to open a file storing 100,000 records) speed up achieved.

[yosukehara]

@mocchira Thanks for sending a PR.

I've found some conflicts. Let me ask you to check the result of git diff.
leo_backend_db is depending on eleveldb-2.0.34 which is eleveldb's 2.0 branch.

[mocchira]

@yosukehara As there are some commits (including bug fixes/performance improvements) on the develop branch after 2.0.34 (and also I will send PR to upstream later in which the develop should be the target branch), the develop looks the first place where this PR should be merged then we'd have to tag 2.0.35 on develop once the PR got merged.

[yosukehara]

@mocchira

As there are some commits (including bug fixes/performance improvements) on the develop branch after 2.0.34 (and also I will send PR to upstream later in which the develop should be the target branch), the develop looks the first place where this PR should be merged then we'd have to tag 2.0.35 on develop once the PR got merged.

I've totally understood.

[yosukehara]

@mocchira I've just checked its latest version, v2.0.35 through the micro-bench. It is enough for us.