Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(collector): add statistics for estimate key number of partition #435

Merged
merged 12 commits into from
Dec 3, 2019
Merged

feat(collector): add statistics for estimate key number of partition #435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
c6d6f0d
add key number
foreverneverer Nov 29, 2019
909df80
Merge branch 'master' into table-data-count
acelyc111 Dec 3, 2019
a5f2d4a
format
foreverneverer Dec 3, 2019
a9b9d16
Merge branch 'table-data-count' of github.com:Shuo-Jia/pegasus into t…
foreverneverer Dec 3, 2019
533a1a7
format and fix
foreverneverer Dec 3, 2019
9f5d9cc
format and fix
foreverneverer Dec 3, 2019
943e18b
format and fix
foreverneverer Dec 3, 2019
8658b53
format and fix
foreverneverer Dec 3, 2019
ca6d634
format and fix
foreverneverer Dec 3, 2019
8dc8229
format and fix
foreverneverer Dec 3, 2019
f9dafcc
format and fix
foreverneverer Dec 3, 2019
e835818
format and fix
foreverneverer Dec 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/server/pegasus_server_impl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,13 @@ pegasus_server_impl::pegasus_server_impl(dsn::replication::replica *r)
snprintf(name, 255, "rdb.memtable.memory_usage@%s", str_gpid.c_str());
_pfc_rdb_memtable_mem_usage.init_app_counter(
"app.pegasus", name, COUNTER_TYPE_NUMBER, "statistic the memory usage of rocksdb memtable");

snprintf(name, 255, "rdb.estimate_num_keys@%s", str_gpid.c_str());
_pfc_rdb_estimate_num_keys.init_app_counter(
"app.pegasus",
name,
COUNTER_TYPE_NUMBER,
"statistics the estimated number of keys inside the rocksdb");
}

void pegasus_server_impl::parse_checkpoints()
Expand Down Expand Up @@ -2242,6 +2249,14 @@ void pegasus_server_impl::update_replica_rocksdb_statistics()
_pfc_rdb_memtable_mem_usage->set(val);
dinfo_replica("_pfc_rdb_memtable_mem_usage: {} bytes", val);
}

// for the same n kv pairs, kEstimateNumKeys will be counted n times, you need compaction to
// remove duplicate
if (_db->GetProperty(rocksdb::DB::Properties::kEstimateNumKeys, &str_val) &&
dsn::buf2uint64(str_val, val)) {
_pfc_rdb_estimate_num_keys->set(val);
dinfo_replica("_pfc_rdb_estimate_num_keys: {}", val);
}
}

void pegasus_server_impl::update_server_rocksdb_statistics()
Expand Down
1 change: 1 addition & 0 deletions src/server/pegasus_server_impl.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,7 @@ class pegasus_server_impl : public ::dsn::apps::rrdb_service
::dsn::perf_counter_wrapper _pfc_rdb_block_cache_total_count;
::dsn::perf_counter_wrapper _pfc_rdb_index_and_filter_blocks_mem_usage;
::dsn::perf_counter_wrapper _pfc_rdb_memtable_mem_usage;
::dsn::perf_counter_wrapper _pfc_rdb_estimate_num_keys;
};

} // namespace server
Expand Down