[wip][abi-incompat.] Introduce background stale slot coalescence #9319
Conversation
ryoqun
added
work in progress
| bank.process_dead_slots(); | ||
|
|
||
| // Currently, given INTERVAL_MS, we process 1 slot/100 ms | ||
| if i % 10 == 0 { |
There was a problem hiding this comment.
This is tooo rough. But I intend a validator to scan its all appendvecs once (or twice) a day.
| total_alive_count += alive_count; | ||
| all_stored_accounts.extend(stored_accounts); | ||
|
|
||
| if total_alive_count > 100 { |
There was a problem hiding this comment.
This is tooo rough. But I intend a validator to hold many appendvecs of minimum size of 2MiB or 4MiB?
| @@ -1371,6 +1389,55 @@ impl AccountsDB { | |||
| infos | |||
| } | |||
|
|
|||
| fn store_accounts_to2<F: FnMut(Slot) -> Arc<AccountStorageEntry>, W: Iterator<Item = u64>>( | |||
There was a problem hiding this comment.
copy & paste for the sake of quick early review....
| @@ -1605,16 +1672,44 @@ impl AccountsDB { | |||
| reclaims | |||
| } | |||
|
|
|||
| fn update_index2( | |||
There was a problem hiding this comment.
copy & paste for the sake of quick early review....
| @@ -34,6 +36,7 @@ const MAXIMUM_APPEND_VEC_FILE_SIZE: usize = 16 * 1024 * 1024 * 1024; // 16 GiB | |||
| /// So the data layout must be stable and consistent across the entire cluster! | |||
| #[derive(Clone, PartialEq, Debug)] | |||
| pub struct StoredMeta { | |||
| pub slot: Slot, | |||
There was a problem hiding this comment.
This is the ABI break.
| pub fn new_relative_path(slot: Slot, id: usize) -> PathBuf { | ||
| PathBuf::from(&format!("{}.{}", slot, id)) | ||
| pub fn new_relative_path(id: AppendVecId) -> PathBuf { | ||
| PathBuf::from(&format!("{:016x}.dat", id)) |
There was a problem hiding this comment.
Well, this is my taste... xD
| @@ -473,6 +481,7 @@ pub mod test_utils { | |||
| let mut account = Account::new(sample as u64, 0, &Pubkey::default()); | |||
| account.data = (0..data_len).map(|_| data_len as u8).collect(); | |||
| let stored_meta = StoredMeta { | |||
| slot: 0, | |||
| let mut s = storage.0.values().flatten().collect::<Vec<_>>(); | ||
|
|
||
| s.sort_by(|(_a, a), (_x, b)| a.id.cmp(&b.id)); | ||
| s.iter().map(|(_a, entry)| Arc::downgrade(entry)).collect() |
There was a problem hiding this comment.
Glad, we had Weak of the shelf. :)
|
@sakridge I've filled the description with minimal explanation. Could you share your thought on this? |
|
I think we might want to explore more aggressive rent collection. We actually didn't have per-slot stores originally, but it was expensive to maintain a the large structures that grew with a flat setup. |
I see! Then, I'll try the rent approach. And already digging in some rent related code. Also, I've noticed when changing the size of |
yea, I think if I did the math right: |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
stale
bot
added
the
stale
|
This stale pull request has been automatically closed. Thank you for your contributions. |
Quick follow-up to #9219 for the final fix for the break & devnet issue.
Problem
Too many DOS-able
mmaps (#8931) as exemplified by devnet with the break app.Even with #9219, an attacker can force a validator to use too much system recources (AppendVec/mmap).
Summary of Changes
After much thought, I've come up with this PR's strategy: Make it possible for a single AppendVec to hold multiple slots' accounts by exploiting the existing reference-count in AccountStorageEntry.
So, AccountsBackgroundService now starts to pack several neighbor small AppendVecs into one, not only shrinking them individually.
pros:
con:
Alternatives:
Given ease of implementation, effect on performance, efficacy for the DoS attach, I think this idea might be good. :)
Fixes #8931
Part of #7167