DAOS-16749 vos: OI iterator for phase2 pool #15465
Conversation
To minimize bucket eviction/load when iterating objects, vos_iterate_obj() is introduced to iterate objects in bucket ID order instead of OI order. The caller of vos_iterate_obj() needs to provide a filter callback to call the vos_bkt_iter_skip() properly. Applied the vos_iterate_obj() for EC & VOS aggregation. Required-githooks: true Signed-off-by: Niu Yawei <yawei.niu@intel.com>
|
Ticket title is 'VOS iteration optimization' |
NiuYawei
requested review from
sherintg,
jolivier23,
Nasf-Fan,
gnailzenh and
wangshilong
| for (i = UMEM_DEFAULT_MBKT_ID; i < bkt_iter->bi_bkt_tot; i++) { | ||
| if (i > UMEM_DEFAULT_MBKT_ID) { | ||
| /* The bucket wasn't skipped in prior rounds of iterating */ | ||
| if (!isset(&bkt_iter->bi_skipped[0], i)) |
There was a problem hiding this comment.
if it's not set, which means the bucket wasn't skipped by checking bucket ID, then we can skip iterating on this bucket ID, so it's "!isset()".
There was a problem hiding this comment.
Sorry, confused. If it is not marked as skipped, then we should call the subsequent vos_iterate_internal or not?
There was a problem hiding this comment.
If it's not marked as skipped (skipped due to unmatched bkt ID), then we don't need to call following vos_iterate_internal() for this bucket, otherwise (there were some objects was skipped due to unmatched bkt ID), we have to call vos_iterate_internal() for this bucket. Does it make sense?
|
|
||
| /* This MUST be the last check */ | ||
| if (desc->id_type == VOS_ITER_OBJ && vos_bkt_iter_skip(ih, desc)) { | ||
| credits_consume(&agg_param->ap_credits, AGG_OP_SCAN); |
There was a problem hiding this comment.
If there are a lot of objects to be skipped, will it exhaust the credits before arriving at the object(s) that needs aggregation?
There was a problem hiding this comment.
Yes, that's a very common case today.
| if (desc->id_type == VOS_ITER_OBJ && vos_bkt_iter_skip(ih, desc)) { | ||
| agg_param->ap_credits++; | ||
| *acts |= VOS_ITER_CB_SKIP; | ||
| goto done; |
There was a problem hiding this comment.
If it must be the last check, then do not need "goto done".
There was a problem hiding this comment.
Well, I was just thinking that some other operations (not related to skip check) could be added in the future. I can remove it if the patch needs be refreshed.
To minimize bucket eviction/load when iterating objects, vos_iterate_obj() is introduced to iterate objects in bucket ID order instead of OI order. The caller of vos_iterate_obj() needs to provide a filter callback to call the vos_bkt_iter_skip() properly.
Applied the vos_iterate_obj() for EC & VOS aggregation.
Required-githooks: true
Before requesting gatekeeper:
Features:(orTest-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.Gatekeeper: