MRU, MFU don't adapt to their targets #16576
Comments
|
I am probably missing something, but should line
rather than
? |
|
I have an idea where the problem might be. The current
I think |
I am not sure.
However, it's not clear to me what
Why shifting 32 to right? My naive implementation would be something as:
I understand divisions are very slow and should be avoided, but the two approaches results in vastly different numbers. Moreover, to my untrained eyes, the logic seems upside-down: metadata are generally more valuable than data, so why start evicting from metadata? Ordering eviction by MRU data > MRU metadata > MFU data > MFU metadata seems preferable. I am sure there are good reasons to the current code, but I fail to understand them. Maybe @amotin can shed some light? |
I agree. But for mfu data,
According to #14359, I think the shifting has to do with 32-bit fixed point fractions. arc_meta is a portion of 1 (1ULL) that has been shifted left 32 bits. So I think a simple but incorrect version would be
I believe mfu data is evicted last to allow it to grow beyond its target by taking all available space ( |
Ok, so the logic is:
Does this means that part of MRU is effectively "protected" against eviction, putting too much pressure on MFU?
Thanks for pointing me to #14359 - I forgot about it. |
The part of fixed point math @tkittich explained right.
My motivation was that it might be impossible to evict from each state as much as we'd like. It primarily affects metadata, since those blocks are often held by data blocks or something else and are unevictable. In case we can not satisfy eviction goal by metadata, we have to compensate it with whatever else we can, which is data, as usually more evictable. Order between MRU and MFU IIRC was pretty arbitrary, just this way already complicated math was a bit easier, but I can see a benefits of doing other way around, so that in case of eviction problems MRU suffered more. |
|
Looking on the provided @tkittich could you try to save the |
Thank you for looking into this. I am not quite sure how, but the suggested modification apparently works. The MFU size is no longer shrinking and stays near its target in my workload. ^^ |
Since arc_evict() run can take some time, arc_c change during it may result in undesired shift in ARC states balance. Primarily in case of arc_c reduction it may cause eviction from MFU data state despite its being below the target already. Instead we should evict as originally planned and if needed do another round after. Fixes: openzfs#16576 Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc.
Since arc_evict() run can take some time, arc_c change during it may result in undesired shift in ARC states balance. Primarily in case of arc_c reduction it may cause eviction from MFU data state despite its being below the target already. Instead we should evict as originally planned and if needed do another round after. Fixes: openzfs#16576 Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc.
Since arc_evict() run can take some time, arc_c change during it may result in undesired shift in ARC states balance. Primarily in case of arc_c reduction it may cause eviction from MFU data state despite its being below the target already. Instead we should evict as originally planned and if needed do another round after. Reviewed-by: Theera K. <tkittich@hotmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc. Closes openzfs#16576 Closes openzfs#16605
Since arc_evict() run can take some time, arc_c change during it may result in undesired shift in ARC states balance. Primarily in case of arc_c reduction it may cause eviction from MFU data state despite its being below the target already. Instead we should evict as originally planned and if needed do another round after. Reviewed-by: Theera K. <tkittich@hotmail.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc. Closes openzfs#16576 Closes openzfs#16605
System information
Describe the problem you're observing
This is a continuation of #16521. I'm still seeing a similar problem after applying PR #16546. The MFU size (mfusz) kept shrinking despite having a much higher MFU target (mfutg) and some ghost hits. arcstat was updated every minute. There're no file deletions or overwrites during the whole period. The MFU size still kept shrinking after this but the trace would get very long. On the other hand, MRU size (mrusz) kept growing despite being way over target (mrutg).
Include any warning/errors/backtraces from the system logs
It's strange that
arc_evict_impl()for MFU data (the 4th ones?) would still be positive despite MFU size that's way lower than MFU target. Perhaps the amount to be evicted should be based on the difference from its target instead?The text was updated successfully, but these errors were encountered: