Commits on Jul 17, 2020

1. Remove skc_reclaim, hdr_recl, kmem_cache shrinker …

   The SPL kmem_cache implementation provides a mechanism, `skc_reclaim`,
   whereby individual caches can register a callback to be invoked when
   there is memory pressure.  This mechanism is used in only one place: the
   ARC registers the `hdr_recl()` reclaim function.  This function wakes up
   the `arc_reap_zthr`, whose job is to call `kmem_cache_reap()` and
   `arc_reduce_target_size()`.
   
   The `skc_reclaim` callbacks are invoked only by shrinker callbacks and
   `arc_reap_zthr`, and only callback only wakes up `arc_reap_zthr`.  When
   called from `arc_reap_zthr`, waking `arc_reap_zthr` is a no-op.  When
   called from shrinker callbacks, we are already aware of memory pressure
   and responding to it.  Therefore there is little benefit to ever calling
   the `hdr_recl()` `skc_reclaim` callback.
   
   The `arc_reap_zthr` also wakes once a second, and if memory is low when
   allocating an ARC buffer.  Therefore, additionally waking it from the
   shrinker calbacks has little benefit.
   
   The shrinker callbacks can be invoked very frequently, e.g. 10,000 times
   per second.  Additionally, for invocation of the shrinker callback,
   skc_reclaim is invoked many times.  Therefore, this mechanism consumes
   significant amounts of CPU time.
   
   The kmem_cache shrinker calls `spl_kmem_cache_reap_now()`, which,
   in addition to invoking `skc_reclaim()`, does two things to attempt to
   free pages for use by the system:
    1. Return free objects from the magazine layer to the slab layer
    2. Return entirely-free slabs to the page layer (i.e. free pages)
   
   These actions apply only to caches implemented by the SPL, not those
   that use the underlying kernel SLAB/SLUB caches.  The SPL caches are
   used for objects >=32KB, which are primarily linear ABD's cached in the
   DBUF cache.
   
   These actions (freeing objects from the magazine layer and returning
   entirely-free slabs) are also taken whenever a `kmem_cache_free()` call
   finds a full magazine.  So there would typically be zero entirely-free
   slabs, and the number of objects in magazines is limited (typically no
   more than 64 objects per magazine, and there's one magazine per CPU).
   Therefore the benefit of `spl_kmem_cache_reap_now()`, while nonzero, is
   modest.
   
   We also call `spl_kmem_cache_reap_now()` from the `arc_reap_zthr`, when
   memory pressure is detected.  Therefore, calling
   `spl_kmem_cache_reap_now()` from the kmem_cache shrinker is not needed.
   
   This commit removes the `skc_reclaim` mechanism, its only callback
   `hdr_recl()`, and the kmem_cache shrinker callback.
   
   Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

   

   ahrens committed Jul 17, 2020
   Configuration menu

   • View commit details
   • Copy full SHA for 0358c05
   • Browse repository at this point

   Copy the full SHA

   0358c05 View commit details

   Browse the repository at this point in the history