Skip to content

Commit

Permalink
Follow new naming in GHC.Stats. Fixes haskell-github-trust#29.
Browse files Browse the repository at this point in the history 
GHC commit
	  24e6594cc7890babe69b8ba122d171affabad2d1
changed a lot of the stats names.

Until now, EKG stuck to the old names, which can be very confusing;
especially since some names were clearly misleading and have been
renamed in GHC consequentially.

This commit changes all names to the new names.
For base < 4.10, we translate the old API to their new equivalents
(multiplying to get nanoseconds as needed).
The added comment documents this translation.

The change has been tested on Stackage versions:

* lts-14.27 (GHC 8.6.5)
* nightly-2018-11-08 (GHC 8.6.1)
* lts-12.17 (GHC 8.4.4)
* lts-9.21 (ghc-8.0.2) -- this is base 4.9

This change may break some users that rely on the old names,
so a new major release should be made.

The haddocks no longer explain the meaning of each field;
instead, GHC.Stats docs are linked now.
This is acceptable because the reworked GHC.Stats is much clearer
on meaning and units. It also means we cannot forget updating
ekg-core's haddocks to reflect GHC.Stats docs improvements.
  • Loading branch information
@nh2
nh2 committed Mar 25, 2020
1 parent f3e7d15 commit 80bed30
Showing 1 changed file with 112 additions and 107 deletions.
219 changes: 112 additions & 107 deletions System/Metrics.hs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -332,13 +332,14 @@ createDistribution name store = do
-- function.

#if MIN_VERSION_base(4,10,0)
-- | Convert nanoseconds to milliseconds.
nsToMs :: Int64 -> Int64
nsToMs s = round (realToFrac s / (1000000.0 :: Double))
#else
-- | Convert seconds to milliseconds.
sToMs :: Double -> Int64
sToMs s = round (s * 1000.0)

-- | Convert seconds to nanoseconds.
sToNs :: Double -> Int64
sToNs s = round (s * 1000000000.0)
#endif

-- | Register a number of metrics related to garbage collector
Expand All @@ -355,121 +356,125 @@ sToMs s = round (s * 1000.0)
-- The runtime overhead of @-T@ is very small so it's safe to always
-- leave it enabled.
--
-- Registered counters:
--
-- [@rts.gc.bytes_allocated@] Total number of bytes allocated
--
-- [@rts.gc.num_gcs@] Number of garbage collections performed
--
-- [@rts.gc.num_bytes_usage_samples@] Number of byte usage samples taken
--
-- [@rts.gc.cumulative_bytes_used@] Sum of all byte usage samples, can be
-- used with @numByteUsageSamples@ to calculate averages with
-- arbitrary weighting (if you are sampling this record multiple
-- times).
--
-- [@rts.gc.bytes_copied@] Number of bytes copied during GC
--
-- [@rts.gc.init_cpu_ms@] CPU time used by the init phase, in
-- milliseconds. GHC 8.6+ only.
--
-- [@rts.gc.init_wall_ms@] Wall clock time spent running the init
-- phase, in milliseconds. GHC 8.6+ only.
--
-- [@rts.gc.mutator_cpu_ms@] CPU time spent running mutator threads,
-- in milliseconds. This does not include any profiling overhead or
-- initialization.
--
-- [@rts.gc.mutator_wall_ms@] Wall clock time spent running mutator
-- threads, in milliseconds. This does not include initialization.
--
-- [@rts.gc.gc_cpu_ms@] CPU time spent running GC, in milliseconds.
--
-- [@rts.gc.gc_wall_ms@] Wall clock time spent running GC, in
-- milliseconds.
--
-- [@rts.gc.cpu_ms@] Total CPU time elapsed since program start, in
-- milliseconds.
--
-- [@rts.gc.wall_ms@] Total wall clock time elapsed since start, in
-- milliseconds.
--
-- Registered gauges:
--
-- [@rts.gc.max_bytes_used@] Maximum number of live bytes seen so far
--
-- [@rts.gc.current_bytes_used@] Current number of live bytes
--
-- [@rts.gc.current_bytes_slop@] Current number of bytes lost to slop
--
-- [@rts.gc.max_bytes_slop@] Maximum number of bytes lost to slop at any one time so far
--
-- [@rts.gc.peak_megabytes_allocated@] Maximum number of megabytes allocated
--
-- [@rts.gc.par_tot_bytes_copied@] Number of bytes copied during GC, minus
-- space held by mutable lists held by the capabilities. Can be used
-- with 'parMaxBytesCopied' to determine how well parallel GC utilized
-- all cores.
--
-- [@rts.gc.par_avg_bytes_copied@] Deprecated alias for
-- @par_tot_bytes_copied@.
--
-- [@rts.gc.par_max_bytes_copied@] Sum of number of bytes copied each GC by
-- the most active GC thread each GC. The ratio of
-- @par_tot_bytes_copied@ divided by @par_max_bytes_copied@ approaches
-- 1 for a maximally sequential run and approaches the number of
-- threads (set by the RTS flag @-N@) for a maximally parallel run.
-- Registered counters (see "GHC.Stats" for their meanings:
--
-- > rts.gcs
-- > rts.major_gcs
-- > rts.allocated_bytes
-- > rts.max_live_bytes
-- > rts.max_large_objects_bytes
-- > rts.max_compact_bytes
-- > rts.max_slop_bytes
-- > rts.max_mem_in_use_bytes
-- > rts.cumulative_live_bytes
-- > rts.copied_bytes
-- > rts.par_copied_bytes
-- > rts.cumulative_par_max_copied_bytes
-- > rts.cumulative_par_balanced_copied_bytes
-- > rts.init_cpu_ns
-- > rts.init_elapsed_ns
-- > rts.mutator_cpu_ns
-- > rts.mutator_elapsed_ns
-- > rts.gc_cpu_ns
-- > rts.gc_elapsed_ns
-- > rts.cpu_ns
-- > rts.elapsed_ns
-- > rts.gc.gen
-- > rts.gc.threads
-- > rts.gc.allocated_bytes
-- > rts.gc.live_bytes
-- > rts.gc.large_objects_bytes
-- > rts.gc.compact_bytes
-- > rts.gc.slop_bytes
-- > rts.gc.mem_in_use_bytes
-- > rts.gc.copied_bytes
-- > rts.gc.par_max_copied_bytes
-- > rts.gc.sync_elapsed_ns
-- > rts.gc.cpu_ns
-- > rts.gc.elapsed_ns
registerGcMetrics :: Store -> IO ()
registerGcMetrics store =
registerGroup
#if MIN_VERSION_base(4,10,0)
(M.fromList
[ ("rts.gc.bytes_allocated" , Counter . fromIntegral . Stats.allocated_bytes)
, ("rts.gc.num_gcs" , Counter . fromIntegral . Stats.gcs)
, ("rts.gc.num_bytes_usage_samples" , Counter . fromIntegral . Stats.major_gcs)
, ("rts.gc.cumulative_bytes_used" , Counter . fromIntegral . Stats.cumulative_live_bytes)
, ("rts.gc.bytes_copied" , Counter . fromIntegral . Stats.copied_bytes)
-- We order them the same way as they are in GHC.Stats for easy comparison.
[ ("rts.gcs" , Counter . fromIntegral . Stats.gcs)
, ("rts.major_gcs" , Counter . fromIntegral . Stats.major_gcs)
, ("rts.allocated_bytes" , Counter . fromIntegral . Stats.allocated_bytes)
, ("rts.max_live_bytes" , Gauge . fromIntegral . Stats.max_live_bytes)
, ("rts.max_large_objects_bytes" , Gauge . fromIntegral . Stats.max_large_objects_bytes)
, ("rts.max_compact_bytes" , Gauge . fromIntegral . Stats.max_compact_bytes)
, ("rts.max_slop_bytes" , Gauge . fromIntegral . Stats.max_slop_bytes)
, ("rts.max_mem_in_use_bytes" , Gauge . fromIntegral . Stats.max_mem_in_use_bytes)
, ("rts.cumulative_live_bytes" , Counter . fromIntegral . Stats.cumulative_live_bytes)
, ("rts.copied_bytes" , Counter . fromIntegral . Stats.copied_bytes)
, ("rts.par_copied_bytes" , Gauge . fromIntegral . Stats.par_copied_bytes)
, ("rts.cumulative_par_max_copied_bytes" , Gauge . fromIntegral . Stats.cumulative_par_max_copied_bytes)
#if MIN_VERSION_base(4,11,0)
, ("rts.cumulative_par_balanced_copied_bytes" , Gauge . fromIntegral . Stats.cumulative_par_balanced_copied_bytes)
#endif
#if MIN_VERSION_base(4,12,0)
, ("rts.gc.init_cpu_ms" , Counter . nsToMs . Stats.init_cpu_ns)
, ("rts.gc.init_wall_ms" , Counter . nsToMs . Stats.init_elapsed_ns)
, ("rts.init_cpu_ns" , Counter . Stats.init_cpu_ns)
, ("rts.init_elapsed_ns" , Counter . Stats.init_elapsed_ns)
#endif
, ("rts.gc.mutator_cpu_ms" , Counter . nsToMs . Stats.mutator_cpu_ns)
, ("rts.gc.mutator_wall_ms" , Counter . nsToMs . Stats.mutator_elapsed_ns)
, ("rts.gc.gc_cpu_ms" , Counter . nsToMs . Stats.gc_cpu_ns)
, ("rts.gc.gc_wall_ms" , Counter . nsToMs . Stats.gc_elapsed_ns)
, ("rts.gc.cpu_ms" , Counter . nsToMs . Stats.cpu_ns)
, ("rts.gc.wall_ms" , Counter . nsToMs . Stats.elapsed_ns)
, ("rts.gc.max_bytes_used" , Gauge . fromIntegral . Stats.max_live_bytes)
, ("rts.gc.current_bytes_used" , Gauge . fromIntegral . Stats.gcdetails_live_bytes . Stats.gc)
, ("rts.gc.current_bytes_slop" , Gauge . fromIntegral . Stats.gcdetails_slop_bytes . Stats.gc)
, ("rts.gc.max_bytes_slop" , Gauge . fromIntegral . Stats.max_slop_bytes)
, ("rts.gc.peak_megabytes_allocated" , Gauge . fromIntegral . (`quot` (1024*1024)) . Stats.max_mem_in_use_bytes)
, ("rts.gc.par_tot_bytes_copied" , Gauge . fromIntegral . Stats.par_copied_bytes)
, ("rts.gc.par_avg_bytes_copied" , Gauge . fromIntegral . Stats.par_copied_bytes)
, ("rts.gc.par_max_bytes_copied" , Gauge . fromIntegral . Stats.cumulative_par_max_copied_bytes)
, ("rts.mutator_cpu_ns" , Counter . Stats.mutator_cpu_ns)
, ("rts.mutator_elapsed_ns" , Counter . Stats.mutator_elapsed_ns)
, ("rts.gc_cpu_ns" , Counter . Stats.gc_cpu_ns)
, ("rts.gc_elapsed_ns" , Counter . Stats.gc_elapsed_ns)
, ("rts.cpu_ns" , Counter . Stats.cpu_ns)
, ("rts.elapsed_ns" , Counter . Stats.elapsed_ns)
-- GCDetails
, ("rts.gc.gen" , Gauge . fromIntegral . Stats.gcdetails_gen . Stats.gc)
, ("rts.gc.threads" , Gauge . fromIntegral . Stats.gcdetails_threads . Stats.gc)
, ("rts.gc.allocated_bytes" , Gauge . fromIntegral . Stats.gcdetails_allocated_bytes . Stats.gc)
, ("rts.gc.live_bytes" , Gauge . fromIntegral . Stats.gcdetails_live_bytes . Stats.gc)
, ("rts.gc.large_objects_bytes" , Gauge . fromIntegral . Stats.gcdetails_large_objects_bytes . Stats.gc)
, ("rts.gc.compact_bytes" , Gauge . fromIntegral . Stats.gcdetails_compact_bytes . Stats.gc)
, ("rts.gc.slop_bytes" , Gauge . fromIntegral . Stats.gcdetails_slop_bytes . Stats.gc)
, ("rts.gc.mem_in_use_bytes" , Gauge . fromIntegral . Stats.gcdetails_mem_in_use_bytes . Stats.gc)
, ("rts.gc.copied_bytes" , Gauge . fromIntegral . Stats.gcdetails_copied_bytes . Stats.gc)
, ("rts.gc.par_max_copied_bytes" , Gauge . fromIntegral . Stats.gcdetails_par_max_copied_bytes . Stats.gc)
, ("rts.gc.sync_elapsed_ns" , Gauge . fromIntegral . Stats.gcdetails_sync_elapsed_ns . Stats.gc)
, ("rts.gc.cpu_ns" , Gauge . fromIntegral . Stats.gcdetails_cpu_ns . Stats.gc)
, ("rts.gc.elapsed_ns" , Gauge . fromIntegral . Stats.gcdetails_elapsed_ns . Stats.gc)
])
getRTSStats
#else
-- For pre-base-4.10 we translate the names from before GHC commit
-- 24e6594cc7890babe69b8ba122d171affabad2d1 to their newer equivalents
-- so that ekg-core always presents the same names (given that e.g.
-- the ekg Javascript expects them to exist).
-- The mapping is obtained obtained from
-- https://hackage.haskell.org/package/base-4.10.0.0/docs/GHC-Stats.html
-- which has both the old and the new interface, as well as from
-- the commit diff implementation in `Stats.c`.
(M.fromList
[ ("rts.gc.bytes_allocated" , Counter . Stats.bytesAllocated)
, ("rts.gc.num_gcs" , Counter . Stats.numGcs)
, ("rts.gc.num_bytes_usage_samples" , Counter . Stats.numByteUsageSamples)
, ("rts.gc.cumulative_bytes_used" , Counter . Stats.cumulativeBytesUsed)
, ("rts.gc.bytes_copied" , Counter . Stats.bytesCopied)
, ("rts.gc.mutator_cpu_ms" , Counter . sToMs . Stats.mutatorCpuSeconds)
, ("rts.gc.mutator_wall_ms" , Counter . sToMs . Stats.mutatorWallSeconds)
, ("rts.gc.gc_cpu_ms" , Counter . sToMs . Stats.gcCpuSeconds)
, ("rts.gc.gc_wall_ms" , Counter . sToMs . Stats.gcWallSeconds)
, ("rts.gc.cpu_ms" , Counter . sToMs . Stats.cpuSeconds)
, ("rts.gc.wall_ms" , Counter . sToMs . Stats.wallSeconds)
, ("rts.gc.max_bytes_used" , Gauge . Stats.maxBytesUsed)
, ("rts.gc.current_bytes_used" , Gauge . Stats.currentBytesUsed)
, ("rts.gc.current_bytes_slop" , Gauge . Stats.currentBytesSlop)
, ("rts.gc.max_bytes_slop" , Gauge . Stats.maxBytesSlop)
, ("rts.gc.peak_megabytes_allocated" , Gauge . Stats.peakMegabytesAllocated)
, ("rts.gc.par_tot_bytes_copied" , Gauge . gcParTotBytesCopied)
, ("rts.gc.par_avg_bytes_copied" , Gauge . gcParTotBytesCopied)
, ("rts.gc.par_max_bytes_copied" , Gauge . Stats.parMaxBytesCopied)
[ ("rts.allocated_bytes" , Counter . Stats.bytesAllocated)
, ("rts.gcs" , Counter . Stats.numGcs)
, ("rts.major_gcs" , Counter . Stats.numByteUsageSamples)
, ("rts.cumulative_live_bytes" , Counter . Stats.cumulativeBytesUsed)
, ("rts.copied_bytes" , Counter . Stats.bytesCopied)
, ("rts.mutator_cpu_ns" , Counter . sToNs . Stats.mutatorCpuSeconds)
, ("rts.mutator_elapsed_ns" , Counter . sToNs . Stats.mutatorWallSeconds)
, ("rts.gc_cpu_ns" , Counter . sToNs . Stats.gcCpuSeconds)
, ("rts.gc_elapsed_ns" , Counter . sToNs . Stats.gcWallSeconds)
, ("rts.cpu_ns" , Counter . sToNs . Stats.cpuSeconds)
, ("rts.elapsed_ns" , Counter . sToNs . Stats.wallSeconds)
, ("rts.max_live_bytes" , Gauge . Stats.maxBytesUsed)
, ("rts.gc.live_bytes" , Gauge . Stats.currentBytesUsed)
, ("rts.gc.slop_bytes" , Gauge . Stats.currentBytesSlop)
, ("rts.max_slop_bytes" , Gauge . Stats.maxBytesSlop)
, ("rts.max_mem_in_use_bytes" , Gauge . Stats.peakMegabytesAllocated)
-- Note that historically, the values
-- par_tot_bytes_copied were both
-- par_avg_bytes_copied
-- were both taken from
-- gcParTotBytesCopied
-- after `parAvgBytesCopied` was renamed to `gcParTotBytesCopied`;
-- see `ekg` commit
-- 27467a61 - parAvgBytesCopied was renamed in GHC 7.6.1
, ("rts.par_copied_bytes" , Gauge . gcParTotBytesCopied)
, ("rts.gc.par_max_copied_bytes" , Gauge . Stats.parMaxBytesCopied)
])
getGcStats
#endif
Expand Down

0 comments on commit 80bed30

Please sign in to comment.