Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse BRW Stats #9

Merged
merged 3 commits into from
Apr 5, 2017
Merged

Conversation

roclark
Copy link
Contributor

@roclark roclark commented Apr 4, 2017

The Block Read/Write (BRW) stats file includes the following (paraphrased from the Lustre manual)

  • Pages per block R/W: Counter value of the number of pages per RPC (remote procedure call)
  • Discontiguous pages: Counter value of the number of logical discontinuities per RPC
  • I/O Time: The time in milliseconds the filesystem has spent processing various object sizes
  • Disk I/O Size: Counter value of the number of operations the filesystem has performed for the given size
  • Disk I/O's in Flight: Current number of I/O operations that are processing during the snapshot

@roclark
Copy link
Contributor Author

roclark commented Apr 4, 2017

@joehandzik this is a pretty big refactor, so please let me know if you want me to change anything and I will be glad to do so.

@roclark
Copy link
Contributor Author

roclark commented Apr 4, 2017

Forgot to mention I need to add the help text still. I just want to leave this here in case you have any thoughts on the current structure.

ToDo:

  • Add help text

@@ -63,6 +63,7 @@ func (s *lustreSource) generateOSSMetricTemplates() error {
"obdfilter/*": map[string]string{
"blocksize": "Filesystem block size in bytes",
"brw_size": "Block read/write size in bytes",
"brw_stats": "A collection of block read/write statistrics",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*statistics

@joehandzik
Copy link
Contributor

It looks pretty good so far, but I'll come back to it a few times today to see if I have any useful suggestions. Clearly, some of these files are a pain to parse.

@roclark
Copy link
Contributor Author

roclark commented Apr 4, 2017

Sounds good! Agreed. But in the case with this file, the reward is very much worth it as I'm able to scrape some really useful data about the cluster.

roclark added 3 commits April 4, 2017 15:54
Signed-Off-By: Robert Clark <robert.d.clark@hpe.com>
A new metric description needed to be created for block read/write stats due to the various sub-labels for each metric. In the original 'constMetric' description, only the nodeName (OST, MGT or MDT) was a sub-label. For the various BRW stats to be included with this, a new metric would need to be created for every size and operation. For example, the 'discontiguous pages' metric with a size of '1' and operation of 'read' would require a different metric from 'discontiguous pages' with a size of '1' and operation of 'write'. The new 'brwMetric' description includes sub-labels for 'size' and 'operation' to allow similar metrics to be grouped together, minimizing clutter.

Signed-Off-By: Robert Clark <robert.d.clark@hpe.com>
Signed-Off-By: Robert Clark <robert.d.clark@hpe.com>
@roclark
Copy link
Contributor Author

roclark commented Apr 5, 2017

Help text is added and it compiles and runs as expected!

@joehandzik
Copy link
Contributor

LGTM

@joehandzik joehandzik merged commit 70d91b4 into HewlettPackard:master Apr 5, 2017
@roclark roclark deleted the parse-brw-stats branch April 5, 2017 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants