Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testutil package #445

Merged
merged 9 commits into from
Sep 2, 2018
Merged

Add testutil package #445

merged 9 commits into from
Sep 2, 2018

Conversation

beorn7
Copy link
Member

@beorn7 beorn7 commented Aug 22, 2018

This is derived from #423. I recreated it so that it is pulling from a branch I can push to...

Changes to #423 are based on the review that has already happened. I packaged it nicely in commits with messages, so you can follow along.

I will add more documentation and the single-value retrieval helpers (cf. #230) in a separate PR. Same with the creation of an internal package for the duplicated code. Don't want to put too many things into this one PR.

@grobie @brancz

brancz and others added 6 commits July 3, 2018 13:25
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
It wasn't needed, as is now proven by the tests

Signed-off-by: beorn7 <beorn@soundcloud.com>
`testutil` is more in line with stdlib naming conventions.

The package should be below `prometheus` as it only provides utils to
test exposition code, not to test HTTP client code.

Signed-off-by: beorn7 <beorn@soundcloud.com>
Signed-off-by: beorn7 <beorn@soundcloud.com>
Signed-off-by: beorn7 <beorn@soundcloud.com>
@beorn7 beorn7 requested review from brancz and grobie August 24, 2018 10:03
@beorn7
Copy link
Member Author

beorn7 commented Aug 29, 2018

@brancz @grobie could you have a look?

@@ -1,3 +1,16 @@
// Copyright 2018 he Prometheus Authors
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling "The" here.

// CollectAndCompare registers the provided Collector with a newly created
// pedantic Registry. It then does the same as GatherAndCompare, gathering the
// metrics from the pedantic Registry.
func CollectAndCompare(c prometheus.Collector, expected io.Reader, metricNames ...string) error {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit message says "Expected text format is not read from an io.Reader" but I see an io.Reader here. Perhaps the message should be "... is now read ..." ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. I will fix the commit description.

metrics = filterMetrics(metrics, metricNames)
}
var tp expfmt.TextParser
expectedMetrics, err := tp.TextToMetricFamilies(expected)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why bother parsing the expected text? If the test author is responsible for typing/writing the expected golden values, is it that much more work to match the formatting of the encoder?

Perhaps if the normalizing and sorting in this file is later replaced by calls to some shared package this approach won't seem so complicated. Is that the plan?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's legal in the text format to liberally use whitespace. Knowing exactly the formatting of the encoder would be a burden for the programmer writing those tests, and one for which I cannot really see any gain.

Personally, I wouldn't even like if a "normalized" formatting is advertised (but that's a discussion that is happening elsewhere and definitely out of scope of this PR). In any case, there is no goal right now to provide a tool for "normalizing".

break
}
}
if !drop {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These loops make more sense to me without any boolean var:

if m.GetName() == name {
  filtered = append(filtered, m)
  break
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are right.

NB: This is not my code. So I don't know if there is some non-obvious reason why it was done the way it is.

I'll change it according to your suggestion, and @brancz can chime in if we blundered here.

- Expected text format is now read from an io.Reader.
- Metrics are gathered from a Gatherer.
- Added a convenience wrapper to collect from a Collector.

Signed-off-by: beorn7 <beorn@soundcloud.com>
@beorn7
Copy link
Member Author

beorn7 commented Sep 2, 2018

@cbandy Thanks for your review. I incorporated the suggested changes (except the one requiring the test writer to follow a certain formatting, see my comment above).

Still, I would feel better to get a look from @grobie or @brancz before merging.

This is for types we don't want to export but which are used in
different packages within client_golang.

Currently, that's only NormalizeMetricFamilies (used in the prometheus
package and in the testutil package). More to be added as needed.

Signed-off-by: beorn7 <beorn@soundcloud.com>
@brancz
Copy link
Member

brancz commented Sep 2, 2018

I was about to comment that we should probably move the metricSorter to an internal package, but then I just noticed that you opened #452, so I think that's what you're already going for. Once that's moved to the internal package and this PR uses that, this lgtm 👍 .

@beorn7
Copy link
Member Author

beorn7 commented Sep 2, 2018

That's all mapped out in the initial description of this PR. :o)

@beorn7 beorn7 merged commit a10423e into master Sep 2, 2018
@beorn7 beorn7 deleted the beorn7/testing branch September 2, 2018 22:31
@grobie
Copy link
Member

grobie commented Sep 3, 2018

Looks good to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants