Mention that code coverage can change from release to release #1700
Conversation
As noted in the `-C instrument coverage` documentation itself, the exact format of instrumentation that rustc uses will vary from release to release as LLVM makes changes to it and it is also expected that code coverage data or metrics may change slightly from one release to another due to changes in how rustc generates code, how LLVM interprets coverage data and how we improve the coverage feature.
rustbot
added
the
S-waiting-on-review
| @@ -3,6 +3,7 @@ | |||
| The following [attributes] are used for controlling coverage instrumentation. | |||
|
|
|||
| > **Note**: Coverage instrumentation is controlled in `rustc` with the [`-C instrument-coverage`] compiler flag. | |||
| Exact specifics of how code coverage works or what code is reported in code coverage data may change from release to release. | |||
There was a problem hiding this comment.
My understanding of what @Zalathar has expressed a desire for, here, is that we call out more specifically that "on" and "off" are "best effort" interpretations, and that "on" will inevitably be contested when there is a new coverage knob that could be enabled by coverage(on). We then will get the following two requests:
- "This flag is vitally important for useful coverage in my area, I recommend you enable this by default when
onis indicated and add some manner of#[coverage(partial)]." - "This flag is too costly and doesn't improve useful coverage info in my area, I recommend you disable this by default and require
#[coverage(full)]to opt-in."
And that's assuming there aren't like a dozen other requests of subtle variations on these. We have to make someone unhappy.
There was a problem hiding this comment.
The approach I would personally like to see is to explicitly say that the syntax of the coverage attribute is stable, in terms of what an individual attribute looks like and where it can be placed, but the actual effect of the attribute on coverage instrumentation is not subject to stability promises and may change in the future.
(And then we would still proceed to document the current behaviour, because we want people to be able to use it.)
I don't know whether that is considered OK or not; as far as I'm aware there hasn't been much discussion of this point.
As noted in the
-C instrument coveragedocumentation itself, the exact format of instrumentation that rustc uses will vary from release to release as LLVM makes changes to it and it is also expected that code coverage data or metrics may change slightly from one release to another due to changes in how rustc generates code, how LLVM interprets coverage data and how we improve the coverage feature.