Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Structure and Continuity; remove instantaneous Temporality #181

Closed
wants to merge 5 commits into from
Closed

Add Structure and Continuity; remove instantaneous Temporality #181

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b257c60
Add Structure and Continuity
Jul 28, 2020
f3baec4
Tidy comments
Jul 28, 2020
073dd4d
More tidy
Jul 28, 2020
4ac91cd
More tidy
Jul 28, 2020
4c21cfd
Histogram and Summary MUST set StartTimeUnixNano
Jul 28, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 169 additions & 62 deletions opentelemetry/proto/metrics/v1/metrics.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,13 +92,11 @@ message InstrumentationLibraryMetrics {
//
// All DataPoint types have three common fields:
// - Labels zero or more key-value pairs associated with the data point.
// - StartTimeUnixNano MUST be set to the start of the interval when the
// descriptor Temporality includes CUMULATIVE or DELTA. This field is not set
// for INSTANTANEOUS timeseries, where instead the TimeUnixNano field is
// set for individual points.
// - StartTimeUnixNano MUST be to the start of the interval, when an aggregation
// was applied, but may be omitted for raw data points.
// - TimeUnixNano MUST be set to:
// - the end of the interval (CUMULATIVE or DELTA)
// - the instantaneous time of the event (INSTANTANEOUS).
// - the end of the interval, if aggregation was applied
// - the instant of the event, if no aggregation was applied.
message Metric {
// metric_descriptor describes the Metric.
MetricDescriptor metric_descriptor = 1;
Expand Down Expand Up @@ -134,34 +132,22 @@ message MetricDescriptor {
// A Metric of this Type MUST store its values as Int64DataPoint.
INT64 = 1;

// MONOTONIC_INT64 values are monotonically increasing signed 64-bit
// integers.
//
// A Metric of this Type MUST store its values as Int64DataPoint.
MONOTONIC_INT64 = 2;

// DOUBLE values are double-precision floating-point numbers.
//
// A Metric of this Type MUST store its values as DoubleDataPoint.
DOUBLE = 3;

// MONOTONIC_DOUBLE values are monotonically increasing double-precision
// floating-point numbers.
//
// A Metric of this Type MUST store its values as DoubleDataPoint.
MONOTONIC_DOUBLE = 4;
DOUBLE = 2;

// Histogram measurement.
// Corresponding values are stored in HistogramDataPoint.
HISTOGRAM = 5;
HISTOGRAM = 3;

// Summary value. Some frameworks implemented Histograms as a summary of observations
// (usually things like request durations and response sizes). While it
// also provides a total count of observations and a sum of all observed
// values, it calculates configurable quantiles over a sliding time
// window.
// Corresponding values are stored in SummaryDataPoint.
SUMMARY = 6;
SUMMARY = 4;
}

// type is the type of values this metric has.
Expand All @@ -175,12 +161,6 @@ message MetricDescriptor {
// used.
INVALID_TEMPORALITY = 0;

// INSTANTANEOUS is a metric whose values are measured at a particular
// instant. The values are not aggregated over any time interval and are
// unique per timestamp. As such, these metrics are not expected to have
// an associated start time.
INSTANTANEOUS = 1;

// DELTA is a metric whose values are the aggregation of measurements
// made over a time interval. Successive metrics contain aggregation of
// values from continuous and non-overlapping intervals.
Expand All @@ -205,7 +185,7 @@ message MetricDescriptor {
// 8. The 1 second collection cycle ends. A metric is exported for the
// number of requests received over the interval of time t_0+1 to
// t_0+2 with a value of 2.
DELTA = 2;
DELTA = 1;

// CUMULATIVE is a metric whose values are the aggregation of
// successively made measurements from a fixed start time until the last
Expand Down Expand Up @@ -238,11 +218,139 @@ message MetricDescriptor {
// 12. The 1 second collection cycle ends. A metric is exported for the
// number of requests received over the interval of time t_1 to
// t_0+1 with a value of 1.
CUMULATIVE = 3;
CUMULATIVE = 2;
}

// Structure is the structural quality of a metric, indicating
// whether the input measurements represent part of an individual
// sum or are considered individual measurements.
enum Structure {
// INVALID_STRUCTURE is the default Structure, it MUST not be
// used.
INVALID_STRUCTURE = 0;

// GROUPING structure means the value has been computed from
// individual input values considered part of a distribution.
// GROUPING structure implies the sum of measurements is not
// necessarily meaningful.
//
// Value of this kind are defined as a current measurement of the
// metric when represented by single-value data points.
GROUPING = 1;

// ADDING_MONOTONIC structure means the measurement determines a
// sum. For DELTA kind this is expressed as the change in sum
// since the last collection. For CUMULATIVE kind this is
// expressed as the last collected value of the sum since
// reporting began.
//
// ADDING_MONOTONIC indicates that the calculated sum is
// non-decreasing, therefore can be monitored as a non-negative
// rate of change.
ADDING_MONOTONIC = 2;

// ADDING structure means the measurement determines a sum. For
// DELTA kind this is expressed as the change in sum since the
// last collection. For CUMULATIVE kind this is expressed as the
// last collected value of the sum since reporting began.
ADDING = 3;
}

// Continuity describes how the measurement was captured,
// indicating whether measurements represent application-level
// events or were generated through a callback invoked by the SDK.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is not only the callback that can generate snapshots, aggregating any already aggregated data is a snapshot. Think in collector I can receive points from a Counter aggregated as a sum/delta then building a Histogram there makes the input snapshot

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also for a Counter (aggregating cumulative Sum) vs SumObserver (cumulative Sum calculated outside) - both sums were calculated by an aggregator (in our outside our library), and at the moment you export the sum there is minimal information that Snapshot vs Continuous provide - no more information about the number of measurements available. But in case of a Histogram where we also include the count of measurements this information may be useful (and probably is). So I am not 100% convinced that Continuity makes sense for scalar metrics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aggregating any already aggregated data is a snapshot

I would have stated that:

  • aggregating already-aggregated SNAPSHOT data is SNAPSHOT data
  • aggregating already-aggregated CONTINUOUS data is CONTINUOUS data

Combining multiple input metrics of different Continuity is not well defined in this (variation of the) protocol, and I'm not sure we should be aiming for a protocol that supports multiple input metrics regardless of whether they agree on Structure/Continuity. In all cases, the presence of StartTimeUnixNano != 0 indicates that the input data spanned a period of time as opposed to being defined at an instant where StartTimeUnixNano == TimeUnixNano.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at the moment you export the sum there is minimal information that Snapshot vs Continuous provide

My intention is that both of these fields tell us about the input measurements, which do not change when we aggregate.

There may be minimal information, but there is information here. If you see two raw measurements with the same timestamp and they are SNAPSHOT, they were made by the same callback and can be compared as a ratio. If you see two raw measurements with the same timestamp and they are CONTINUOUS, they are merely coincidental and should not be compared as a ratio.

A sum is a sum, and we know that it is sensible to compute a rate of a sum. A "gauge" (or non-sum) should not be exposed as a rate, because we don't know that the distance between A and B is the same as the distance between (A+x) and (B+x). This quality is the reason that Sums can be displayed as rates regardless of whether they are monotonic or not.

Copy link
Member

@bogdandrutu bogdandrutu Jul 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention is that both of these fields tell us about the input measurements, which do not change when we aggregate.

I think this is not clear to me, will try to show an example to show where my confusion comes from:

  1. An "UpDownSumObserver" for "memory usage", and having an export interval of 10seconds, every 10 second I report the cumulative sum of all "new/delete" calls. User configures the library to just export the value read from the kernel with no changes.
  2. An "UpDownCounter" for "new/delete" calls, by instrumenting tcmalloc library. We have configured a cumulative sum aggregation (maybe without exemplars if that will be possible). Export interval is the same 10 seconds.

In this two cases I get exactly the same data - every 10 second a scalar metric with the sum of new/delete from the beginning of the process. The only difference may be "exemplars" which may or may not be present for the second metric (not guaranteed that they are present, if the user configures to not keep exemplars in the view configuration).

Now the idea is, way do we need to know and complicate the model for this case with continuous vs snapshot? Both are snapshots because even in the second metric we actually take a snapshot of the aggregator at a specific moment of time.

If you see two raw measurements with the same timestamp and they are SNAPSHOT, they were made by the same callback and can be compared as a ratio. If you see two raw measurements with the same timestamp and they are CONTINUOUS, they are merely coincidental and should not be compared as a ratio.

Here is where I agree with you, but I see things a bit different as proposed in #179, we have "Raw measurements" a.k.a CONTINUOUS which represents measurements recorded using the sync instruments with no other aggregation and any other "measurements" are snapshot independent of where that aggregation happens (in our library like in the case of the second metric, or outside our library in case of the first metric) because we actually take a snapshot of an Aggregator.

Probably I miss something, but here is where my current understanding is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in my proposal to clarify "Raw Measurements" imply continuos all the other types imply snapshot because we take a snapshot of an Aggregator that somewhere exists (inside or outside of our library)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this two cases I get exactly the same data - every 10 second a scalar metric with the sum of new/delete from the beginning of the process

The count of label sets means something different in these cases. The Observer instrument produces N points at the same logical instant in time, whereas the Recorder instrument produces N points over a window of time. If I ask how many distinct label sets there are from a Recorder (CONTINUOUS), I have to specify a time window for the question to be meaningful. For an Observer (SNAPSHOT), I can answer the question ("how many distinct label sets") without specifying a time range.

Copy link
Member

@bogdandrutu bogdandrutu Jul 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The count of label sets means something different in these cases. The Observer instrument produces N points at the same logical instant in time, whereas the Recorder instrument produces N points over a window of time. If I ask how many distinct label sets there are from a Recorder (CONTINUOUS), I have to specify a time window for the question to be meaningful. For an Observer (SNAPSHOT), I can answer the question ("how many distinct label sets") without specifying a time range.

I think this is where I miss something (maybe here is where the disagreement comes from). In both cases for me when I look at the N points (N different label sets) they accumulated (sums) values over that time interval:

  • -------- // start time t0'
  • t0 - new object with location "stack" size 10
  • t1 - new object with location "heap" size 100
  • t2 - delete object with location "stack" size 10
  • t3 - new object with location "stack" size 20
  • -------- // exporting t3'
  • t4 - delete object with location "stack" size 20
  • t5 - new object with location "stack" size 30
  • t6 - delete object with location "heap" size 100
  • t7 - new object with location "heap" size 200
  • -------- // exporting t7'
  • t8 - delete object with location "stack" size 30
  • t9 - delete object with location "heap" size 200
  • -------- // exporting t9'

First Metric:

  • first export (start_time= t0'; end_time = t3' - "location"="stack" 20; "location"="heap" 100)
  • second export (start_time= t0'; end_time = t7' - "location"="stack" 30; "location"="heap" 200)
  • third export (start_time= t0'; end_time = t9' - "location"="stack" 0; "location"="heap" 0)

Second Metric:

  • first export (start_time= t0'; end_time = t3' - "location"="stack" 20; "location"="heap" 100)
  • second export (start_time= t0'; end_time = t7' - "location"="stack" 30; "location"="heap" 200)
  • third export (start_time= t0'; end_time = t9' - "location"="stack" 0; "location"="heap" 0)

In both cases we have the same number of unique label sets, and we cannot assign a specific moment when a value for a specific label set was updated, because that information is lost during aggregation, the only thing we know is that in the interval t0' - tx' we had this number of distinct label sets and these are the "current values" corresponding to every label set.

To get a bit more confused, even for the "second metric" we actually read the value at one specific moment "exporting time", so why is that different? I think once we left the library by adding the data to the wire, we see these metrics exactly the same, inside the library there may be some interpretations like the one you had, but once we put the data on the wire they are identical to me.

Copy link
Member

@bogdandrutu bogdandrutu Jul 28, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I misunderstood this:

In all cases, the presence of StartTimeUnixNano != 0 indicates that the input data spanned a period of time as opposed to being defined at an instant where StartTimeUnixNano == TimeUnixNano.

I am trying again to see if that makes sense:

  1. There are some Observers where we know the start time, for example a SumObserver may have the start time equal with the start of the process, and in this case their data will look exactly the same as the "UpDownCounter" example. In this case the metric is considered "CONTINUOUS".
  2. There are some Observers where we cannot determine the start time correctly, even for a SumObserver, in this case the collected points will miss the start_time and hence they will be considered "SNAPSHOT".

If this is the example that you have in mind, I completely agree that this is a possibility in the real world, but this makes me to think that Observers especially "SumObserver" and "UpDownSumObserver" need to support a configuration where users will specify the start_time (or just a boolean to say if it is equal to the start of the process).

enum Continuity {
// INVALID_CONTINUITY is the default Continuity, it MUST not be
// used.
INVALID_CONTINUITY = 0;

// CONTINUOUS implies the event originated from the application
// calling the SDK with a measurement and possibly a tracing
// context.
CONTINUOUS = 1;

// SNAPSHOT may be set for any kind of metric, indicating it
// was generated through a callback invoked deliberately by the
// SDK. In SNAPSHOT measurements, TimeUnixNano values depend
// on the SDK's decision to collect, not the application.
//
// SNAPSHOT measurements may be used to define point-in-time
// ratios, as data points from the same callback execution MUST
// share a single logical timestamp.
SNAPSHOT = 2;
}

// temporality is the Temporality of values this metric has.
Temporality temporality = 5;

// structure is the Structure of values this metric has.
Structure structure = 6;

// continuity is the Continuity of values this metric has.
Continuity continuity = 7;

// All 12 combinations of Temporality, Structure, and Coninuity
// describe a meaningful expression of metric data. Users familiar
// with the user-facing OpenTelemetry Metrics API will recognize
// these 12 values as comprised of 6 kinds of instrument combined
// with 2 values for temporality. Structure and Continuity form the
// semantic basis of the various metric instruments, corresponding
// with the API Specification, while Temporality is an orthogonal
// concept. The instrument mapping:
//
// Instrument Structure Continuity
// ----------------------------------------------
// Counter ADDING_MONOTONIC CONTINUOUS
// UpDownCounter ADDING CONTINUOUS
// ValueRecorder GROUPING CONTINUOUS
// SumObserver ADDING_MONOTONIC SNAPSHOT
// UpDownSumObserver ADDING SNAPSHOT
// ValueObserver GROUPING SNAPSHOT
//
// Several observations help simplify our understanding of
// Temporality and Structure.
//
// About Temporality:
//
// DELTA and CUMULATIVE temporalities are practically identical in
// interpretation, though they differ in intention. In both
// cases, the data point represents part or all of the data
// collected across an interval of time. The CUMULATIVE kind
// indicates that the client does not "reset" and that
// StartTimeUnixNano will be held as a constant across data points
// for a single process.
//
// About Structure:
//
// ADDING_MONOTONIC and ADDING structures are practically
// identical in form, but commonly differ in interpretation. In
// both cases, individual data points are meant to be combined
// into a sum. In both cases, knowing the Metric is a sum
// indicates the variable may be monitored as a rate of change.
// In the monotonic case, the sum has a useful non-negative rate.
//
// GROUPING structure data points represent individual
// measurements, meant to be combined into a distribution. The
// sum of these data may be meaningful, but individual
// measurements are the focus of these metrics.
//
// About Continuity:
//
// CONTINUOUS kind data points result directly from
// application-level events, therefore the count of events defines
// a meaningful rate to the user. Identical CONTINUOUS data
// points (with the same Resource and TimeUnixNano) are considered
// conincidental.
//
// SNAPSHOT kind data points result from callbacks invoked by the
// SDK, therefore are called in SDK context (i.e., not traced) and
// the rate of measurements does not define a meaningful rate to
// the user. Identical SNAPSHOT data points (with the same
// Resource and TimeUnixNano) are considered invalid, as only one
// Snapshot can be taken per instant per SDK. Measurements of a
// SNAPSHOT metric are recorded for all label sets at the same
// logical time, therefore can be used to calculate meaningful
// point-in-time ratios.
}

// Int64DataPoint is a single data point in a timeseries that describes the time-varying
Expand All @@ -251,19 +359,19 @@ message Int64DataPoint {
// The set of labels that uniquely identify this timeseries.
repeated opentelemetry.proto.common.v1.StringKeyValue labels = 1;

// start_time_unix_nano is the time when the cumulative value was reset to zero.
// This is used for Counter type only. For Gauge the value is not specified and
// defaults to 0.
//
// The cumulative value is over the time interval (start_time_unix_nano, time_unix_nano].
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
// StartTimeUnixNano is the moment when aggregation began over a
// period of time to compute the data point. When this value is
// omitted it implies no aggregation was applied, in which case the
// start time may also be considered equal to TimeUnixNano.
//
// Value of 0 indicates that the timestamp is unspecified. In that case the timestamp
// may be decided by the backend.
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 start_time_unix_nano = 2;

// time_unix_nano is the moment when this value was recorded.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 time_unix_nano = 3;

// value itself.
Expand All @@ -276,19 +384,19 @@ message DoubleDataPoint {
// The set of labels that uniquely identify this timeseries.
repeated opentelemetry.proto.common.v1.StringKeyValue labels = 1;

// start_time_unix_nano is the time when the cumulative value was reset to zero.
// This is used for Counter type only. For Gauge the value is not specified and
// defaults to 0.
// StartTimeUnixNano is the moment when aggregation began over a
// period of time to compute the data point. When this value is
// omitted it implies no aggregation was applied, in which case the
// start time may also be considered equal to TimeUnixNano.
//
// The cumulative value is over the time interval (start_time_unix_nano, time_unix_nano].
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// Value of 0 indicates that the timestamp is unspecified. In that case the timestamp
// may be decided by the backend.
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 start_time_unix_nano = 2;

// time_unix_nano is the moment when this value was recorded.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 time_unix_nano = 3;

// value itself.
Expand All @@ -302,18 +410,17 @@ message HistogramDataPoint {
// The set of labels that uniquely identify this timeseries.
repeated opentelemetry.proto.common.v1.StringKeyValue labels = 1;

// start_time_unix_nano is the time when the cumulative value was reset to zero.
// StartTimeUnixNano is the moment when aggregation began over a
// period of time to compute the data point. This MUST be set.
//
// The cumulative value is over the time interval (start_time_unix_nano, time_unix_nano].
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// Value of 0 indicates that the timestamp is unspecified. In that case the timestamp
// may be decided by the backend.
// Note: this field is always unspecified and ignored if MetricDescriptor.type==GAUGE_HISTOGRAM.
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 start_time_unix_nano = 2;

// time_unix_nano is the moment when this value was recorded.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 time_unix_nano = 3;

// count is the number of values in the population. Must be non-negative. This value
Expand Down Expand Up @@ -396,17 +503,17 @@ message SummaryDataPoint {
// The set of labels that uniquely identify this timeseries.
repeated opentelemetry.proto.common.v1.StringKeyValue labels = 1;

// start_time_unix_nano is the time when the cumulative value was reset to zero.
// StartTimeUnixNano is the moment when aggregation began over a
// period of time to compute the data point. This MUST be set.
//
// The cumulative value is over the time interval (start_time_unix_nano, time_unix_nano].
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// Value of 0 indicates that the timestamp is unspecified. In that case the timestamp
// may be decided by the backend.
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 start_time_unix_nano = 2;

// time_unix_nano is the moment when this value was recorded.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// The value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on
// 1 January 1970.
fixed64 time_unix_nano = 3;

// The total number of recorded values since start_time. Optional since
Expand Down