diff --git a/specification/metrics/sdk.md b/specification/metrics/sdk.md index b65ee88a895..d6e8b520eb5 100644 --- a/specification/metrics/sdk.md +++ b/specification/metrics/sdk.md @@ -88,6 +88,9 @@ These are the significant data types used in the model architecture: - **ExportRecord**: consists of Instrument, Label Set, Resource, Timestamp(s), and Aggregation - **ExportRecordSet**: a set of export records. +TODO(jmacd): rename ExportKind to AggregationTemporality, +ExportKindSelector to AggregationTemporalitySelector. + The term **SDK instrument** refers to the underlying implementation of an instrument. @@ -228,18 +231,33 @@ Accumulator, with detail shown for synchronous instruments. ![Metrics SDK Accumulator Detail Diagram](img/accumulator-detail.png) -For a synchronous instrument, the Accumulator will: +The Accumulator's primary tasks are to aggregate synchronous metric +events over a collection interval, and then at end of the interval, to +evaluate asynchronous callbacks and finally snapshot current +Aggregators for passing to the Processor. + +The Accumulator MUST be configured with an AggregatorSelector +interface that is used to assign new Aggregators to instruments as +they are needed. -1. Map each active Label Set to a record, consisting of two instances of the same type Aggregator -2. Enter new records into the mapping, calling the AggregationSelector if needed -3. Update the current Aggregator instance, responding to concurrent API events -4. Call Aggregator.SynchronizedMove on the current Aggregator instance to: (a) copy its value into the snapshot Aggregator instance and (b) reset the current Aggregator to the zero state -5. Call Processor.Process for every resulting Accumulation (i.e., Instrument, Label Set, Resource, and Aggregator snapshot) +The Accumulator MUST ensure that metric events for a given instrument +and identical label set that occur within a single collection interval +are passed to the same Aggregator. -The Accumulator MUST provide the option to associate a -[`Resource`](../resource/sdk.md) with the Accumulations that it +The Accumulator MUST snapshot the current value and reset the state of +every Aggregator that was used during a collection interval. The +Aggregator snapshot, together with the instrument descriptor, label +set, and Resource, define an Accumulation and are passed to the +Processor. + +The Accumulator MUST provide the option to configure the +[`Resource`](../resource/sdk.md) associated with Accumulations that it produces. +Aggregators that are not used during a collection interval MUST not +yield Accumulations for that collection interval, when no events or +observations happen. + Synchronous metric instruments are expected to be used concurrently. Unless concurrency is not a feature of the source language, the SDK Accumulator component SHOULD be designed with concurrent performance @@ -250,6 +268,18 @@ synchronous instrument updates. The Accumulator SHOULD NOT hold an exclusive lock while calling an Aggregator (see below), since some Aggregators may have higher concurrency expectations. +The Accumulator MUST eliminate from memory (i.e. "forget") state +associated with label sets used in earlier collection intervals, after +they are not for a suitable amount of time. A "suitable amount of +time" is intentionally not specific, since implementations may wish to +optimize memory management and have to contend with concurrent access. +This requirement ensures that export pipelines constructed for +stateless exporters (e.g. Statsd, OTLP with a stateless +ExportKindSelector) are not forced into the use of permanent state in +the Accumulator. This implies that the use of long-term state in a +Metrics export pipeline should be elective, and such state if present +should be managed in the Processor component. + #### Accumulator: Collect() function The Accumulator MUST implement a Collect method that builds and @@ -268,9 +298,77 @@ Label Set, Resource, and metric Descriptor. TODO: _Are there more Accumulator functional requirements?_ -### Processor +### Processor: Component interface + +The Processor component interface supports interaction from the +Controller and Accumulator. The Controller, responsible for +initiating a new round of collection, informs the Processor when a +collection interval starts and finishes. After finishing the +collection interval, the Controller gets the ExportRecordSet before +calling the Exporter to export data. + +The Accumulator interacts with the Processor during the call to its +`Collect()` method, during which it calls `Process()` once per +ExportRecord on the Processor. + +The Processor component is meant to be used for managing long-term +state; it is also one of the locations in the Metrics export pipeline +where we can impemlement control over cardinality. There are two +reasons that long-term state is typically required in a Metric export +pipeline: + +1. Because the Exporter requests Cumulative aggregation temporality for Sum and/or Histogram data points +2. Because the Exporter requests keeping Memory about all metric label sets, regardless of the requested aggregation temporality. + +Note that both of these behaviors are typically required for a +Prometheus exporter and that when none of these behaviors are +configured, the Metrics export pipeline can be expected not to develop +long-term state. + +#### Basic Processor + +The basic Processor supports two standard ExportKindSelectors and the +independent Memory behavior described above. The default +OpenTelemetry Metrics SDK MUST provide a basic Processor meeting these +requirements. + +##### Basic Processor: CumulativeExportKindSelector + +CumulativeExportKindSelector is the default behavior, which requests +exporting Cumulative aggregation temporality for Sums and Histograms +and implies that label sets used with synchronous instruments will be +remembered indefinitely in the SDK. This ExportKindSelector is the +default in order support downstream Prometheus exporters "out of the +box". + +##### Basic Processor: StatelessExportKindSelector + +The StatelessExportKindSelector configures a Metric export pipeline +with no long-term memory requirements. In this selector, the Counter, +UpDownCounter, ValueRecorder, and ValueObserver instruments are +configured for Delta aggregation temporality while SumObserver and +UpDownSumObserver instruments are configured for Cumulative +aggregation temporality. This basic Processor configuration has no +long-term memory requirements because the instrument temporality +matches the aggregation temporality, meaning Accumulations "pass +through" the Processor without requiring additional memory for +temporality conversion. + +##### Basic Processor: Memory + +Some metrics exporter configurations request that the Metric export +pipeline maintain long-term state about historically reported Metric +timeseries. This option is a simple boolean that, when set, requires +the Processor to retain memory about all timeseries it has ever +exported. This option is only meaningful when reporting Cumulative +aggregation temporality. + +#### Reducing Processor -TODO _Processor functional requirements_ +The reducing Processor is a Processor interface implementation used in +conjunction with another (e.g., basic) Processor to drop labels in a +Metric export pipeline. The default OpenTelemetry SDK SHOULD provide +a Reducing Processor implementation. ### Controller