From c112068b72a7330046072fe704caba0b6bd449fd Mon Sep 17 00:00:00 2001 From: Chris Kleinknecht Date: Mon, 21 Oct 2019 18:51:59 -0700 Subject: [PATCH 1/5] Flesh out ProbabilitySampler details --- specification/sdk-tracing.md | 63 ++++++++++++++++++++++++++++++++++-- 1 file changed, 61 insertions(+), 2 deletions(-) diff --git a/specification/sdk-tracing.md b/specification/sdk-tracing.md index a66dcc3f015..9d748a115e0 100644 --- a/specification/sdk-tracing.md +++ b/specification/sdk-tracing.md @@ -122,8 +122,67 @@ These are the default samplers implemented in the OpenTelemetry SDK: #### Probability Sampler algorithm -TODO: Add details about how the probability sampler is implemented as a function -of the `TraceID`. +The `ProbabilitySampler` makes a determistic sampling decision for each span +based on the span's trace ID. It samples a fixed percentage of all spans +following a user-supplied sampling rate. + +According to the [W3C Trace Context +spec](https://www.w3.org/TR/trace-context/#trace-id), vendor-supplied trace IDs +may include both random and non-random components. To avoid sampling based on +the non-random component, the sampler should consider only the leftmost portion +(i.e. some number of high-order bits) of the trace ID. Implementations MAY +allow the user to configure the number of bits of the trace ID that the sampler +considers. + +The `ProbabilitySampler` should generally only sample traces with trace IDs +less than a certain value. This value can be computed from the sampling rate, +and implementations may choose to cache it. The sampling decision should follow +the formula: + +``` +to_sample = traceID >> (128 - precision) < round(rate * 2^precision) +``` + +Where: + +- `rate` is the sampling rate, in `[0.0, 1.0]` +- `precision`: the number of bits of the trace ID to consider, in `[1, 128]` +- `traceID`: is the integer trace ID of the span to be created, assuming + big-endian byte order +- `round(float f)`: is a function that rounds `f` to the nearest integer + +Note that the effective sampling rate is the number closest to `rate` than can +be expressed as a multiple of `2^-precision`. As a consequence, it's not +possible to set arbitrarily low sampling rates, even on platforms that support +arbitrary-precision arithmetic. + +A `ProbabilitySampler` with rate `0.0` MUST NOT choose to sample any traces, +even those with trace ID `0x0`. Similarly, a `ProbabilitySampler` with rate +`1.0` MUST choose to sample all traces, even those with trace ID `2^precision - +1`. + +**Example:** + +Consider a ProbabilitySampler with rate `.25` and 16 bit precision. + +First, find the lowest truncated trace ID that will not be sampled. This number +represents the 25th percentile of the range of possible values: + +``` +.25 * 2^16 = 0x4000 +``` + +Make a sampling decision for a given trace by comparing this number to the +leftmost 16 bits of its 128 bit trace ID: + +``` +x = traceID >> (128 - 16) +to_sample = x < 0x4000 +``` + +This sampler should sample traces with truncated trace ID in the range +`[0x0000, 0x4000)`. Assuming `x` is uniformly distributed over `[0x0000, +0xffff]`, this should account for 25% of all traces. ## Tracer Creation From 46eaf408e4eb008e79017399a9cfa56aee636d49 Mon Sep 17 00:00:00 2001 From: Chris Kleinknecht Date: Tue, 22 Oct 2019 12:32:39 -0700 Subject: [PATCH 2/5] s/^/pow/ --- specification/sdk-tracing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/sdk-tracing.md b/specification/sdk-tracing.md index 9d748a115e0..f8fafd5e0a5 100644 --- a/specification/sdk-tracing.md +++ b/specification/sdk-tracing.md @@ -140,7 +140,7 @@ and implementations may choose to cache it. The sampling decision should follow the formula: ``` -to_sample = traceID >> (128 - precision) < round(rate * 2^precision) +to_sample = traceID >> (128 - precision) < round(rate * pow(2, precision)) ``` Where: From 9223a28dc6be6ab0a3ca7aa9dbb3c136d2c0e21c Mon Sep 17 00:00:00 2001 From: Chris Kleinknecht Date: Tue, 22 Oct 2019 12:41:47 -0700 Subject: [PATCH 3/5] Don't use 0x0 as trace ID example --- specification/sdk-tracing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specification/sdk-tracing.md b/specification/sdk-tracing.md index f8fafd5e0a5..e8f735ab837 100644 --- a/specification/sdk-tracing.md +++ b/specification/sdk-tracing.md @@ -157,9 +157,9 @@ possible to set arbitrarily low sampling rates, even on platforms that support arbitrary-precision arithmetic. A `ProbabilitySampler` with rate `0.0` MUST NOT choose to sample any traces, -even those with trace ID `0x0`. Similarly, a `ProbabilitySampler` with rate -`1.0` MUST choose to sample all traces, even those with trace ID `2^precision - -1`. +even if the leftmost precision-many bits of trace ID are all `0`. Similarly, a +`ProbabilitySampler` with rate `1.0` MUST choose to sample all traces, even if +the leftmost precision-many bits of trace ID trace ID are all `1`. **Example:** From aa53399030e4b0a3b2a8ca59a511c5025eda0e7d Mon Sep 17 00:00:00 2001 From: Chris Kleinknecht Date: Tue, 22 Oct 2019 14:27:02 -0700 Subject: [PATCH 4/5] Misc style and typo fixes --- specification/sdk-tracing.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/specification/sdk-tracing.md b/specification/sdk-tracing.md index e8f735ab837..c27e0959eba 100644 --- a/specification/sdk-tracing.md +++ b/specification/sdk-tracing.md @@ -130,9 +130,8 @@ According to the [W3C Trace Context spec](https://www.w3.org/TR/trace-context/#trace-id), vendor-supplied trace IDs may include both random and non-random components. To avoid sampling based on the non-random component, the sampler should consider only the leftmost portion -(i.e. some number of high-order bits) of the trace ID. Implementations MAY -allow the user to configure the number of bits of the trace ID that the sampler -considers. +of the trace ID. Implementations MAY allow the user to configure the number of +bits of the trace ID that the sampler considers. The `ProbabilitySampler` should generally only sample traces with trace IDs less than a certain value. This value can be computed from the sampling rate, @@ -151,19 +150,20 @@ Where: big-endian byte order - `round(float f)`: is a function that rounds `f` to the nearest integer -Note that the effective sampling rate is the number closest to `rate` than can +Note that the effective sampling rate is the number closest to `rate` that can be expressed as a multiple of `2^-precision`. As a consequence, it's not possible to set arbitrarily low sampling rates, even on platforms that support arbitrary-precision arithmetic. A `ProbabilitySampler` with rate `0.0` MUST NOT choose to sample any traces, -even if the leftmost precision-many bits of trace ID are all `0`. Similarly, a -`ProbabilitySampler` with rate `1.0` MUST choose to sample all traces, even if -the leftmost precision-many bits of trace ID trace ID are all `1`. +even if the leftmost precision-many bits of the trace ID are all `0`. +Similarly, a `ProbabilitySampler` with rate `1.0` MUST choose to sample all +traces, even if the leftmost precision-many bits of the trace ID trace ID are +all `1`. **Example:** -Consider a ProbabilitySampler with rate `.25` and 16 bit precision. +Consider a `ProbabilitySampler` with rate `.25` and 16 bit precision. First, find the lowest truncated trace ID that will not be sampled. This number represents the 25th percentile of the range of possible values: From 3b3d321865cf46364bdfb292c179b6444dc96bf9 Mon Sep 17 00:00:00 2001 From: Chris Kleinknecht Date: Wed, 23 Oct 2019 16:12:22 -0700 Subject: [PATCH 5/5] Make trace ID byte array explicit --- specification/sdk-tracing.md | 58 ++++++++++++++++++++++++++---------- 1 file changed, 42 insertions(+), 16 deletions(-) diff --git a/specification/sdk-tracing.md b/specification/sdk-tracing.md index c27e0959eba..2b256677ac1 100644 --- a/specification/sdk-tracing.md +++ b/specification/sdk-tracing.md @@ -131,24 +131,51 @@ spec](https://www.w3.org/TR/trace-context/#trace-id), vendor-supplied trace IDs may include both random and non-random components. To avoid sampling based on the non-random component, the sampler should consider only the leftmost portion of the trace ID. Implementations MAY allow the user to configure the number of -bits of the trace ID that the sampler considers. +bits of the trace ID that the sampler considers. We define this number as +the sampler's *precision*. -The `ProbabilitySampler` should generally only sample traces with trace IDs -less than a certain value. This value can be computed from the sampling rate, -and implementations may choose to cache it. The sampling decision should follow -the formula: +A trace ID is a 16-byte array. We define the *truncated trace ID* to be the +leftmost precision-many bits of the trace ID: ``` -to_sample = traceID >> (128 - precision) < round(rate * pow(2, precision)) +truncated_trace_id = traceID[0:ceil(precision / 8))] ``` Where: +- `precision` is the number of bits of the trace ID to consider, in `[1, 128]` +- `ceil(float f)` returns the smallest integer greater than `f` +- `a[l:h]` is the slice operator: it returns the subsequence of `a` from index + `l` to `h` inclusive + +This is equivalent to converting the trace ID to an unsigned integer assuming +big-endian byte order, and shifting to remove the unused bits: + +``` +truncated_trace_id = (int) traceID >> (128 - precision) +``` + +The `ProbabilitySampler` should only sample traces with truncated trace IDs +less than a certain value. We define this value as the sampler's *bound*: + +``` +bound = round(rate * pow(2, precision)) +``` + +Where: + +- `round(float f)` rounds `f` to the nearest integer - `rate` is the sampling rate, in `[0.0, 1.0]` -- `precision`: the number of bits of the trace ID to consider, in `[1, 128]` -- `traceID`: is the integer trace ID of the span to be created, assuming - big-endian byte order -- `round(float f)`: is a function that rounds `f` to the nearest integer +- `pow(a, b)` is the exponentiation operator: `a` to the power of `b` + +Note that this value doesn't depend on the trace to be sampled. Implementations +should generally compute this once, not on every sampling decision. + +The sampling decision for a trace is given by: + +``` +to_sample = truncated_trace_id < bound +``` Note that the effective sampling rate is the number closest to `rate` that can be expressed as a multiple of `2^-precision`. As a consequence, it's not @@ -156,10 +183,9 @@ possible to set arbitrarily low sampling rates, even on platforms that support arbitrary-precision arithmetic. A `ProbabilitySampler` with rate `0.0` MUST NOT choose to sample any traces, -even if the leftmost precision-many bits of the trace ID are all `0`. -Similarly, a `ProbabilitySampler` with rate `1.0` MUST choose to sample all -traces, even if the leftmost precision-many bits of the trace ID trace ID are -all `1`. +even if every bit of the truncated trace ID is `0`. Similarly, a +`ProbabilitySampler` with rate `1.0` MUST choose to sample all traces, even if +every bit of the the truncated trace ID is `1`. **Example:** @@ -176,8 +202,8 @@ Make a sampling decision for a given trace by comparing this number to the leftmost 16 bits of its 128 bit trace ID: ``` -x = traceID >> (128 - 16) -to_sample = x < 0x4000 +trunc_tid = (int) traceID >> (128 - 16) +to_sample = trunc_tid < 0x4000 ``` This sampler should sample traces with truncated trace ID in the range