Equivalent of opentracing's 'sampling.priority 1' tag?

[VivekSubr]

Hello!

Opentracing had a way to force sample a span, by setting 'sampling.priority' tag to 1 we could force sampling of a particular span by a trace ratio sampler.

Looking at this library, there seems to be no equivalent way? A similar discussion in the golang lib suggests using a custom sampler: open-telemetry/opentelemetry-go#2651

Can something like that be done here? Or some other, better way?

[marcalff]

A SpanContext consist of a trace id, a span id, and trace flags.

See TraceFlags::isSampled().

[VivekSubr]

@marcalff - but there seems to be no way to set that?

[marcalff]

@VivekSubr

The trace flags should typically:

• be set when creating a root span,
• be inherited from the parent span when creating a child span.

See StartSpanOptions::parent

The SpanContext constructor can be invoked with a given trace flag.

[VivekSubr]

@marcalff - thanks for the hints, you meant like this?

    auto ctx = trace::SpanContext(true, false);
    ASSERT_TRUE(ctx.trace_flags().IsSampled());
    
    trace::StartSpanOptions options; options.parent = ctx;
    for(int i=0; i<10; i++)
    {
        std::ostringstream s;
        s << "span" << i;
        auto span = m_tracer->StartSpan(s.str(), {}, {}, options);
        ASSERT_TRUE(span->GetContext().trace_flags().IsSampled());
        span->End();
    }

The ASSERT inside the for failed, looks like it won't take the flag from the parent context.

[lalitb]

@VivekSubr - Do you want to force sample a Span by override the default behavior of the configured sampler? I think you would need a custom sampler to achieve it.

[VivekSubr]

@lalitb - are there any examples of how to write custom samplers?

[esigo]

@VivekSubr please see https://github.com/open-telemetry/opentelemetry-cpp/blob/main/sdk/src/trace/samplers/trace_id_ratio.cc as an example.

[VivekSubr]

Thanks for all the help! I think I'm almost there, just one weird issue with custom sampler - it doesn't seem to receive set attributes? Here's a test case -

TEST_F(TestTracer, Span)
{
    std::unique_ptr<memory::InMemorySpanExporter> exporter(new memory::InMemorySpanExporter());

    auto resource_attributes = opentelemetry::sdk::resource::ResourceAttributes
    {
        {"service.name", "Test"},
        {"service.instance.id", "Test-12"}
    };
    auto resource = opentelemetry::sdk::resource::Resource::Create(resource_attributes);
    
    std::shared_ptr<memory::InMemorySpanData> span_data = exporter->GetData();
    auto sampler = std::unique_ptr<CustomSampler>(new CustomSampler(0.1));
    auto processor = std::unique_ptr<trace_sdk::SpanProcessor>(new trace_sdk::SimpleSpanProcessor(std::move(exporter)));
    std::vector<std::unique_ptr<trace_sdk::SpanProcessor>> v; v.push_back(std::move(processor));
    auto m_tracer_ctx = std::make_shared<trace_sdk::TracerContext>(
                              std::move(v), 
                              resource, 
                              std::move(sampler));
        
    auto m_trace_provider = std::shared_ptr<trace::TracerProvider>(new trace_sdk::TracerProvider(m_tracer_ctx));
    auto m_tracer = m_trace_provider->GetTracer("TestTracer", "1.0.0");

    trace::StartSpanOptions options; 
    for(int i=0; i<10; i++)
    {
        std::ostringstream s;
        s << "span" << i;
        auto span = m_tracer->StartSpan(s.str(), {}, {}, options);
        span->SetAttribute("ForceSampling", true);
        span->End();
    }

    ASSERT_EQ(span_data->GetSpans().size(), 10);
}

And here's my sampler: https://github.com/VivekSubr/Client-Server/blob/main/opentelemetry/jaeger/sampler.h

I can see that attributes recieved by 'ShouldSample' is always size 0, it should have picked by the SetAttribute one right?

[VivekSubr]

Yeah, since it has be done with startSpan...

https://github.com/open-telemetry/opentelemetry-cpp/blob/59e7496f94a0e1c3f8daa087dd3571c81ad5f92c/sdk/src/trace/tracer.cc

Not really a replacement for the sampling.priorty tag, since that could be done any any time.

[lalitb]

Yeah, since it has be done with startSpan...

Yes that's correct, the sampling decision is taken during span creation and the span attributes are not populated at that time.

[lalitb]

Making sampling decision through span attribute - this is not possible as of now, and not supported as per the specs.

• There is ongoing specs PR to enable/disable tracing based on a Boolean flag set in the context (add SuppressTracing flag opentelemetry-specification#3103).
• otel-collector allows adding tail based samplers, which can be used to filter spans at the collector end. Tail based samplers are executed at processor level, and so can be used to take sampling decision based on span values (https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor).

[lalitb]

@VivekSubr - Do you have further comments here, or if we can close this issue as something not supported in specs.

[VivekSubr]

No, thanks, closing.