Adds Profiling Signal

[petethepig]

This PR was previously opened in opentelemetry-proto repo. Now it lives in this new opentelemetry-proto-profile repo.

Changelog from previous PR:

• changed things like function_id to function_index for clarity.
• switched to uint32 for indexes
• replaced has_functions, has_filenames, etc with an enum (considering removing altogether)

Relevant documents / previous discussions:

• OTEL Profiling Vision
• Profiling SIG Meeting Notes
• Draft PR in OTEL-collector
• Discussion in the previous PR — many good comments there
• OTEL Profiles Decision Log

---

The overall design is based on/heavily inspired by:

• other OTEL signals
• Felix Geisendörfer's draft for a profiles proto
• Anton Kolesnikov's draft for a profiles proto
• pprof proto

This PR consists of 2 parts:

• The profiles.proto and profiles_service.proto which are mostly boilerplate and more or less finished from my perspective
• 3 alternative Profile implementations, still very much work-in-progress
  • pprof — included as a reference and as a baseline for future benchmarks
  • denormalized — most messages are embedded, no string table, a lot of potential for duplication => easier to work with, potentially larger payloads
  • normalized — most messages are linked using a foreign key, includes a string table, not much duplication => harder to work with, potentially smaller payloads

Things that are currently supported:

• support for OTEL-compatible attributes (labels) on Profile and Samples level. Meaning you can label both whole profiles and individual stack traces
• linking with traces. TODO: do we need linking with other signals?
• support for multiple profile types in one Profile (similar to how it's implemented in pprof and used in go memory profiles via multiple value values)
• ability to specify AggregationTemporality for each SampleType (e.g something like alloc_space in go is cumulative)
• optional per-sample timestamps
• support for embedded original profile (original_payload field)

Design TODOs / Areas where feedback would be most valuable:

• The biggest thing is we need to decide which one of these approaches (denormalized / normalized / mix of the two) we're going to take. I personally think that the optimal end product should be closer to normalized implementation, especially considering that on-the-wire efficiency is a particularly important part of OTEL's profiling vision. But we still need to do some benchmarks to confirm that that would actually be significantly more efficient. going with normalized.
• Figure out what to do with pprof messages like Mapping — I'm not sure if we need those keep
• Improve SampleType message. Currently Type and Unit are strings. We might want to turn them into enums, or standardize them some other way, e.g making those a part of OTEL semantic conventions
• Things I'm forgetting / not thinking about

Implementation TODOs / next steps

• get a corpus of some real production profiling data
• run benchmarks with this corpus of data -> fine-tune various aspects of the format
• implement a few simple OTEL processors -> fine-tune various aspects of the format
• create a reference implementation for a client -> fine-tune various aspects of the format
• create a reference implementation for a server -> fine-tune various aspects of the format
• verify that the spec works with those reference implementations -> fine-tune various aspects of the format

Reference Diagrams

For reference, here are graph representations of other signals:

metrics

logs

traces

---

And these are graph representations of alternatives formats under consideration:

pprof for reference

denormalized

normalized

[florianl]

just a quick look 🙈

[florianl]

Instead of SymbolicInfo we maybe should use repeated Label here. Similar to the original discussion open-telemetry#488 (comment)

[florianl]

Can we add some

Suggested change

	uint32 filename_index = 4; // Index into string table
	File file = 4;

and later on :

message File {
uint32 filename_index = 1;  // Index into string table
  // A unique identifier of a file. The ID is a 16-byte array.
  bytes file_id = 2;
}

Filename are usually less unique. And to combine it, message Profile could be extended to hold a repeated File file_tabe, so that different locations can reference it.

[florianl]

Suggested change

	// A unique identifier for the linked span. The ID is an 8-byte array.
	// A unique identifier for the linked span. The ID is an 16-byte array.

Similar to the trace_id a 16 byte array should be preferred over a 8-byte array for global uniqueness.

[florianl]

For easier handling, maybe we could use an enum here?

Suggested change

	string type = 3;
	enum SamplingType {
	SAMPLING_TYPE_UNSPECIFIED = 0;
	SAMPLING_TYPE_CPU = 1;
	[...]
	}
	[..]
	SamplingType type = 3;

[petethepig]

could potentially be better if inlined

[aalexand]

nit: perhaps stack_traces? as two words

[aalexand]

Maybe not. "stacktrace" is fine as a term, I actually like it more. As long as we are consistent in using it as one word :)

[aalexand]

This field is fairly exotic. I'd drop it for now.

[aalexand]

As discussed in the last sync, it's unclear whether we want to start with tracking all these symbolization options from the start.

Aside, I also find "info" a bit too generic, so I'd name this enum SymbolFidelity if we decide to keep it.

And then in order to simplify it, I also mentioned in the sync that maybe we should start with just

enum SymbolFidelity {
   SYMBOL_FIDELITY_UNSPECIFIED = 0;
  SYMBOL_FIDELITY_FULL = 1;
}

But one downside of this is that given this choices, producers will likely start to put the FULL value when any symbolic info is present.

Given that, I wonder if we should start with a simple bool is_symbolized field and once we encounter a real need for carrying fidelity information, we add it then. Possibly by adding the separate SymbolFidelity enum field - I don't see much problem with having two fields where one is just a yes/no boolean and the other one "zooms" into the details.