[cmd/mdatagen] Document metadata.yaml schema

[mx-psi]

Component(s)

cmd/mdatagen

Describe the issue you're reporting

To help developers understand the format used by metadata.yaml, we should document its fields and their meaning in a Markdown file inside cmd/mdatagen. This blocks #20908 IMO.

[github-actions]

Pinging code owners for cmd/mdatagen: @dmitryax. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[DewaldDeJager]

Hi. I'm looking for an issue to get started with contributing to this repo and this issue seems fairly straightforward. I see there's already a YAML file in the mdatagen component that seems to document most of the fields in the comments: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/mdatagen/metric-metadata.yaml

Would this issue entail taking this info, ensuring it's all still relevant and complete, and then adding it to the README in an easy to read format? If so, please assign this issue to me and I'll get started on it.

[mx-psi]

Hi. I'm looking for an issue to get started with contributing to this repo and this issue seems fairly straightforward. I see there's already a YAML file in the mdatagen component that seems to document most of the fields in the comments: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/cmd/mdatagen/metric-metadata.yaml

Would this issue entail taking this info, ensuring it's all still relevant and complete, and then adding it to the README in an easy to read format? If so, please assign this issue to me and I'll get started on it.

Hi @DewaldDeJager! Some fields are missing from the file you linked, see this struct

opentelemetry-collector-contrib/cmd/mdatagen/loader.go

Line 210 in d6c8a06

type metadata struct {

. Specifically, the type field and the fields on the status section (that come from the Status struct) are missing.

The end goal of this issue is for component developers to be able to tell what to fill these fields with. I think we can do the following to achieve that:

1. Add to the metric-metadata.yaml file you linked the fields that are missing, together with a description of what they are
2. Rename metric-metadata.yaml to something more descriptive (metadata-schema.yaml sounds better to me) and update the link on the README to it.
3. Expand the existing README reference to help users understand how to fill in the fields. The file you linked should be a full reference with all the possible fields, but component developers may not need to fill all of them (see e.g. a typical component metadata.yaml), so we want some example with the minimum information needed for a typical component, while still linking to the full schema for reference.

I will assign this to you, feel free to ask more questions if anything is unclear (or just take a stab at it and we can help you in the PR itself!)

[DewaldDeJager]

I've worked on (1) and (2) above and created a draft PR (linked above). I've put questions specific to that on the PR. @mx-psi please could you take a look?

Some more general questions I have:

1. Should we extend the attribute type to contain information on whether an attribute is mandatory/optional what the default value is (if any)? This will go a long way in generating documentation on the attributes if that's where we see this going.
2. The attributes and resource attributes are very similar so are sharing types at the moment but some fields might not make sense to share. For example, enabled might not make sense for attributes and name_override might not make sense for resource attributes (not sure about this one though). Should we split the types or put some other control in place to avoid these fields being added where it doesn't make sense?
3. What is ScopeName on the root metadata type?

Including @atoulme as he has given some helpful feedback on the pull request already.

[DewaldDeJager]

Just noting that we may need to include the syntax for declaring connector stability levels (eg beta: [logs_to_metrics]) introduced in this PR (Currently a draft): #22800

[mx-psi]

Some more general questions I have:

I think these deserve separate issues, let's keep this issue only for documenting the existing fields and open separate issues for these improvements. Would you mind opening them @DewaldDeJager ?

Just noting that we may need to include the syntax for declaring connector stability levels (eg beta: [logs_to_metrics]) introduced in this PR (Currently a draft): #22800

Commented on the issue, #22800 (review) (the same comment applies in the other direction!)