Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High cardinality for HTTP span names #2067

Closed
felixbarny opened this issue Apr 30, 2021 · 3 comments
Closed

High cardinality for HTTP span names #2067

felixbarny opened this issue Apr 30, 2021 · 3 comments
Labels
agent-nodejs Make available for APM Agents project planning.
Milestone

Comments

@felixbarny
Copy link
Member

I've noticed deviations from the HTTP span spec.

The Node.js agent sets the span.name to the full URL path, such as GET opbeans:3000/api/products/top. This leads to the span.name having a high cardinality. That's an issue as we'd like to create metrics that have the span name as a dimension. However, it's unclear whether we'd roll up metrics for http spans in the first iteration, or only for db spans.

Also, the properties context.http.url and context.http.status_code are not populated.

@github-actions github-actions bot added the agent-nodejs Make available for APM Agents project planning. label Apr 30, 2021
@AlexanderWert AlexanderWert modified the milestones: 7.14, 7.15 Jun 1, 2021
astorm added a commit that referenced this issue Jul 19, 2021
* fix: span name should comply with spec

#2067
@astorm
Copy link
Contributor

astorm commented Jul 20, 2021

A fix for the high cardinality in http span names has been merged.

We're unable to reproduce the following behavior

Also, the properties context.http.url and context.http.status_code are not populated.

That is -- we do appear to be setting these values. Moving ticket to blocked until we can get a repro/description of the context behavior.

@astorm
Copy link
Contributor

astorm commented Jul 26, 2021

Re: the missing context.http.url and context.http.status_code -- when using an up to date agent and APM Server 7.12, the values from these fields end up populating the span.http object in the final Elasticsearch document.

screenshot of span.http.* fields

Per this morning's planning meeting, the edge instance that didn't have these fields appears to be running APM Server 8.0 -- you can tell by both the _index field and the observer.version field.

old-apm-server-2

old-apm-server-1

It looks like the APM Server code that handles this transformation didn't land until 7.10.

In other words -- the agent is fulfilling its contract by setting these fields and modern versions of APM Server are correctly transforming them. The reason these fields are missing from the edge instance is an old version of APM server is shipping this data.

Moving this back into blocked for the time being.

@astorm
Copy link
Contributor

astorm commented Aug 3, 2021

Also, the properties context.http.url and context.http.status_code are not populated.

Per research done, this appears to have been an edge ingest issue. Current versions of the agent are setting these properties. Closing this out as done for now, but will reopen if more evidence comes forward.

@astorm astorm closed this as completed Aug 3, 2021
dgieselaar pushed a commit to dgieselaar/apm-agent-nodejs that referenced this issue Sep 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-nodejs Make available for APM Agents project planning.
Projects
None yet
Development

No branches or pull requests

3 participants