Negate condition is not working for multiple attributes in filter processor #34605
Comments
meSATYA
added
bug
|
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
Try increasing the log level to debug the issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor#troubleshooting Maybe the problem is with the first condition. |
|
@meSATYA
is evaluated as
Try:
|
|
@TylerHelmuth that's what he wrote in the OpenTelemetry configuration shared in the first comment. |
|
Then I agree with your assessment to enable debug logging. The other option is to add a debug exporter with This issues often end up being caused by a mismatch between how the user thinks the data looks like and how the collector is actually seeing the data. Likely an attribute is named differently or is located somewhere else in the data hierarchy (like on a resource instead of a span or vise versa). |
|
I enabled debug logging and here is the log snippet for few spans. As you can find here |
|
@meSATYA can you provide a unit tests or a sample of input data that could be used to reproduce this issue. |
|
@TylerHelmuth , sure. Attached the export of the trace from Grafana Tempo where we store these traces. I don't have the input data to be particular as these events are generated from the services where otel is instrumented. Thanks. |
|
I updated the filter to be more constraint about resource attribute or span attribute. But, there is no improvement and there is no event is exported where the attribute platform_env is present whereas we are receiving spans with checkpoint.log and platform_env attribute. Screenshot of the debug logs with |
|
I tried to reproduce the issue with some dummy and smaller trace data. Here is the input data: Here is the filter used: Here is the log output: |
|
Hi @TylerHelmuth @iblancasa, I observed that the filter processor works only with one ottl negate condition, but not multiple. |
|
@meSATYA can you change the value of |
|
@iblancasa, I tried with For example, |
|
@meSATYA in the test scenario you described here the span is not dropped because it has an attribute named If I update the condition to Likewise, when you split the statements into separate statements here, the data will fail the |
|
Hi @TylerHelmuth, no I don't want to drop the span where it holds the attribute To simplify the scenario here, I want to export data where it matches below conditions.
|
|
@meSATYA please supply 2 test inputs like you did here, one that should be dropped and one that should be kept. In your previous test input, the value of processors:
filter/checkpoint-traces-preprod:
error_mode: ignore
traces:
span:
- attributes["checkpoint.log"] != nil and (resource.attributes["platform_env"] != "apr" or resource.attributes["platform.env"] != "approval-eu")Which is correct for the way the statement is written. If your goal is to drop data where processors:
filter/checkpoint-traces-preprod:
error_mode: ignore
traces:
span:
- attributes["checkpoint.log"] != nil and (resource.attributes["platform_env"] == "apr" or resource.attributes["platform.env"] == "approval-eu") |
|
Resolved in #36564. |
Component(s)
processor/filter
What happened?
Description
I am using filter processor to drop the spans which doesn't match to some environments.
For example, I want to drop the spans where below 3 conditions should match.
attributes["checkpoint.log"] == nil and resource.attributes["platform_env"] != "apr" or resource.attributes["platform.env"] != "approval-eu"But, it is not working as expected and also it is also filtering out span where
platform.envisapproval-nar,live-euorlive-naralong with the mentioned environments and it is dropping spans whereresource.attributes["platform_env"] != "apr"Steps to Reproduce
Expected Result
It should only filter out the spans having span attribute
checkpoint.logand the resource attribute matchesplatform_envasaprand the resource attribute matchesplatform.envasapproval-eu.Actual Result
It is filtering out span where
platform.envisapproval-nar,live-euorlive-naralong with the mentioned environments and it is dropping spans whereresource.attributes["platform_env"] != "apr"Collector version
0.102.1
Environment information
Environment
OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
Log output
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: