-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pkg/stanza] container: remove time field from attributes after parsing #33389
Comments
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
…logs (#33946) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This PR adds support for removing the original `time` attribute/field from the parsed log record as it was suggested at #33389. Users should use the `Timestamp` field which holds the parsed time. This patch introduces this change behind a feature flag called `filelog.container.removeOriginalTimeField` which is disabled by default following the [feature lifecycle](https://github.com/open-telemetry/opentelemetry-collector/tree/main/featuregate#feature-lifecycle). **Link to tracking Issue:** <Issue number if applicable> #33389 **Testing:** <Describe what testing was performed and which tests were added.> 1. Added unit-test 2. Run with `./bin/otelcontribcol_linux_amd64 --config container_config.yaml --feature-gates=filelog.container.removeOriginalTimeField ` and verify the output: ```console 2024-07-08T14:26:50.936+0300 info LogsExporter {"kind": "exporter", "data_type": "logs", "name": "debug", "resource logs": 1, "log records": 2} 2024-07-08T14:26:50.936+0300 info ResourceLog #0 Resource SchemaURL: Resource attributes: -> k8s.namespace.name: Str(some) -> k8s.pod.name: Str(kube-controller-kind-control-plane) -> k8s.container.restart_count: Str(1) -> k8s.pod.uid: Str(49cc7c1fd3702c40b2686ea7486091d6) -> k8s.container.name: Str(kube-controller) ScopeLogs #0 ScopeLogs SchemaURL: InstrumentationScope LogRecord #0 ObservedTimestamp: 2024-07-08 11:26:50.836517638 +0000 UTC Timestamp: 2029-03-30 08:31:20.545192187 +0000 UTC SeverityText: SeverityNumber: Unspecified(0) Body: Str(INFO: log line here) Attributes: -> logtag: Str(F) -> log.file.path: Str(/var/log/pods/some_kube-controller-kind-control-plane_49cc7c1fd3702c40b2686ea7486091d6/kube-controller/1.log) -> log.iostream: Str(stdout) Trace ID: Span ID: Flags: 0 ``` **Documentation:** <Describe the documentation added.> Added. --------- Signed-off-by: ChrsMark <chrismarkou92@gmail.com> Co-authored-by: Andrzej Stencel <andrzej.stencel@elastic.co>
Since #33946 was merged with the feature gate, let's keep this issue open to track the graduation of this feature gate. |
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
…ate to beta (#35169) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This PR moves the feature gate to delete the original time field from parsed container logs to beta as part of #33389. **Link to tracking Issue:** <Issue number if applicable> **Testing:** <Describe what testing was performed and which tests were added.> **Documentation:** <Describe the documentation added.> Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
…ate to beta (open-telemetry#35169) **Description:** <Describe what has changed.> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> This PR moves the feature gate to delete the original time field from parsed container logs to beta as part of open-telemetry#33389. **Link to tracking Issue:** <Issue number if applicable> **Testing:** <Describe what testing was performed and which tests were added.> **Documentation:** <Describe the documentation added.> Signed-off-by: ChrsMark <chrismarkou92@gmail.com>
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
Component(s)
pkg/stanza
Is your feature request related to a problem? Please describe.
This issue keeps track of what was suggested at #33353 (comment).
At the moment when the
container
parser parses the logs, it parses the time field moving the result to theTimestamp
.However the original
time
field is preserved in theAttributes
. Example:Describe the solution you'd like
As mentioned at #33353 (comment), it be would reasonable to remove the original field once its parsed.
One thing to consider here is the possibility of this field being used by some users today. This could happen for users that use the existing Helm preset.
Since the
container
parser will be a replacement for this preset we need to take this into account.Maybe we can do this removal through a setting/feature-flag to not aggressively break any users that might use the original time field.
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: