Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_loki: drop quotes after drop_single_key #4353

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mifrost
Copy link

@mifrost mifrost commented Nov 19, 2021

Removes quotes from log lines when line_format=json and drop_single_key=true

Resolves concerns raised in #3005


Testing

  • Example configuration file for the change
[OUTPUT]
    Name              loki
    ....
    label_keys        $filename
    line_format       json
    remove_keys       filename
    Drop_single_key   true
  • Debug log output from testing the change
    Before:
    "Nov 19 15:48:36 hostname telegraf: 2021-11-19T15:48:36Z D! [outputs.prometheus_client] Wrote batch of 82 metrics in 648.331µs"
    After:
    Nov 19 16:05:36 hostname telegraf: 2021-11-19T16:05:36Z D! [outputs.prometheus_client] Wrote batch of 82 metrics in 809.535µs

  • Attached Valgrind output that shows no leaks or memory corruption was found

==5663== 
==5663== HEAP SUMMARY:
==5663==     in use at exit: 102,192 bytes in 3,426 blocks
==5663==   total heap usage: 36,646 allocs, 33,220 frees, 7,755,146 bytes allocated
==5663== 
==5663== LEAK SUMMARY:
==5663==    definitely lost: 0 bytes in 0 blocks
==5663==    indirectly lost: 0 bytes in 0 blocks
==5663==      possibly lost: 0 bytes in 0 blocks
==5663==    still reachable: 102,192 bytes in 3,426 blocks
==5663==         suppressed: 0 bytes in 0 blocks
==5663== Rerun with --leak-check=full to see details of leaked memory
==5663== 
==5663== For counts of detected and suppressed errors, rerun with: -v
==5663== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Documentation

  • Documentation required for this feature

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Signed-off-by: Mike Frost <Mike.Frost@man.com>
@l2dy
Copy link
Contributor

l2dy commented Dec 26, 2021

Valid JSON by definition has quotes. Could you use key_value line_format instead?

@github-actions
Copy link
Contributor

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the Stale label Mar 27, 2022
@tsk9
Copy link

tsk9 commented Apr 22, 2024

What is necessary to accept this PR?

The PR would help to prevent the needless quotes escape (\").

@l2dy
Copy link
Contributor

l2dy commented Apr 22, 2024

In my opinion, a new option should be introduced instead of breaking the existing behavior that conforms to line_format.

@tsk9
Copy link

tsk9 commented Apr 22, 2024

Something like a drop_single_key_as_raw_value configuration option?

@l2dy
Copy link
Contributor

l2dy commented Apr 22, 2024

Something like a drop_single_key_as_raw_value configuration option?

It could also be a special value for the drop_single_key option. Either way, this is a stale PR and discussion should move elsewhere.

Copy link
Contributor

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants