Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

promtail: Add max-line-size-truncate #8233

Merged
merged 4 commits into from
Jan 25, 2023
Merged

promtail: Add max-line-size-truncate #8233

merged 4 commits into from
Jan 25, 2023

Conversation

nicoche
Copy link
Contributor

@nicoche nicoche commented Jan 22, 2023

What this PR does / why we need it:

It allows promtail to truncate too long lines instead of just dropping them. See #8153 (comment) for details

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • CHANGELOG.md updated
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

@nicoche nicoche requested a review from a team as a code owner January 22, 2023 21:09
@CLAassistant
Copy link

CLAassistant commented Jan 22, 2023 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Jan 22, 2023
@nicoche nicoche force-pushed the add-truncate-option branch from dd2def9 to d842131 Compare January 22, 2023 21:11
@nicoche
Copy link
Contributor Author

nicoche commented Jan 22, 2023

@kavirajk cf. our discussion in #8202, let me know what you think 🙂

@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

kavirajk
kavirajk reviewed Jan 23, 2023
View reviewed changes
Copy link
Contributor

@kavirajk kavirajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nicoche for the PR. Overall looks good. Left few comments

clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Show resolved Hide resolved
@nicoche nicoche force-pushed the add-truncate-option branch from d842131 to 8cee5e2 Compare January 23, 2023 19:45
@nicoche
Copy link
Contributor Author

nicoche commented Jan 23, 2023

Thanks for the feedback! I incorporated your suggestions 🙂

@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

kavirajk
kavirajk reviewed Jan 24, 2023
View reviewed changes
Copy link
Contributor

@kavirajk kavirajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nicoche. Almost there :) Left few minor suggestions. Happy to merge once they are addressed.

clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client_test.go Outdated Show resolved Hide resolved
clients/pkg/promtail/client/client_test.go Outdated Show resolved Hide resolved
@nicoche nicoche force-pushed the add-truncate-option branch from 8cee5e2 to 54dfb2c Compare January 24, 2023 11:38
@nicoche
Copy link
Contributor Author

nicoche commented Jan 24, 2023

Nice catches 🙂. Updated the PR

@nicoche
promtail: Add max-line-size-truncate
794425a 
It allows promtail to truncate too long lines instead of just dropping
them.
@nicoche nicoche force-pushed the add-truncate-option branch from 54dfb2c to 794425a Compare January 24, 2023 11:39
@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

kavirajk
kavirajk reviewed Jan 25, 2023
View reviewed changes
Copy link
Contributor

@kavirajk kavirajk left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nicoche. LGTM. Two minor changes needed before merge.

NOTE: In general, force push make it hard to know what changed after my last review. Please do normal push after addressing the comments (if needed rebase, do git merge) :)

clients/pkg/promtail/limit/config.go Outdated
@@ -13,6 +13,7 @@ type Config struct {
ReadlineRateDrop bool `mapstructure:"readline_rate_drop,omitempty" yaml:"readline_rate_drop,omitempty" json:"readline_rate_drop"`
MaxStreams int `mapstructure:"max_streams" yaml:"max_streams" json:"max_streams"`
MaxLineSize flagext.ByteSize `mapstructure:"max_line_size" yaml:"max_line_size" json:"max_line_size"`
MaxLineSizeTruncate bool `mapstructure:"max_line_size" yaml:"max_line_size_truncate" json:"max_line_size_truncate"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mapstructure:"max_line_size" -> mapstructure:"max_line_size_truncate"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!! This one could have easily slipped through.

docs/sources/clients/promtail/configuration.md Outdated Show resolved Hide resolved
@nicoche nicoche requested a review from JStickler as a code owner January 25, 2023 11:19
@grafanabot
Copy link
Collaborator

./tools/diff_coverage.sh ../loki-target-branch/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

kavirajk
kavirajk approved these changes Jan 25, 2023
View reviewed changes
Copy link
Contributor

@kavirajk kavirajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice work @nicoche 👍

@kavirajk kavirajk merged commit 80ea621 into grafana:main Jan 25, 2023
@nicoche
Copy link
Contributor Author

nicoche commented Jan 25, 2023

LGTM. Nice work @nicoche 👍

Thanks for the feedbacks 🙂

@nicoche nicoche deleted the add-truncate-option branch January 25, 2023 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kavirajk kavirajk kavirajk approved these changes

@JStickler JStickler Awaiting requested review from JStickler

Assignees
No one assigned
Labels
size/L type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nicoche @CLAassistant @grafanabot @kavirajk