use grpc for communicating with compactor for query time filtering of data requested for deletion #7804
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…f data requested for deletion
sandeepsukhani
requested review from
periklis,
MasslessParticle and
MichelHollands
|
@sandeepsukhani You are addressing this issue btw: #7595 |
MasslessParticle
approved these changes
Nov 29, 2022
# Conflicts: # pkg/loki/modules.go
github-actions
bot
added
the
type/docs
periklis
reviewed
Nov 30, 2022
pkg/loki/modules.go
Outdated
| @@ -713,7 +713,7 @@ func (t *Loki) supportIndexDeleteRequest() bool { | |||
| func (t *Loki) compactorAddress() (string, bool, error) { | |||
| if t.Cfg.isModuleEnabled(All) || t.Cfg.isModuleEnabled(Read) { | |||
| // In single binary or read modes, this module depends on Server | |||
| return fmt.Sprintf("127.0.0.1:%d", t.Cfg.Server.GRPCListenPort), true, nil | |||
| return fmt.Sprintf("%s:%d", t.Cfg.Server.HTTPListenAddress, t.Cfg.Server.GRPCListenPort), true, nil | |||
There was a problem hiding this comment.
Suggested change
| return fmt.Sprintf("%s:%d", t.Cfg.Server.HTTPListenAddress, t.Cfg.Server.GRPCListenPort), true, nil | |
| return fmt.Sprintf("%s:%d", t.Cfg.Server.GRPCListenAddress, t.Cfg.Server.GRPCListenPort), true, nil |
Did you mean GRPC?
There was a problem hiding this comment.
yeah, sorry, my bad, messed up while merging conflicts. Thanks for noticing it and pointing it out!
sandeepsukhani
added a commit
that referenced
this pull request
Nov 30, 2022
**What this PR does / why we need it**: I had enabled auto-merge in PR #7804, but somehow it still merged the PR without all the checks passing. This PR fixes the failing lint and tests.
Abuelodelanada
pushed a commit
to canonical/loki
that referenced
this pull request
Dec 1, 2022
… data requested for deletion (grafana#7804) **What this PR does / why we need it**: Add grpc support to compactor for getting delete requests and gen number for query time filtering. Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki. I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config. **Checklist** - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated
Abuelodelanada
pushed a commit
to canonical/loki
that referenced
this pull request
Dec 1, 2022
**What this PR does / why we need it**: I had enabled auto-merge in PR grafana#7804, but somehow it still merged the PR without all the checks passing. This PR fixes the failing lint and tests.
trevorwhitney
added
the
backport release-2.7.x
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-7804-to-release-2.7.x origin/release-2.7.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 1410808ee9f20917476fabaa78aa8849ba7c7d20
# Push it to GitHub
git push --set-upstream origin backport-7804-to-release-2.7.x
git switch main
# Remove the local backport branch
git branch -D backport-7804-to-release-2.7.xThen, create a pull request where the |
trevorwhitney
pushed a commit
to trevorwhitney/loki
that referenced
this pull request
Dec 9, 2022
… data requested for deletion (grafana#7804) **What this PR does / why we need it**: Add grpc support to compactor for getting delete requests and gen number for query time filtering. Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki. I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config. **Checklist** - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated (cherry picked from commit 1410808)
trevorwhitney
pushed a commit
to trevorwhitney/loki
that referenced
this pull request
Dec 9, 2022
**What this PR does / why we need it**: I had enabled auto-merge in PR grafana#7804, but somehow it still merged the PR without all the checks passing. This PR fixes the failing lint and tests.
trevorwhitney
added a commit
that referenced
this pull request
Dec 9, 2022
use grpc for communicating with compactor for query time filtering of data requested for deletion (#7804) Manual backport of #7804 and #7814 **What this PR does / why we need it**: Add grpc support to compactor for getting delete requests and gen number for query time filtering. Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki. I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config. **Checklist** - [x] Documentation added - [x] Tests updated - [x] `CHANGELOG.md` updated (cherry picked from commit 1410808) Co-authored-by: Sandeep Sukhani <sandeep.d.sukhani@gmail.com>
Merged
9 tasks
5 tasks
JoaoBraveCoding
added a commit
to JoaoBraveCoding/loki
that referenced
this pull request
Mar 21, 2023
Problem: in grafana#7804 the possibility to use gRPC for communicating with compactor for query time filtering of data requested for deletion was added, however the loki mixin still generates the compactor service without a gRPC port. Solution: make the compactor service/deployment expose both a HTTP and gRPC port. Signed-off-by: JoaoBraveCoding <jmarcal@redhat.com>
chaudum
pushed a commit
that referenced
this pull request
Mar 21, 2023
**What this PR does / why we need it**: Problem: in #7804 the possibility to use gRPC for communicating with the compactor for query time filtering of data requested for deletion was added, however, the Loki mixin still generates the compactor service without a gRPC port. Solution: make the compactor service/deployment expose both an HTTP and gRPC port. Signed-off-by: JoaoBraveCoding <jmarcal@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Add grpc support to compactor for getting delete requests and gen number for query time filtering.
Since these requests are internal to Loki, it would be good to use grpc instead of HTTP same as all the internal requests we do in Loki.
I have added a new config for accepting the grpc address of the compactor. I tried having just the existing config and detecting if it is a grpc server, but it was hard to do it reliably, considering the different deployment modes we support. I think it is safe to keep it the same and eventually deprecate the existing config.
Checklist
CHANGELOG.mdupdated