-
Notifications
You must be signed in to change notification settings - Fork 812
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update code to fix almost all pre-existing lint errors #2008
Conversation
@jtlisi let me know if you have feeling or thoughts on how to handle the last outstanding lint issue. Tagged you since you were the original issue reporter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a minor nit. Also for that final linting error, feel free to add //nolint:staticcheck
and open an issue.
Thanks for taking this on
d7848a8
to
8b6ce8e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work! I made a lot of comments; I think the FramedSnappy one is a real issue.
@@ -75,7 +75,7 @@ func (i *instrumentedCache) Store(ctx context.Context, keys []string, bufs [][]b | |||
} | |||
|
|||
method := i.name + ".store" | |||
instr.CollectedRequest(ctx, method, requestDuration, instr.ErrorCode, func(ctx context.Context) error { | |||
_ = instr.CollectedRequest(ctx, method, requestDuration, instr.ErrorCode, func(ctx context.Context) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I question why Store() has no error return.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FWIW from pkg/chunk/cache/cache.go
// Cache byte arrays by key.
//
// NB we intentionally do not return errors in this interface - caching is best
// effort by definition. We found that when these methods did return errors,
// the caller would just log them - so its easier for implementation to do that.
// Whats more, we found partially successful Fetchs were often treated as failed
// when they returned an error.
type Cache interface {
Store(ctx context.Context, key []string, buf [][]byte)
Fetch(ctx context.Context, keys []string) (found []string, bufs [][]byte, missing []string)
Stop() error
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bboreham just double checking are we good with this?? as per the comment above it was very intentional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good job @zendern. Looking at the change set, and the possible issues spotted, it's easy to understand the benefits of the linter. I've also left some comments.
fcc1c31
to
de4c97e
Compare
pkg/util/grpcclient/ratelimit.go
Outdated
limiter.Wait(ctx) | ||
err := limiter.Wait(ctx) | ||
if err != nil { | ||
return err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When used in conjunction with NewBackoffRetry()
(see grpcclient.go
), the backoff expects a ResourceExhausted
error code.
@bboreham what do you think returning that GRPC error here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That sounds like a bug.
"Error: I'm not going to wait because the context will expire before the wait finishes"
"Never mind, try again!"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So @bboreham / @pracucci if i am understanding correctly what you are saying is that making this change is the wrong thing to do. Basically since the wait would not return a ResourceExhausted that would cause the BackoffRetry, if its being used, to now exit and no longer retry like we expect it to.
So if that is correct the right course of action is to just go back to what we had in originally here
#2008 (comment)
and ignore the error or should the BackoffRetry code check for another error code along with ResourceExhausted
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not rollback but fix it. We may do:
return status.New(codes.ResourceExhausted, err.Error())
But I would be glad to also see a unit test on it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll look at adding a unit test for this and getting that change in place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok thanks! This looks like the very last thing left to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pracucci So i wrote up this unit test last night.....which might not be enough honestly. It's fine that it can validate that its a status with the expected result but an integration test would probably drive home the fact that it's necessary if you configure it with the backoffRetry. I just ran out of time.....I'll see if i can get to it over the weekend.
import (
"context"
"testing"
"github.com/cortexproject/cortex/pkg/util/grpcclient"
assert2 "github.com/stretchr/testify/assert"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func TestRateLimiterFailureResultsInResourceExhaustedError(t *testing.T) {
config := grpcclient.Config{
RateLimitBurst: 0,
RateLimit: 0,
}
conn := grpc.ClientConn{}
invoker := func(currentCtx context.Context, currentMethod string, currentReq, currentRepl interface{}, currentConn *grpc.ClientConn, currentOpts ...grpc.CallOption) error {
return nil
}
limiter := grpcclient.NewRateLimiter(&config)
err := limiter(context.Background(), "methodName", "", "expectedReply", &conn, invoker)
if se, ok := err.(interface {
GRPCStatus() *status.Status
}); ok {
assert2.Equal(t, se.GRPCStatus().Code(), codes.ResourceExhausted)
assert2.Equal(t, se.GRPCStatus().Message(), "rate: Wait(n=1) exceeds limiter's burst 0")
} else{
assert2.Fail(t, "Could not convert error into expected Status type")
}
}
@bboreham @pracucci so looking into golangci and errcheck i dont believe I am able to add the excludes on Using errcheck directly the above exclude works b/c they have built in support for embedded interfaces. Whereas the golangci fork of errcheck is out of date and does not have that code. https://github.com/golangci/errcheck/tree/master/internal/errcheck https://github.com/kisielk/errcheck/tree/master/internal/errcheck I'm working on a PR to try to bring the golangci-lint version up to the same as errcheck master but that will take a little time. So how would you like to proceed?? |
Personally I'm fine even without an override if it's not easy to do. All in all, it affect 3 places and we can always re-iterate on it once supported. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the scope is limited, I'm fine without the override as well
6c3f5ad
to
741e440
Compare
if err != nil { | ||
panic(err) | ||
} | ||
return chunk | ||
} | ||
|
||
func TeardownFixture(t *testing.T, fixture Fixture) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any specific reason why this function was introduced? It looks a bit superfluous to me, given we can directly call require.NoError(t, fixture.Teardown())
instead of TeardownFixture()
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was done in this commit. 8bf4365 It was just something i noticed we were doing 2 places so felt like a natural thing since the fixture was already in the testutils.go class. I could go either way....ill leave that up to you but that was the reasoning.
pkg/util/grpcclient/ratelimit.go
Outdated
limiter.Wait(ctx) | ||
err := limiter.Wait(ctx) | ||
if err != nil { | ||
return err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not rollback but fix it. We may do:
return status.New(codes.ResourceExhausted, err.Error())
But I would be glad to also see a unit test on it.
@zendern Could you attempt another rebase and address the open comment, please? I'm hearing your pain continuously rebasing this: I'm going to stress other maintainers to review this as well, so that we can get this merged. |
87f183c
to
ea7b3aa
Compare
To be specific ineffassign, deadcode and unused errors Signed-off-by: Nathan Zender <github@nathanzender.com>
keys using hashing Signed-off-by: Nathan Zender <github@nathanzender.com>
Now that we have an iterator there is no need to also have a consumed bool on the underlying object. Signed-off-by: Nathan Zender <github@nathanzender.com>
Will never actually be used. Only necessary to pad out CPU cache lines. Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Attempted not to change any existing logic so if an error was ignored we will now either explicitly ignore it or in the case of a goroutine being called with a func that is ignoring the error we will just put a //noling:errcheck on that line. If it was in a test case we went ahead and did the extra assertion checks b/c its always good to know where something might have errored in your test cases. Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
We have opened up issue cortexproject#2015 to address the deprecation of this type. Signed-off-by: Nathan Zender <github@nathanzender.com>
The test is currently failing due to a data race. I believe it is due to this bug in golang. golang/go#30597 As far as this test cares it does not really matter that this happens so removing the need to check for NoError "fixes" it. Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Since Fixture is already in testutils and it is being used in both places pulled it out into a common helper method in the testutils package. Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Signed-off-by: Nathan Zender <github@nathanzender.com>
Formatting and import issues that got missed when merging. Signed-off-by: Nathan Zender <github@nathanzender.com>
This is necessary so that when it is used with the backoff retry it will allow for the backoff to continue to work as expected. Signed-off-by: Nathan Zender <github@nathanzender.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent work, thanks!
Let's merge it quick before anyone else blinks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, awesome work! Let's merge this! 🎉
What this PR does:
Cleanups all but one of the lint errors that were found when the hash was in place.
Last outstanding issue is as follows
Not 100% clear yet on path forward on this. From what i can tell even the underlying library is still using naming.Watcher. Wondering if for now we put a //nolint:staticcheck and open an issue to fix just that issue.
Which issue(s) this PR fixes:
Fixes #1912
Checklist
CHANGELOG.md
updated - the order of entries should be[CHANGE]
,[FEATURE]
,[ENHANCEMENT]
,[BUGFIX]