Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore ErrCanceled errors returned by verifyState #2224

Closed
bartekn opened this issue Feb 4, 2020 · 0 comments · Fixed by #2366
Closed

Ignore ErrCanceled errors returned by verifyState #2224

bartekn opened this issue Feb 4, 2020 · 0 comments · Fixed by #2366
Assignees
Labels
Milestone

Comments

@bartekn
Copy link
Contributor

bartekn commented Feb 4, 2020

What version are you using?

What did you do?

Horizon shutdown during state verification.

What did you expect to see?

No errors.

What did you see instead?

WARN[2020-02-04T18:14:19.987+01:00] State verification errored                    err="addAccountsToStateVerifier failed: Error running history.Q.GetAccountsByIDs: select failed: context canceled" pid=99452 service=expingest

Looks like we should add similar condition like in:

if cause := errors.Cause(err); cause != nil &&
cause != context.Canceled &&
cause != db.ErrCancelled {
log.WithFields(logpkg.F{
"error": err,
"current_state": cur,
"next_state": next.node,
}).Error("Error in ingestion state machine")
}

Seems like it's a simple fix but we're going to release beta really soon so adding to stable milestone.

@bartekn bartekn added the horizon label Feb 4, 2020
@bartekn bartekn added this to the Horizon 1.0.0-stable milestone Feb 4, 2020
@abuiles abuiles self-assigned this Mar 6, 2020
abuiles added a commit to abuiles/go that referenced this issue Mar 6, 2020
If the system is shutdown during state verification, there is a
warning in the log showing

```
WARN[2020-02-04T18:14:19.987+01:00] State verification errored                    err="addAccountsToStateVerifier failed: Error running history.Q.GetAccountsByIDs: select failed: context canceled" pid=99452 service=expingest
```

If the system is being shutdown gracefully which is the case when we
get this errors, then we shouldn't log an error.

Fix stellar#2224
abuiles added a commit to abuiles/go that referenced this issue Mar 10, 2020
If the system is shutdown during state verification, there is a
warning in the log showing

```
WARN[2020-02-04T18:14:19.987+01:00] State verification errored                    err="addAccountsToStateVerifier failed: Error running history.Q.GetAccountsByIDs: select failed: context canceled" pid=99452 service=expingest
```

If the system is being shutdown gracefully which is the case when we
get this errors, then we shouldn't log an error.

Fix stellar#2224
abuiles added a commit that referenced this issue Mar 10, 2020
…verifyState. (#2366)

* Ignore db.ErrCanceled and ctx.Canceled returned by verifyState

If the system is shutdown during state verification, there is a
warning in the log showing

```
WARN[2020-02-04T18:14:19.987+01:00] State verification errored                    err="addAccountsToStateVerifier failed: Error running history.Q.GetAccountsByIDs: select failed: context canceled" pid=99452 service=expingest
```

If the system is being shutdown gracefully which is the case when we
get this errors, then we shouldn't log an error.

Fix #2224

* Add isCancelledError.

* Use isCancelledError in runStateMachine.

* Simplify isCancelledError.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment