Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exp/ingest/ledgerbackend: Handle Stellar-Core process exit gracefully #2803

Merged

Conversation

bartekn
Copy link
Contributor

@bartekn bartekn commented Jul 10, 2020

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

This commit adds a code that handles Stellar-Core process exit gracefully. Added processExit unbuffered channel to stellarCoreRunner that receives the error in case Stellar-Core crashed or was terminated with an error or nil error if it was shutdown gracefully. The channel is later used in PrepareRange that returns an error if the process was stopped while prepare was running and in sendLedgerMeta that adds error to a meta channel that will be read by one of the following GetLedger calls.

Close #2705.

Why

Right now when Stellar-Core crashes, CaptiveStellarCore simply wait for the next ledger without restarting it.

@bartekn bartekn requested a review from a team July 10, 2020 12:59
@cla-bot cla-bot bot added the cla: yes label Jul 10, 2020
return
case c.metaC <- metaResult{meta, nil}:
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I simplified this code. There's no need to check c.stop here because even we discard one result from metaC in Close to unblock it. So even if the code is waiting here, it will be unlocked by Close.

Copy link
Contributor

@abuiles abuiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -271,6 +286,20 @@ func (c *CaptiveStellarCore) PrepareRange(ledgerRange Range) error {
}

for {
select {
case <-c.shutdown:
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bartekn should we call c.Close() here too before returning - is seem like sendLedgerMeta could still be running on the goroutine

Copy link
Contributor Author

@bartekn bartekn Jul 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because c.shutdown can be only closed by calling c.Close. So it implies that c.Close was called.

@bartekn bartekn merged commit 66417a5 into stellar:captive-core-online Jul 13, 2020
@bartekn bartekn deleted the captive-handle-process-exit branch July 13, 2020 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants