Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync master with devel #1673

Merged
merged 70 commits into from
Sep 16, 2020
Merged

Sync master with devel #1673

merged 70 commits into from
Sep 16, 2020

Conversation

mratsim
Copy link
Contributor

@mratsim mratsim commented Sep 16, 2020

Devel has been more stable on medalla than master for a while.

TODO: announcement that master is now the new recommended way to run the nim-beacon-chain, expect some instability on devel while we upgrade to the new Gossipsub 1.1 and Discovery 5.1 (cc @sachayves )

stefantalpalaru and others added 30 commits August 23, 2020 18:44
* book updates

* save progress

* save progress

* work on deposit data section + edits

* title edit

* minor edits

* edits and clarifications

* add images, launchpad steps, and a note on expected wait time

* add images + edits

* restructure, simplify, edits

* faq corrections

* rework intro: cp

* update intro and faq

* add note on requestiong goerli eth

* typo fix

* add numbers to steps in walletstore guide"

* chapter 8 title: replace Nimbus with NBC

* Update docs/the_nimbus_book/src/medalla.md

Co-authored-by: Jacek Sieka <jacek@status.im>

Co-authored-by: Jacek Sieka <jacek@status.im>
and log the BLS backend
new key cache can be used for old epochs in the same tree
* initial - cheaper pruning - addresses  #1534

* Pass tests: update offset when pruning, proper handling of pruned parents

* Use options instead of nil for nilable newHead (finalization passing but rootcause not solved)

* First line of defense against stackoverflow in tests

* Fix compute_delta offset after pruning

* Rebase fix - medalla ready

* Remove Option[BlockRef]
…1569)

* Request manager ignores non-critical errors while processing blocks. Only BlockError.Invalid become critical error.

* Add some comments.
…1566)

* harden beacon_pending_deposits metrics calculation

* ...

* move beacon_pending_deposits and beacon_processed_deposits_total out of specs and into chain DAG
* Jenkins: run benchmarks

* don't exit the workspace

* try calling the benchmark plugin from the config file

* pass all params

* separate Jenkins job for benchmarks
* book updates

* troubleshooting: introduce devel branch, and link to discord

* add disclaimer and recommendation to fall back to master

* minor edits
This implements disparity, resolving a part of
#1367

* make BeaconTime a duration for fractional seconds
* factor out attestation/aggregate validation
* simplify recording of queued attestations
* simplify attestation signature check
* fix blocks_received metric
* add some trivial validation tests
* remove unresolved attestation table - attestations for unknown blocks
are dropped instead (cannot verify their signature)
* bound block quarantine size

* add additional logging for block quarantining

* re-add quarantine.add() call

* remove pre-finalization blocks; add logging for full quarantine

* clear quarantine on chain reorganization

* update block_sim and tests

* update test_attestation_pool
…tation/block sending (#1590)

* ignore sqlite WAL journal files in git; switch attestation resolved from info to debug

* promote sent attestations/blocks to notice rather than demote resolved attestations/blocks to debug
avoid copy of validator as well
because it changes by simply building the software, creating problems
for beta testers
Co-authored-by: = <arnetheduck@users.noreply.github.com>
…is a new signing_process binary which loads all validators of the beacon node and the BN dictates through stdin of the signing process what to be signed and when and reads from stdout of the process
* address issue #1580

* Update beacon_chain/spec/beaconstate.nim

Co-authored-by: Jacek Sieka <jacek@status.im>
arnetheduck and others added 26 commits September 10, 2020 08:22
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* bump libp2p, introduce AsyncQueue BufferStream

* bump

Co-authored-by: cheatfate <eugene.kabanov@status.im>
per spec, we must half-close request stream - not doing so may lead to
failure of the other end to start processing our request leading to
timeouts.

In particular, this fixes many sync problems that have been seen on
medalla.

* remove safeClose - close no longer raises
* use per-chunk timeouts in request processing
* Quick fix to prune some states, pending smarter state storage

Adverse effects might include slow rewinds - typically the protocol
doesn't ask for pre-finalized states but RPC might

* document issue, add test

* fix cache miss log
Validator duties proceed slot-by-slot - we should not start a new
validator duty iteration before the previous one is gone or we might run
into consistency and voting issues
* Syncing workers now not bound to peers.
Sync status is now printed in statusbar.

* Add `SyncQueue.outSlot` to statusbar too.

* Add `inRangeEvent` and `rangeAge` parameter.

* Fix rangeAge is not depends on SyncQueue latest slot.
Fix syncManager to start from latest local head slot.

* Add notInRange event.

* Remove suspects field.
…ates (#1593)

* add ncli_db subcommand to prune database of unnecessary blocks, states, and state roots

* tweak comments

* reduce default aggressiveness in pruning old states

* move copyPrunedDatabase() to ncli_db, as it's not generally useful as part of beacon_chain_db and doesn't use any internal interfaces
uses a lot of memory, specially during sync
* cleanly close kvstore databases

* close databases for all subcommands and during error conditions
Add database pruning subsection
* update medalla guide: advanced options

* minor edits
so the Ctr+C hook and the metrics server go before the potentially slow
`BeaconNode.init()`
* skeleton of attester slashing pool & validators

* add skeleton for proposer slashings and voluntary exits; rename pool to more inclusive exit pool to stay consistent with all three; ensure is initialized by beacon_node so is safe to merge, even if it doesn't do much yet
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
* enable topic unsubscribing and attestation subnet cycling

* remove refences to states

* waitFor -> await

* revert exit pool changes for a cleaner PR
Co-authored-by: = <arnetheduck@users.noreply.github.com>
* Refactor peer_pool.
Fix eth2_network peer counters.
Fix PeerPool do not allow to add more peers when empty space available.

* Remove unused imports.

* Add test for a bug.

* Fix eth2_network disconnect should deletePeer not release.
More PeerPool refactoring.
* Slashing protection + interchange initial commit

* Restrict the when UseSlashingProtection dance in other modules

* Integrate slashing tests in other all_tests

* Add attestation slashing protection support

* Add a message that mention if built with/without slashing protection

* no op the initialization proc

* test slashing protection in Jenkins (temp)

* where to configure NIMFLAGS in Jenkins ...

* Jenkins -> ensure Built with slashing protection

* Add slashing protection complete import

* use Opt.get(otherwise)

* Don't use negation in proc name

* Turn slashing protection on by default
Co-authored-by: = <arnetheduck@users.noreply.github.com>
@mratsim
Copy link
Contributor Author

mratsim commented Sep 16, 2020

I think we hit a Github/Travis bug. The Travis PR is wrongly referencing PR #1672

https://travis-ci.org/github/status-im/nim-beacon-chain/builds/727750749

image

And failing because it doesn't exist

image

Build 1673 is passing

image
https://travis-ci.org/github/status-im/nim-beacon-chain/builds/727750749

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants