Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase bitcore patches to 0.14 #43

Closed
wants to merge 2,783 commits into from
Closed

Rebase bitcore patches to 0.14 #43

wants to merge 2,783 commits into from

Conversation

karelbilek
Copy link

@karelbilek karelbilek commented Apr 2, 2017

I rebased bitcore patches and made some corrections in the tests (the tx test was not running, because there is a built-in fee limit now) and in the code (changes in serialization calls, changes in segwit API, ...).

Since there was a big refactoring from 0.13.* to 0.14, I had to move the changes from main.cpp to other files; so I had to change the original bitcore patches slightly.

Both the qa tests and the c++ tests should be passing.

I don't know to which branch to put this PR, so I put it to master (which is an outdated version of upstream apparently), do as you please with it. (The commit list is giant, since it includes all the bitcoin commits between 0.12 and 0.14, plus the patches to bitcore.)

Since you have abandoned bitcore, I don't hope you will merge this, but maybe it will be helpful for others.

sipa and others added 30 commits January 23, 2017 10:42
…n DumpMempool

bd92f24 [bugfix] save feeDelta instead of priorityDelta in DumpMempool (Alex Morcos)
Add notification signals to make it possible to subscribe to mempool
changes:

- NotifyEntryAdded(CTransactionRef)>
- NotifyEntryRemoved(CTransactionRef, MemPoolRemovalReason)>

Also add a mempool removal reason enumeration, which is passed to the
removed notification based on why the transaction was removed from
the mempool.
Analogue to ConnectTrace that tracks transactions that have been removed from the mempool due to conflicts and then passes them through SyncTransaction at the end of its scope.
fa4d478 qt: Use nPowTargetSpacing constant (MarcoFalke)
094e4b3 Better document usage of SyncTransaction (Alex Morcos)
4afbde6 Introduce MemPoolConflictRemovalTracker (Alex Morcos)
ff25c32 mempool: add notification for added/removed entries (Wladimir J. van der Laan)
be31a2b [Trivial] Update license year range to 2017 (Lauda)
ac9a846 [Trivial] fix logging typo in FlushStateToDisk() (John Newbery)
The use of mocktime in test logic means that comparisons between
GetTime() and GetTimeMicros()/1000000 are unreliable since the former
can use mocktime values while the latter always gets the system clock;
this changes the networking code's inactivity checks to consistently
use the system clock for inactivity comparisons.

Also remove some hacks from setmocktime() that are no longer needed,
now that we're using the system clock for nLastSend and nLastRecv.
- The last-timestamp-encountered variable wasn’t being used properly. Rewrite code to properly allow for new blockchain files to be written when split by month.
- Properly set a blockchain file’s access and modify times.
- Add a “debug output” option to quiet certain output that might not always be desirable.
- Update the README.
2f10f06 qa: Increase a sync_blocks timeout in pruning.py (Suhas Daftuar)
…ity checks

99464bc net: Consistently use GetTimeMicros() for inactivity checks (Suhas Daftuar)
…n interaction with bumpfee

5a00659 [wallet] Clarify getbalance help string to explain interaction with bumpfee (Russell Yanofsky)
44f2baa Do not shadow local variable named `tx`. (Pavel Janík)
c36ec71 depends: qt: disable printer for all platforms, not just osx (Cory Fields)
…necting.

8ff8d21 Send final alert message to older peers after connecting. (Gregory Maxwell)
de1ae32 Exclude RBF txs from fee estimation (Alex Morcos)
Use the wallet's fee calculation logic to properly clamp fee against minimums and maximums when calculating the fee for a bumpfee transaction.  Unless totalFee is explictly given, in which case, manually check against min, but do nothing to adjust given fee.

In all cases do a final check against maxTxFee (after adding any incremental amount).
Also refactor to use wrapper for stop_node
Have wallet's default bump value be higher than the default incrementalRelayFee to future proof against changes to incremental relay fee.  Only applies when not setting the fee rate directly.
The result value indicates the actual fee on the transaction that was replaced. But there is an error message which uses the description 'oldfee' to refer to the original fee rate applied to the new transaction's estimated max size.  It was confusing that two different uses of 'oldfee' had two different numeric values.
laanwj and others added 16 commits March 5, 2017 14:33
589cd63 Allow any subkey in verify-commits (Matt Corallo)

Tree-SHA512: e3175273c648ed2d990ac931efae5e4bf3bd5ddce7b591f5e64a6831f3c029b252bc5d241dd8d3874467747c3ded87aa1fa334ff53d940cde32c22e584a2c4d0
Adds new bitcoin.conf configuration options for three new indexes:
-addressindex=1
-spentindex=1
-timestampindex=1

The addressindex records all changes to an address for retrieving txids, balances and
unspent outputs for addresses. Changes are stored and sorted in block order. Both p2sh
and p2pkh address types are supported. The index records two sets of key/value pairs.
The first records all activity and is useful for viewing transaction history and
all changes. The second is specifically for retrieving unspent outputs by address, and
is smaller as values are removed once they are spent.

The spentindex has multiple purposes and brings closer together inputs and outputs of
transactions. The main purpose is to efficiently determine the address and amount of
an input's previous output. The second purpose is to be able to determine which
input spent an output.

The timestampindex keeps track of timestamps with block hashes and is useful for searching
blocks by date instead of by height. This is useful for a block explorer that will give
search options by date. The index uses logical time correction to make sure that the
results are sorted in block order. The logical time of a block is actual timestamp of the
block, unless it is less than (earlier) the previous block's logical time, and in that
case it is one second greater than the previous block's logical time.

Includes logical time fix by Chethan Krishna

Conflicts:
	src/main.cpp
	src/main.h
	src/txdb.cpp
	src/txdb.h
	src/txmempool.h
Adds new rpc commands that use the address, spent and timestamp indexes,
including the new commands:

- getblockdeltas
- getblockhashes
- getaddressmempool
- getaddressutxos
- getaddressdeltas
- getaddressbalance
- getaddresstxids
- getspentinfo

And modifications to the command:
- getrawtransaction

Conflicts:
	src/rpc/blockchain.cpp
	src/rpc/client.cpp
	src/rpc/misc.cpp
	src/rpc/rawtransaction.cpp
	src/rpc/server.cpp
	src/rpcserver.h
Tests the functionality of the indexes as well as the rpc commands
There was a previous assumption that blockindex would be quite small. With addressindex
and spentindex enabled the blockindex is much larger and the amount of cache allocated for
it should also increase. Furthermore, enabling compression should decrease the amount of
disk space required and less data to write/read. The default leveldb max_open_files is set to
1000, for the blockindex the default is set to 1000 with compression. The 64 value that is
current is kept for the utxo database and does not enable compression. Two additional options
are added here to be able to configure the values for leveldb and the block index:

- `-dbmaxopenfiles` A number of files for leveldb to keep open
- `-dbcompression` Boolean 0 or 1 to enable snappy leveldb compression

Conflicts:
	src/dbwrapper.cpp
	src/init.cpp
@karelbilek
Copy link
Author

karelbilek commented Apr 3, 2017

I am testing the address index on segwit testnet and it seems like it is not behaving correctly with segwit transactions. :/

edit: nevermind, error on my end. Segwit txs are running fine.

@FrancisPouliot
Copy link

THanks so much @Runn1ng ! @gasteve some attention to this would greatly increase your users' confidence that Bitcore project has not been abandoned.

@ghost
Copy link

ghost commented May 2, 2017

Personal blockchain explorers are important! Please keep this project alive!

@FrancisPouliot
Copy link

FrancisPouliot commented May 2, 2017

@gasteve seems like all is left to do here is merge? I think there have been further upgrades by @Runn1ng since this pr

@karelbilek
Copy link
Author

For some reason, while this worked fine on testnet and regtest, it got strangely slow on livenet, and the database is strangely sluggish even on SSH while downloading the blockchain for the first time; when the chain is about 50GB big, it syncs chain only 10 minutes from 1 hour and the rest is doing something, probably in leveldb.

However, it does work in the end, only there is this heavy performance hit.

I don't remember this happening in older version; so there might be some errors in the rebase; I will investigate later, but now don't have the time unfortunately.

(My personal, long-term plan is to get to know this code even more deeply and then try to get it into actual bitcoin core, but I am not sure I will have time for that either :D )

@karelbilek
Copy link
Author

Some people are linking to this for some reason .... just FYI

I however have to say I cannot promise we will keep developing it long term or even add features etc; we are keeping it on "life support" so we can use it as a backend for Trezor Web Wallet.

@karelbilek
Copy link
Author

oh the memories. Closing this though.

@karelbilek karelbilek closed this May 27, 2020
LarryRuane pushed a commit to LarryRuane/bitcoin that referenced this pull request Dec 10, 2020
b5ef9be675 Merge #1: Merge changes from upstream
9e7f512430 Merge remote-tracking branch 'origin/master' into bitcoin-fork
1f85030246 Add support for ARM64 darwin (bitpay#43)
3bb959c982 Remove unnecessary reinterpret_cast (bitpay#42)
2e97ab26b1 Fix (unused) ReadUint64LE for BE machines (bitpay#41)
47b40d2209 Bump dependencies. (bitpay#40)
ba74185625 Move CI to Visual Studio 2019.
efa301a7e5 Allow different C/C++ standards when this is used as a subproject.
cc6d71465e CMake: Use configure_package_config_file()

git-subtree-dir: src/crc32c
git-subtree-split: b5ef9be6755a2e61e2988bb238f13d1c0ee1fa0a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.