Fix DAA

[tromp]

Link to rendered text

[phyro]

typo availble

[quentinlesceller]

Link to rendered doc https://github.com/tromp/grin-rfcs/blob/fixDAA/text/0000-fix-daa.md

[phyro]

DDA -> DAA

[zawy12]

I have a few comments that I'll just blurt out. You can use algebra to simplify wtema. 2 hours with 1 minute blocks (M=120) is a little low. There will be an error in average solvetime like relative ASERT that increases with M. In BTC coins nBits can have an err=2^(-15) in a certain range and the error will accumulate as M*err which is 3% for M=1000. It's interesting that ETH independently evolved a similar algo and pointing out this is better. ETH had a divided by zero and even negative difficulty possibility and their patch to prevent it opens up a selfish mining attack that allows about 75% (or more?) blocks in a given amount of time than a typical attack. ETH also only uses integer division of t/T (giving 0, 1, 2, 3... instead of the actual ratio), which works fine except for making the average solvetime lower by about 7% and gives possibly more leverage to a selfish mining attack than the 75%. These problems would be serious in coins who are not the biggest for their POW. The only problem with wtema and ASERT is that they allow about 39% ~ 1/e more blocks in a selfish mining attack via the "lowering by e" problem you described. SMA's allow ~44% more and digishield/grin are somewhere between 39% and 44%. LWMA is the only one I can't get excess blocks in (Toomim's testing of attacks had an incorrect LWMA), but it has to loop over all the blocks. There's an alternate form of LWMA that does not have a loop, but but it's not fully vetted (if it accumulates error like wtema, it could get thrown way off).

[tromp]

Grin will use the maximum precision available (64 bits) for difficulty computations. We don't use nbits and certainly won't truncate t/T. My personal preference is for M=240. It looks like Grin will see an influx of ASICs, see https://forum.grin.mw/t/2020-09-06-grin-meetup-hangzhou/7790/5

[j01tz]

Thanks for putting this together and apologies for the review lag. I do like the idea of keeping up with research to have an efficient and robust DAA.

I'm afraid I share Peter Todd's discomfort around reliance on strictly accurate time keeping for a use like this [thread for reference: https://twitter.com/peterktodd/status/1295928073448304640].

For me, the best argument against a tighter reliance on timestamps is the risk of reliance on the security and global availability of NTP. It is a very central point of failure for an otherwise relatively decentralized network. Honestly the 12min limit we are currently using feels too low as is, but I do understand the desire to limit impact from timestamp manipulation.

This is just my gut reaction- I'm highly wary of a reliance on "tight timing" for a decentralized system like Grin, especially as tight as one minute.

There is obviously a tradeoff to be made here, just like anything else. It seems we are sacrificing some security and decentralization in exchange for a DAA that is more adaptive, resulting in more stable blocktimes (and potentially better UX by extension).

I'm curious for more feedback from others to see if I'm being too paranoid as usual or if it makes sense to limit dependence on NTP working as intended, especially with a window as small as one minute.

[zawy12]

@j01tz I can't give any simple proof or explanation (or any perfect proof at all) that there is not a security loss in going from 60 min to 12 min or from 12 min to 1 min. But I believe it is safe. I have advised about 50 coins to use less than 5 minutes and I have not yet heard of any problem. To reduce fears of a problem, if NTP or peer time is not currently the default, then maybe nodes could be polled for their current time to see how many are actually off.

NTP or any other official source of time should not be used. Node operators must make sure their time is right.

Correct time is only important for pools or those nodes regularly receiving coin. The risk is greater the more a node is behind current time because it enables a small miner more time to do valid blocks while the node is rejecting the main chain for being (it thinks) too far into the future. The node is creating an easy eclipse attack on himself, which the attacker can exploit if he finds out the node is behind in time and if he can send it coin in exchange for some value. The attacker first sends coin to himself on the main chain, then begins work on a chain that only the victim will see as currently the lead tip. The small attacker is not going to get much of an advantage when going from 12 min to 1 min. If 10 blocks are confirmation, a 10% miner needs to find a node that is about 10/(0.10) * 0.51 = 50 blocks behind in time, beyond the FTL limit so 12 min needs the victim to be 62 minutes behind while a 1 min limit needs victim to be 51 minutes behind.

If a node is ahead in time, I can't think of an attack. It allows a >50% attacker to get more blocks than normal, but but only on the nodes that have the wrong time. If he quits early, before other nodes reach that time, he will lose chain work.

[tromp]

I cannot take Peter Todd seriously when he suggests a FTL of 24 hours. That's ridiculous. As is wanting to compensate for nodes set to the wrong timezone. Time-stamp manipulation is a common affair in cryptocurrencies. Attacks on NTP that leave clocks off by more than a minute for extended periods is something I never heard of.
Note that we don't use Bitcoin's median of past 11 timestamps. We require strictly increasing timestamps. Which makes timestamp set in the future near the limit more problematic, restricting the next miner to increase the timestamp by 1 sec (rather than use their earlier local time).
A shorter window makes this less of a problem...

[lehnberg]

@j01tz assigned as shepherd

[quentinlesceller]

Friendly reminder that this RFC needs to be reformatted like the updated template https://github.com/mimblewimble/grin-rfcs/blob/master/0000-template.md

[j01tz]

In line with our governance process, this RFC can be considered being in Final Comment Period, with a disposition to merge in two weeks time, on October 20.

Please ensure any comments are made before then!

[antiochp]

Just been looking at the FlyClient paper (revised Aug 2020) https://eprint.iacr.org/2019/226.pdf
Specifically the new sections on Variable Difficulty Chains and the "Variable Difficulty MMR".

Is the proposed DAA compatible with this concept of validating difficulty adjustments across subtrees of the header MMR?
A Variable Difficulty MMR commits to various difficulty related data in addition to hashes of the child nodes. Can we do this with wtema impl or are there any fundamental reasons why this would not be possible?

i.e. Given the leftmost-child and the rightmost-child leaf node beneath a given subtree root can we construct a "Variable Difficulty MMR" such that we can validate the overall difficulty from leftmost-child to rightmost-child?
This would then allow every node in the Merkle path to be validated, not just based on child hashes but overall child difficulty adjustment. Presumably within certain bounds based on the largest difficulty adjustment permitted given the period between lc and rc.

[zawy12]

Instead of flyclient, if the goal is to use NIPoPoW, you could change difficulty only once per month and in between use wtema to change the reward instead of difficulty to keep the correct block times.

I wonder if there is an algorithm that could result in the area under the difficulty curve in between a start difficulty and an end difficulty being a known value at the end of a month so that you know the total difficulty by only looking at the start and end difficulties and times.

P.S. wtema should not cause a problem. ETH's algo is just a bad version of it.

[tromp]

This is the relevant section of the flyclient paper on handling difficulty transitions:

The Variable-Difficulty Model. To adapt FlyClient to the case where blocks have different difficulties, we use the same sampling distribution g(x) but x now denotes the relative aggregate difficulty. For example, x = 1/2 refers to a point on the chain where half of the difficulty has been amassed, and g(1/2) is the probability that the block at that point is sampled by FlyClient. To ensure that the full node returns the correct blocks according to the difficulty distribution, we modify the MMR commitments such that each node in the Merkle tree now additionally contains the aggregate difficulty of all nodes below it. This means that each block header is now committing to not only the sequence of all blocks up to the given block but also to the total difficulty amassed by the network up to that block. Therefore, a Merkle inclusion proof, which is generated in a way similar to a standard Merkle tree proof, allows the client to verify that the provided block is indeed located at the x-th percentile of the total difficulty.
Additionally the new MMR proofs ensure that the difficulty transitions are done correctly. This is done
by storing more information such as total time in each internal MMR node. The information suffices to
check that invalid difficulty transitions do not give the adversary an advantage. Failing to do so could lead to devastating difficulty raising attacks [13].

We already store cumulative difficulty and timestamps in the header, so they are effectively MMR commitments. The new DAA makes the next difficulty dependent only on the last 2 headers, which minimizes the size of merkle proofs to validate the difficulty computation. With the old DAA, you need 2 headers which are 60 blocks apart (assuming only primary pow, else you need all 60 to determine primary ratio), which reduces the overlap between their merkle proof paths and would thus result in somewhat larger flyclient proofs.

In summary, yes, the proposed DAA is compatible with this concept of validating difficulty adjustments across subtrees of the header MMR.

[antiochp]

In summary, yes, the proposed DAA is compatible with this concept of validating difficulty adjustments across subtrees of the header MMR.

That sounds good. I just wanted to get ahead of any potential complications now and not later given that we are still interested in exploring FlyClient at some point.

This is done by storing more information such as total time in each internal MMR node.

This part is interesting to me, but not necessarily in a good way. I suspect discussion around this is off-topic for this RFC. I'll open a separate issue to track discussion. but tl;dr it looks like our header MMR is a starting point, but not sufficient for any implementation of FlyClient itself. I'm interpreting "Internal node" here as the parent nodes in the MMR, where we currently only store the hash. The paper discusses storing additional fields which we do not currently do.

[lehnberg]

In line with our governance process, this proposal has now been accepted as RFC#0018.