-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Implementer's Guide: Integrate DMP into PersistentValidationData + DMQ watermarks #1629
Conversation
0c2c250
to
a2510c1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks mostly good as far as I can tell, it would be nice to avoid using acronyms in headers,
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
7bf4578
to
398cf4a
Compare
On the second thought, the DMQ watermarks are not such a great idea. The problem with them is that they are coarse grained and require reading all messages within a block. Earlier, we decided that we don't want to restrict the relay-chain on how many messages to send to a parachain. With that, we can potentially end up in a situation the relay-chain sent a too many messages to even fit into a parablock. We could potentially leverage a compound watermark, similar to what we are going to use for XCMP where the watermark consists of the block number and para id, which allows to read only a part of messages in a block. In the DMP we have only the single sender - the relay-chain and only meaningful tie-breaker for the messages within the block is the order in which they were sent, e.g. That, however, seems to be an over-engineered version of the original Therefore, I am closing this PR |
Based on top of #1588
Closes #1643
Summary
PersistentValidationData
("ground truth") through a DMQ MQC head.This PR is accompanied by some drive-by wording fixes here and there.
Description
I decided to take the approach of putting only the MQC head into
PersistentValidationData
to allow the authorization of the messages that a collator provides while building a parablock.Background
The alternative is to put the DMQ contents, all messages, into `PersistentValidationData`. We need to put all messages in because `PersistentValidationData` is used to form the inputs and is collected before the execution of the PVF. Putting only data to authorize the sequence of messages into `PersistentValidationData` allows the collator to include the messages that were actually consumed into the PoV.All in all, this should positively affect the amount of data made available.
While working on it I realized that it might be useful to supply the block number in which a particular downward message was put in the DMQ. That also makes it similar to HRMP/XCMP MQC which may be good from the implementation standpoint.
It seemed to me that it would be harder for the PVF to return
processed_downward_messages
and trying to keep track of the on-the-relay-chain state of DMQ rather than dealing with watermarks. So, i finally introduced DMQ watermarks. They cost a bit more, since we need to keep track the digest (the list of all blocks where the DMP messages were sent to correctly check the DMQ watermark)Those DMQ watermarks, however, are still different from HRMP watermarks. I am still not confident about merging them.
What to look for
Here are some clues how to approach this PR
TransientValidationData
has all data to check if the resulting candidate passes the acceptance criteria (by collator or primary checker)