[Feature Request]: Comprehensive proof API

[damiannolan]

Overview

Use case

As an on-chain lightclient I would like to verify blob inclusion against the dataHash (data availability root) tracked in celestia-core commit headers.

For example, given a blob.Commitment in namespace NS I would like to query some RPC endpoint at height H which returns a proof veriable against the data availability root.

Problem defintion

As it exists today a blob proof queried from celestia-node's blob.GetProof api is not verifiable against the celestia data availability root signed by the validator set. The proof returned - blob.Proof is alias of []*nmt.Proof (ref).

This proof(s) is intended to be verified via an RPC call to celestia-node's Included api. Unfortunately, this does not satisfy the requirements of an on-chain lightclient, where a proof should be submitted and verified within the state machine against the data availability root.

Furthermore, namespaced merkle tree (nmt) proofs are only verifiable against their corresponding RowRoots (ref).
And a collection of RowRoots for a block at a particular height N are merklized to produce the data availability root (ref). As I understand the data availability header (DAH), containing Row and Column roots of the da square is discarded by the celestia application in PrepareProposal, only storing the hash (root) in the commit header.

This leads me to believe that as an off-chain actor (e.g. ibc relayer), I would be forced to additionally query the ExtendedHeader and manually work out and reconstruct a ShareProof which is veriable against the DA root.

Or alternatively to manually work out the share indices for a particular blob and rely on the ProveShares rpc offered by celestia-core. I believe this would also require querying additional info to retrieve data such as:

• txIndex
• blobIndex
• block for txs

Some sample go code might look something like:

version := blockRes.Block.Header.Version.App
maxSquareSize := appconsts.SquareSizeUpperBound(version)
subtreeRootThreshold := appconsts.SubtreeRootThreshold(version)
blobShareRange, err := square.BlobShareRange(
	blockRes.Block.Txs.ToSliceOfBytes(),
	int(tx.Index),
	int(blob.Index),
	maxSquareSize,
	subtreeRootThreshold,
)

// get the proof of the shares containing the blob provabable to the data root
sharesProof, err := trpc.ProveShares(ctx, uint64(tx.Height), uint64(blobShareRange.Start), uint64(blobShareRange.End))
if err != nil {
	panic(err)
}

Proposal

As explained to me by @renaynay an RPC call to blob.GetProof rather returns a proof for an entire namespace. As far as I know this can be seen in the retrieve method of blob/service.go in celestia-node.

From what I understand about the celestia architecutre, a more comprehensive and cohesive proof API may look something like the following:

• GetShareProof(height, startShare, endShare uint64)
• GetBlobProof(height uint64, namespace share.Namespace, commitment blob.Commitment)
• GetNamespaceProof(height uint64, namespace share.Namespace)

As a user I would expect each of these proofs to be veriable against the data availability root. Due to my limited knowledge of the system this may not be possible but from a user's perspective I would expect this to be a desireable feature or property of the proof APIs offered by celestia.

Feel free to close this issue if it is duplicate or if you could point me another issue or resource with work or discussion about this topic I would be very interested!

[rach-id]

@damiannolan Check this PR: #3470, I guess it solves what's needed in this issue. Your feedback is appreciated

[damiannolan]

Awesome! Thank you @rach-id, I'll check it out