Add FVM Randomness FIP #817

arajasek · 2023-08-30T16:36:10Z

As discussed in #790 and #791

anorth · 2023-08-30T21:10:59Z

FIPS/fip-randomness.md

+---
+fip: "<to be assigned>" <!--keep the qoutes around the fip number, i.e: `fip: "0001"`-->
+title: Improvements to the FVM randomness syscalls
+author: Aayush Rajasekaran (@arajasek), Steven Allen (@stebalien)


Nit: please quote the author line to avoid YAML warnings

anorth · 2023-08-30T21:11:32Z

FIPS/fip-randomness.md

+discussions-to: https://github.com/filecoin-project/FIPs/discussions/790, https://github.com/filecoin-project/FIPs/discussions/791
+status: Draft
+type: Technical (Core)
+category (*only required for Standard Track): <Core>


Suggested change

category (*only required for Standard Track): <Core>

category: Core

anorth · 2023-08-30T21:13:34Z

FIPS/fip-randomness.md

+<!--"If you can't explain it simply, you don't understand it well enough." Provide a simplified and layman-accessible explanation of the FIP.-->
+
+- Actors/Contracts: Don't expect clients to "hash" randomness for you, do it yourself
+- FVM: Don't impose any lookback limit on randomness and tipset CID fetching, but charge proportional to the lookback amount


Could you please phrase this as a declarative statement of what's changing, rather than imperative? It's hard to avoid some duplication with the abstract, but I usually to orient this summary toward the solution, while the abstract expands a bit more on the problem/goal (in this case, simplicity?).

"Changes VM-provided randomness to .... Removes limits on chain lookback but charges gas ...."

anorth · 2023-08-30T21:17:56Z

FIPS/fip-randomness.md

+
+Clients no longer receive the `dst` and `entropy` parameters for the randomness Extern methods, and should skip the final step of "drawing" randomness in which this DST and entropy is mixed into the hash.
+
+We also seek consensus amongst Core Devs that Filecoin clients are expected to be able to support chain walkbacks all the way to genesis, with a performance no _worse_ than that based on a skiplist of length 20. Clients are free to be more performant (eg. constant-time).


My understanding of a true skiplist would be logarithmic lookup time. It's not clear that's what you mean here though – I get the impression of a linear scan moving in steps of 20 (and then <20 steps of 1 to finish. Could you clarify this?

We have a "single layer" skiplist. But yes, a "real" skiplist would be faster.

But as I said above, in the FIP, I'd describe the requirements (linear lookback according to the fee structure), then describe how we do it in lotus in the design rational.

FIPS/fip-randomness.md

anorth · 2023-08-30T21:20:29Z

FIPS/fip-randomness.md

+## Security Considerations
+<!--All FIPs must contain a section that discusses the security implications/considerations relevant to the proposed change. Include information that might be important for security discussions, surfaces risks and can be used throughout the life cycle of the proposal. E.g. include security-relevant design decisions, concerns, important discussions, implementation-specific guidance and pitfalls, an outline of threats and risks and how they are being addressed. FIP submissions missing the "Security Considerations" section will be rejected. A FIP cannot proceed to status "Final" without a Security Considerations discussion deemed sufficient by the reviewers.-->
+
+This FIP is an overall improvement to security, as it is hardens some aspects of the system. This is especially important if we plan to eventually support native WASM actors.


I know it's a bit duplicative, but could you describe what aspects are hardened?

anorth · 2023-08-30T21:20:47Z

README.md

@@ -110,3 +110,4 @@ This improvement protocol helps achieve that objective for all members of the Fi
 |[0070](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0070.md)   | Allow SPs to move partitions between deadlines  | FIP  |Steven Li (@steven004), Alan Xu (@zhiqiangxu), Mike Li (@hunjixin), Alex North (@anorth), Nicola (@nicola)| Draft  |
 |[0071](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0071.md )   | Deterministic State Access (IPLD Reachability)  | FIP  |@stebalien| Draft  |
 |[0072](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0072.md)   | Improved event syscall API  | FIP  | @fridrik01, @Stebalien  | Draft  |
+|[XXXX](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-randomness.md)   | Improvements to the FVM randomness syscalls  | FIP  | @arajasek, @Stebalien  | Draft  |


Stebalien

nits. Basically, I like keeping the spec as clean as possible

FIPS/fip-randomness.md

Stebalien · 2023-08-30T21:50:04Z

FIPS/fip-randomness.md

+The client looks up the randomness from the chain (with no lookback limit), hashes it to "draw" the requested value, and returns it.
+This FIP proposes:
+1. moving the hashing operation out of the FVM/client, and into the calling actor
+1. formalizing that no lookback limit is enforced on randomness and tipset CID fetching, and modifying the gas pricing to be based on a skiplist of length 20


I... wouldn't include the whole skiplist thing, because that will still depend on the client. Any algorithm works as long as it has at most linear runtime with respect to the lookback length and it fits within the specified gas parameters.

We can describe our algorithm in the discussion, but it's not really a part of the spec.

Stebalien · 2023-08-30T21:56:42Z

FIPS/fip-randomness.md

+
+Clients no longer receive the `dst` and `entropy` parameters for the randomness Extern methods, and should skip the final step of "drawing" randomness in which this DST and entropy is mixed into the hash.
+
+We also seek consensus amongst Core Devs that Filecoin clients are expected to be able to support chain walkbacks all the way to genesis, with a performance no _worse_ than that based on a skiplist of length 20. Clients are free to be more performant (eg. constant-time).


We have a "single layer" skiplist. But yes, a "real" skiplist would be faster.

But as I said above, in the FIP, I'd describe the requirements (linear lookback according to the fee structure), then describe how we do it in lotus in the design rational.

Stebalien · 2023-08-30T22:02:19Z

FIPS/fip-randomness.md

+
+### Gas Pricing
+
+We change the gas pricing for both `get_randomness` operations, as well as the `get_tipset_cid` operation, to be proportional to the lookback distance. Specifically, we:


I'd just specify the values here and move the how down to the design rational. I.e.,

The spec cares about the actual gas fees and what they imply for the client.

The reviewers care about how we arrived at those gas fees. But that's basically: we picked them based on lotus and all existing clients use the same logic.

FIPS/fip-randomness.md

Stebalien · 2023-08-31T19:43:31Z

FIPS/fip-randomness.md

+
+The implementation of the `Runtime` interface for the FVM has to change to:
+- no longer supply the `dst` and `entropy` params when calling into the FVM SDK
+- hash the randomness returned by the FVM with the `dst` and `entropy` params


Let's reproduce the function here.

Stebalien · 2023-08-31T19:44:49Z

FIPS/fip-randomness.md

+### Gas Pricing
+
+We change the gas pricing for both `get_randomness` operations, as well as the `get_tipset_cid` operation, to be proportional to the lookback distance. Specifically, we set it to:
+`75 * lookback + 110200 + 15000 + 21000`, where `lookback` is equal to the difference between the current epoch, and the requested epoch.


I would still simplify this to two parameters:

The per-tipset cost.

The offset.

arajasek · 2023-09-01T13:59:51Z

@Stebalien Thanks for review, all feedback addressed!

BlocksOnAChain · 2023-09-05T10:37:46Z

@arajasek - do we still plan to merge this security improvement and get the FIP number + move it to the next call during this week?
CC: @Stebalien for visibility and potential final reviews

arajasek · 2023-09-05T13:32:31Z

@arajasek - do we still plan to merge this security improvement and get the FIP number + move it to the next call during this week? CC: @Stebalien for visibility and potential final reviews

Yup, that's the plan. Gonna give folks a little break before poking for more review.

Stebalien · 2023-09-05T22:52:28Z

FIPS/fip-randomness.md

+```
+pub fn draw_randomness(
+    rbase: &[u8; RANDOMNESS_LENGTH],
+    pers: DomainSeparationTag,


I.e., make this code stand by itself.

Stebalien · 2023-09-05T22:52:32Z

FIPS/fip-randomness.md

+pub fn draw_randomness(
+    rbase: &[u8; RANDOMNESS_LENGTH],
+    pers: DomainSeparationTag,
+    round: ChainEpoch,


Stebalien · 2023-09-05T22:52:41Z

FIPS/fip-randomness.md

+
+```
+pub fn draw_randomness(
+    rbase: &[u8; RANDOMNESS_LENGTH],


hard code or define the length.

Stebalien · 2023-09-05T22:53:47Z

FIPS/fip-randomness.md

+We charge `75 * lookback + 5800 * 19` for the lookback cost, where `lookback` is equal to the difference between the current epoch, and the requested epoch.
+We additionally charge `15000` for the offset (the cost of producing the randomness/tipset CID itself), and `21000` for the extern cost.
+
+The total gas formula is thus: `75 * lookback + 110200 + 15000 + 21000`.


Please include the final number. I'd prefer to just drop all math from the spec and move it to the rational, but we can keep it if you feel strongly about it. However, we should include the final number regardless.

arajasek · 2023-09-06T13:20:37Z

@Stebalien Review addressed, thank you!

arajasek requested review from momack2, jennijuju, kaitlin-beegle, anorth and raulk as code owners August 30, 2023 16:36

arajasek requested a review from Stebalien August 30, 2023 16:37

anorth reviewed Aug 30, 2023

View reviewed changes

Stebalien reviewed Aug 30, 2023

View reviewed changes

Stebalien reviewed Aug 31, 2023

View reviewed changes

arajasek force-pushed the asr/fip-randomness branch from 5fb25c4 to 68cc231 Compare September 1, 2023 13:58

arajasek requested a review from jsoares as a code owner September 1, 2023 13:58

arajasek mentioned this pull request Sep 1, 2023

feat: pricelist: implement randomness and tipset_cid repricing filecoin-project/ref-fvm#1872

Merged

Add FVM Randomness FIP

9889db4

arajasek force-pushed the asr/fip-randomness branch from 68cc231 to 9889db4 Compare September 5, 2023 13:33

Stebalien reviewed Sep 5, 2023

View reviewed changes

Stebalien approved these changes Sep 6, 2023

View reviewed changes

anorth approved these changes Sep 6, 2023

View reviewed changes

address review: simplify the spec section

576d2ef

arajasek force-pushed the asr/fip-randomness branch from 37c598e to 576d2ef Compare September 7, 2023 13:09

arajasek merged commit 3412bed into master Sep 7, 2023

arajasek deleted the asr/fip-randomness branch September 7, 2023 13:12

rjan90 mentioned this pull request Mar 20, 2024

FIP 0075 contains an invalid created date in the frontmatter #969

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add FVM Randomness FIP #817

Add FVM Randomness FIP #817

arajasek commented Aug 30, 2023 •

edited

Loading

anorth Aug 30, 2023

anorth Aug 30, 2023

anorth Aug 30, 2023

anorth Aug 30, 2023

Stebalien Aug 30, 2023

anorth Aug 30, 2023

anorth Aug 30, 2023

Stebalien left a comment

Stebalien Aug 30, 2023

Stebalien Aug 30, 2023

Stebalien Aug 30, 2023

Stebalien Aug 30, 2023

Stebalien Aug 31, 2023

Stebalien Aug 31, 2023

arajasek commented Sep 1, 2023

BlocksOnAChain commented Sep 5, 2023

arajasek commented Sep 5, 2023

Stebalien Sep 5, 2023

Stebalien Sep 5, 2023

Stebalien Sep 5, 2023

Stebalien Sep 5, 2023

Stebalien Sep 5, 2023

arajasek commented Sep 6, 2023

	category (*only required for Standard Track): <Core>
	category: Core


		Clients no longer receive the `dst` and `entropy` parameters for the randomness Extern methods, and should skip the final step of "drawing" randomness in which this DST and entropy is mixed into the hash.

		We also seek consensus amongst Core Devs that Filecoin clients are expected to be able to support chain walkbacks all the way to genesis, with a performance no _worse_ than that based on a skiplist of length 20. Clients are free to be more performant (eg. constant-time).


		### Gas Pricing

		We change the gas pricing for both `get_randomness` operations, as well as the `get_tipset_cid` operation, to be proportional to the lookback distance. Specifically, we:

Add FVM Randomness FIP #817

Add FVM Randomness FIP #817

Conversation

arajasek commented Aug 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stebalien left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arajasek commented Sep 1, 2023

BlocksOnAChain commented Sep 5, 2023

arajasek commented Sep 5, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arajasek commented Sep 6, 2023

arajasek commented Aug 30, 2023 •

edited

Loading