Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor HasherLookup #424

Merged
merged 5 commits into from
Oct 17, 2022
Merged

Refactor HasherLookup #424

merged 5 commits into from
Oct 17, 2022

Conversation

tohrnii
Copy link
Contributor

@tohrnii tohrnii commented Oct 7, 2022

Describe your changes

Partly addressing #360.

  • Remove HasherState from HasherLookup and get the state from main_trace instead.
  • Refactor hash_span_block to avoid using multiple is_memoized checks.

Benchmark:
Execution time for keccak256 reduced from 91.833 ms to 87.36 ms, an improvement of ~4.8% after removing HasherState from HasherLookup.

Checklist before requesting a review

  • Repo forked and branch created from next according to naming convention.
  • Commit messages and codestyle follow conventions.
  • Relevant issues are linked in the PR description.
  • Tests added for new functionality.
  • Documentation/comments updated according to changes.

@tohrnii tohrnii force-pushed the hasher_refactor branch 5 times, most recently from 8cc48fd to a861a05 Compare October 12, 2022 12:22
@tohrnii tohrnii marked this pull request as ready for review October 12, 2022 12:24
Copy link
Contributor

@grjte grjte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tohrnii , that's a great improvement on keccak! I left a few comments inline. Also, it would be great if you can search the processor crate for "TODO" and make any other updates related to removing the state from the HasherLookup. I think there are a couple other ones still hanging around

processor/src/chiplets/hasher/lookups.rs Show resolved Hide resolved
processor/src/trace/utils.rs Outdated Show resolved Hide resolved
processor/src/trace/utils.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/lookups.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/lookups.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/lookups.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/mod.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/mod.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/mod.rs Outdated Show resolved Hide resolved
processor/src/chiplets/hasher/mod.rs Outdated Show resolved Hide resolved
@tohrnii tohrnii requested a review from grjte October 13, 2022 11:55
Copy link
Contributor

@grjte grjte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tohrnii! There are just a couple more small things that I left inline

@@ -122,7 +122,6 @@ impl HasherTrace {
self.append_row(selectors, &hasher_state, node_index);
}

// TODO: remove this after the state is removed from the HasherLookup struct
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the state parameter and the code below? The point of this comment was that I thought we wouldn't need these once the state was removed from the lookups, since this method appends the rows to the trace directly. Have I missed something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need it to get the final state to check that the resultant state digest is the same as the expected hash.
I thought your comment was to avoid copying the state for intermediate states after removing lookups but since we only copy the trace once at the end in each case now, we only copy the final state.

Copy link
Contributor

@grjte grjte Oct 17, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm yeah, I see. However, the result (which comes from the digest in this state array) and expected_hash are only checked in debug mode, which means this state is only used in debug mode, so I'm not sure we want to be passing it around and going through this extra work when not in debug.

I think what we want is to move that debug_assert_eq from the chiplets module into the hasher module. We would then check the assertion in debug mode in 2 cases:

  1. after building the trace for the first time (in the control block merge methods before saving the memoized version)
  2. after copying the trace (inside the copy_trace method) so that the work required for the check only occurs in debug mode

@bobbinth what are your thoughts?

If we do this, we can do it as a separate PR so this one can be merged

processor/src/range/request.rs Outdated Show resolved Hide resolved
processor/src/trace/decoder/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@grjte grjte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, thanks @tohrnii! As noted, I think there's something we want to revisit, but we can wait for Bobbin to weigh in and then address it in a future PR

@grjte grjte merged commit 9b5d8de into 0xPolygonMiden:next Oct 17, 2022
@grjte grjte mentioned this pull request Oct 17, 2022
26 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants