You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@hawyar further subtask breakdown, apologies yet again for starting with such a large scope
Upload events.json to the base directory as you have them (stake create)
shanejearley
changed the title
Add IoTeX essentials to parma crawler and push events to test bucket for Athena sandbox
Add IoTeX support to crawler and push events to test bucket for Athena sandbox
Jul 25, 2022
@hawyar let's use block filenames to start, so a flat directory with {blockid}-events.json files. The event count per block and data amount per event will be small and our query patterns will be more date/time focused – so we won't benefit much from the block partitions.
Instead, we'll implement partitions of our events by year, month, and day. We can also define these ahead of time in the Glue Metastore using the CDK. But we have no need to implement that just yet.
@hawyar let's further narrow this issue in on completing the event schema (1st draft to support all IoTeX Actions) and then adding those events to the offline, non-streaming version of the crawler. Our goal is to get a full dataset (to date, not real-time) in the event bucket.
Then we'll have you turn your focus back to block-by-block streaming. Then we'll attack Ethereum with a solid foundation.
No description provided.
The text was updated successfully, but these errors were encountered: