Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

split frames for large graphs #8684

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fjetter
Copy link
Member

@fjetter fjetter commented Jun 10, 2024

There's been a report about HLG serialization to be broken for very large graphs, see #5581 (comment) due to a msgpack limitation

I'm not entirely sure what's happening but this serialize/serialize_and_aplit thing may be related. We've seen slow transmissions of large grpahs in the past and I will have to review this closely because I don't want our transmission to be slower than it already is. This would break up the frames into 64MB chunks by default which might not be what we want.

Copy link
Contributor

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    29 files  ±0      29 suites  ±0   11h 4m 35s ⏱️ + 2m 3s
 4 058 tests ±0   3 957 ✅ ±0     97 💤 ±0  4 ❌ ±0 
55 897 runs  ±0  53 729 ✅  - 1  2 162 💤 ±0  6 ❌ +1 

For more details on these failures, see this check.

Results for commit 9ce84ab. ± Comparison against base commit 9672121.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant