Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Lower default spill run size to reduce overhead memory usage #7463

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

zhztheplayer
Copy link
Member

@zhztheplayer zhztheplayer commented Oct 10, 2024

It's observed in CI Q67/Q95 OOM test that large global spill pool usage is causing overhead memory OOM. The PR tries to tweak on default config option of Velox spill run size to fix the issue.

The OOM message was like:

"allocateNonContiguous failed with 2 pages from Memory Pool[__sys_spilling__ LEAF root[__sys_root__] parent[__sys_root__] MALLOC no-usage-track thread-safe]<unlimited max capacity unlimited capacity used 0B available 0B reservation [used 0B, reserved 0B, min 0B] counters [allocs 149202, frees 121609, reserves 0, releases 0, collisions 0])> Exceeded memory allocator limit when allocating 2 new pages, the memory allocator capacity is 1.35GB"

@github-actions github-actions bot added CORE works for Gluten Core VELOX INFRA labels Oct 10, 2024
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

@FelixYBW
Copy link
Contributor

why recent velox PR caused the error?

Copy link

Run Gluten Clickhouse CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CORE works for Gluten Core INFRA VELOX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants