Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Buffer: Use 256GB mmap size instead of MaxInt64 #198

Merged
merged 3 commits into from
Sep 30, 2020
Merged

Conversation

jarifibrahim
Copy link
Contributor

@jarifibrahim jarifibrahim commented Sep 28, 2020

MaxInt64 is 9.2 Exabyte and the test fails with cannot allocate memory on my computer.
This PR also fixes the build (it is failing on master).


This change is Reviewable

Copy link
Contributor

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @anurags92, @jarifibrahim, @karlmcguire, and @manishrjain)


z/buffer.go, line 73 at r1 (raw file):

// Newbuffer is a helper utility, which creates a virtually unlimited Buffer in UseCalloc mode.
func NewBuffer(sz int) *Buffer {
	buf, err := NewBufferWith(sz, math.MaxInt32, UseCalloc)

so this is now 2GB. not virtually unlimited anymore. Is that ok?

Copy link
Contributor Author

@jarifibrahim jarifibrahim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @anurags92, @karlmcguire, @manishrjain, and @martinmr)


z/buffer.go, line 73 at r1 (raw file):

Previously, martinmr (Martin Martinez Rivera) wrote…

so this is now 2GB. not virtually unlimited anymore. Is that ok?

I can make it 100*math.maxInt32 or something. math.MaxInt64 crashes on my computer.

@jarifibrahim jarifibrahim changed the title Buffer: Use MaxInt32 instead of MaxInt64 Buffer: Use 100xMaxInt32 instead of MaxInt64 Sep 28, 2020
@jarifibrahim jarifibrahim changed the title Buffer: Use 100xMaxInt32 instead of MaxInt64 Buffer: Use 256GB mmap size instead of MaxInt64 Sep 29, 2020
@martinmr martinmr merged commit 163c5d4 into master Sep 30, 2020
@martinmr martinmr deleted the ibrahim/buffer-fix branch September 30, 2020 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants