Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MB-62427: seeking the pointer correctly while reading segmentBase data #252

Merged
merged 3 commits into from
Jul 26, 2024

Conversation

Thejas-bhat
Copy link
Member

@Thejas-bhat Thejas-bhat commented Jul 24, 2024

v16 file format introduced the concept of sections in which as part of metadata the fieldsIndex part was replaced by a sectionsIndex which serves the same purpose of fieldsIndex of pointing to a file offset where the actual data of field's section is stored.

The PR tries to resolve the situation - where there was only one field's data being written out while creating the in-memory segmentBase and the process was crashing while initializing the fields related information in the loadFieldsNew() API at

zapx/segment.go

Line 323 in 0c7027f

numFields, sz := binary.Uvarint(s.mem[pos : pos+binary.MaxVarintLen64])

The reason for the crash is while writing out the data, because there is only one field metadata being written out at the very end of the buffer

zapx/write.go

Line 89 in 0c7027f

rv = uint64(w.Count())
is worth 9 bytes. However we were doing an out-of-bounds access over here

zapx/segment.go

Line 323 in 0c7027f

numFields, sz := binary.Uvarint(s.mem[pos : pos+binary.MaxVarintLen64])
because binary.MaxVarintLen64 = 10.

The solution proposed here is to guard this kind of access by keeping in mind the length of the backing buffer like how it was done in v15

zapx/segment.go

Line 268 in a72794c

fieldsIndexEnd := uint64(len(s.mem))

@Thejas-bhat Thejas-bhat changed the title WIP handling buffer overflow bug fix: seeking the pointer appropriately while reading a segment's data Jul 25, 2024
@Thejas-bhat Thejas-bhat marked this pull request as ready for review July 25, 2024 13:52
@Thejas-bhat Thejas-bhat changed the title bug fix: seeking the pointer appropriately while reading a segment's data MB-62427: seeking the pointer appropriately while reading segmentBase data Jul 25, 2024
@Thejas-bhat Thejas-bhat changed the title MB-62427: seeking the pointer appropriately while reading segmentBase data MB-62427: seeking the pointer correctly while reading segmentBase data Jul 25, 2024
abhinavdangeti
abhinavdangeti previously approved these changes Jul 25, 2024
@abhinavdangeti abhinavdangeti dismissed their stale review July 25, 2024 16:28

Pending one clarification

@abhinavdangeti abhinavdangeti merged commit 097ce48 into master Jul 26, 2024
6 checks passed
@abhinavdangeti abhinavdangeti deleted the overflow branch July 26, 2024 14:38
moshaad7 pushed a commit that referenced this pull request Sep 12, 2024
#252)

* handling buffer overflow while creating new segmentBase

* cleaning up code instrumentation

* more comments around the edge case
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants