Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update PostingsList's chunkSize only when err == nil #151

Merged
merged 1 commit into from
Apr 25, 2023

Conversation

abhinavdangeti
Copy link
Member

@abhinavdangeti abhinavdangeti commented Apr 11, 2023

Possibly related to: blevesearch/bleve#1811

@@ -303,12 +303,14 @@ func (rv *PostingsList) read(postingsOffset uint64, d *Dictionary) error {
return fmt.Errorf("error loading roaring bitmap: %v", err)
}

rv.chunkSize, err = getChunkSize(d.sb.chunkMode,
chunkSize, err := getChunkSize(d.sb.chunkMode,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A doubt here, from the code:

	case chunkMode == 1026:
		// improve upon the ideas tested in chunkMode 1025
		// the observation that the fewest number of dense chunks is the most
		// desirable layout, given the built-in assumptions of chunking
		// (that we want to put an upper-bound on the number of items you must
		//  walk over without skipping, currently tuned to 1024)
		//
		// 1.  compute the number of chunks needed (max 1024/chunk)
		// 2.  convert to chunkSize, dividing into maxDocs
		numChunks := (cardinality / 1024) + 1
		chunkSize := maxDocs / numChunks

		return chunkSize, nil

it seems like we could potentially return a zero chunkSize if the passed value of d.sb.numDocs is zero. Do we handle that part as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya good point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants