Ensure factor additions do not overflow index type #645
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eddelbuettel
requested review from
ihnorton,
aaronwolen,
johnkerl,
awenocur and
mojaveazure
|
This pull request has been linked to Shortcut Story #39073: Ensure factor additions do not overflow index type. |
awenocur
approved these changes
Jan 11, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a check for possible overflow in factor index level values when data is appended and factor levels grow. This arguably a corner case as most arrays (at least when created from R) will have an int32_t index. But as schemas can of course be based on factors with int8_t (and uint8_t, ...) it is possible. A unit test has been adding trying to write 130 distinct factor values which correctly errors with int8_t (max is 127 and lower) but passes with uint8_t.