Avoid accessing empty vector in realloc_deepdata in checkCoreFile
#1648
Conversation
…e()` The code was taking `&(*ud)[0]` even when `ud` is null (in the case of no samples to decode), and even though it wasn't actually referencing the value, the address sanitizer catches the reference, and it appears GCC 14 aborts with an failed assert. This hopefully address AcademySoftwareFoundation#1639. Signed-off-by: Cary Phillips <cary@ilm.com>
There was a problem hiding this comment.
huh, I don't believe ud is actually null - that (in this case anyway) is always passed in from a stack variable. Rather, the vector itself is empty, meaning bytes is calculated to be 0 (which happens an empty scanline with no deep samples), so it's actually the [] operator doing math on a null pointer, at least that's what I see. I have a fix for that (such that it ignores this step entirely on an empty scanline), which I believe to be the more appropriate fix
|
I'd still argue that |
|
oh, probably, I write &(foo[0]) to be more generic, but it's not a generic routine so using data would be clearer... although if all we did was change it to ->data(), it would just fail a few lines later, adding an offset to a null pointer anyway... I will switch it to ->data which will come in some later PR |
The code was taking
&(*ud)[0]even whenudis null (in the case of no samples to decode), and even though it wasn't actually referencing the value, the address sanitizer catches the reference, and it appears GCC 14 aborts with an failed assert.This hopefully address #1639.