-
Notifications
You must be signed in to change notification settings - Fork 88
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VLenUTF8().encode(buffer) fails is buffer is read-only #514
Comments
Does |
Do you mean like:
Apparently not. I would have thought that this is handleable since |
Pandas has recently started wrapping the low-level arrays into immutable ones, which is maybe why you are seeing this now. I assume they internally access the low-level writable buffer somewhere. I think this is part of their move towards arrow, since arrow buffers are supposed to be immutable (which makes sense when there are offsets/indexes around, rather than just values). |
It looks like if you access the |
If you're not doing any Or we could require the caller to always provide a raw, writable numpy-like. |
My concern is that pandas may not intentionally be giving me a writable view, and may change this behaviour in the future. I was pointed at: For how pandas deals with this case. AFAICT, it's basically changing the typing from a |
I've opened a PR which should handle this on the numcodecs side. Does the approach look fine to you @martindurant? |
Yes, I suppose it's fine. We should maybe document this somewhere, since having to make a copy of the data, even temporarily, may surprise some people. |
Linking the upstream Cython issue: cython/cython#2485 |
Minimal, reproducible code sample, a copy-pastable example if possible
Problem description
Short description: this shouldn't error, as the codec shouldn't care whether it can write to the buffer it's passed.
Long description:
copy-on-write=True
on import in the latest releaseI can't think of a reason that
.encode
would need to modify the buffer, so it shouldn't care that it's read-only.Version and installation information
Please provide the following:
numcodecs.__version__
'0.12.1'Also, if you think it might be relevant, please provide the output from
pip list
orconda list
depending on which was used to install NumCodecs.conda list
The text was updated successfully, but these errors were encountered: