Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors when downloading gzipped objects in chunks from GCS #1009

Closed
mziccard opened this issue May 5, 2016 · 5 comments
Closed

Errors when downloading gzipped objects in chunks from GCS #1009

mziccard opened this issue May 5, 2016 · 5 comments
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. 🚨 This issue needs some love. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@mziccard
Copy link
Contributor

mziccard commented May 5, 2016

This issue emanates from an issue posted here.

We are trying to get a gzipped GCS file in chunks. The problem comes when we try to read the response content. The problem seems to be related with the fact that response.getContent() automatically creates a GZIPInputStream on top of the raw stream (if the content encoding is gzip). When we attempt to read
the GZIPInputStream given by getContent() we get an error as that stream only holds one chunk of the original file.

Would it be possible to somehow disable the wrapping of the response content into a GzipInputStream when calling getContent()?

@Capstan
Copy link

Capstan commented Jun 16, 2016

Or have it notice there's a Content-Range header and disable it itself.

@mziccard
Copy link
Contributor Author

Any updates on this?

@dalbani
Copy link

dalbani commented Oct 9, 2017

Hi, I'm very much new to this GCS library and I'm kind of surprised that this simple use case is not currently supported.
Well, a workaround has indeed been implemented AFAIK but I still fail to see how to properly handle reading a gzip-compressed blob from a ReadChannel, as in:

ReadChannel readChannel = storage.reader(blobId);

How can I then determine that the data coming out this channel is actually encoded (or not) using GZIP?
Thanks for your help.

@mattwhisenhunt mattwhisenhunt added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Jan 8, 2018
@JustinBeckwith JustinBeckwith added priority: p2 Moderately-important priority. Fix may not be included in next release. 🚨 This issue needs some love. and removed priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. labels Jun 7, 2018
@ajaaym
Copy link
Contributor

ajaaym commented Jan 24, 2019

@mziccard do you expect the chunks to be of compressed bytes or non compressed bytes?

@sduskis
Copy link

sduskis commented Mar 8, 2019

I'm going to close this due to staleness. Please open a new issue if this is still a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. 🚨 This issue needs some love. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

7 participants