return zip instead of tar or gzip #724
Conversation
|
Instance profiling may need to be updated as well (which is what issue targets at) |
Yep, working on it now. |
|
TODO:
Tested in windows and it works fine. @breeswish PTAL, thanks! |
pkg/apiserver/logsearch/pack.go
Outdated
|
|
||
| _, err = io.Copy(tw, f) | ||
| err = utils.CreateZipPack(temp, filePaths) |
There was a problem hiding this comment.
Will it be too slow to zip again when there are a lot of logs? (for example, 5 GB)
Note that previously it is a tar stream on the fly using io.Copy, which is roughly a memory copy and does not need to wait for everything to finish as well.
There was a problem hiding this comment.
Yep, this may be a problem, let me check it.
There was a problem hiding this comment.
zip provides an option to allow us not to compress the files, ref: https://stackoverflow.com/a/23717581/2998877 , has updated the code.
There was a problem hiding this comment.
stream the zip pack, ref: https://stackoverflow.com/a/57434338/2998877
|
Hi @breeswish , now the zip pack file is streamed, and works fine when testing, PTAL, thanks! ref: https://stackoverflow.com/questions/57429256/how-to-generate-zip-7z-archive-on-the-fly-in-a-http-server-using-gin/57434338#57434338 |
|
Good job! Mostly lgtm. The only thing I'm worrying is that you are using defer in loops, which usually behaves like something you are not expecting. Would you like to move it to a standalone function so that defer will run immediately after each loop iteration? |
Ok, let me try. |
(cherry picked from commit 1dc0480)
* feature: Query Editor (#713) * Backend: replace UNIX_TIMESTAMP with FROM_UNIXTIME in statement query (#731) * test: Stablize e2e tests (#732) * ui: Add store location tree (#728) * log_search: Display instance port (#722) * ui: show store location topology (#719) * ui: Online Config (#733) * ui: Support sharing session (#741) * doc: Update to sig-diagnosis (#742) * statement: Display number of plans in the list (#746) * log search: return zip instead of tar or gzip (#724)
resolve #718