Document how temporary storage is used #1801
Conversation
docs/Filesystem-Paths.md
Outdated
| appropriate location for each supported platform. As a result, temporary | ||
| storage should always be freed up automatically by the OS during normal | ||
| operations, such as when performing a system restart. However, bugs or abrupt |
There was a problem hiding this comment.
As a result, temporary storage should always be freed up automatically by the OS during normal operations, such as when performing a system restart.
I don't think Windows does this.
There was a problem hiding this comment.
@nwt: Thanks for mentioning this! My bad... I'd checked on macOS and Ubuntu and made a bit of "QED" leap. Indeed, I just ran a quick test and confirmed that Windows does not clean %TMP% on a reboot. Then I was reminded of the many free (or paid!) "cleanup" tools for Windows that purport to address this pressing challenge. 🙄
As if that's not bad enough, this made me paranoid enough that I figured I couldn't necessarily assume that what I observed for Ubuntu could hold for other Linux distros, so I tried CentOS. I figured if I covered at least those two I could call it a trend. Sure enough, CentOS didn't clean /tmp between reboots either!
I've now pushed a commit that's more vague about how reboots or "common cleanup utilities" might reap the temp space, but no guarantees. I know I'm probably overcompensating by saying any of this, but since in light of recent bugs we know Zed has the capacity to chow a whole lot of temp space, I'd like to be able to say we'd warned the people somewhere...
Co-authored-by: Noah Treuhaft <noah.treuhaft@gmail.com>
When working on an issue-at-scale recently with a community user, I became concerned that maybe our software was consuming too much temporary storage on their system. This made me realize that we're overdue to reveal this information so users have a chance of doing this kind of debug on our own, or at least I could easily point them to an article if this comes up again. As it turns out, I think this info grafts nicely onto the existing Filesystem Paths article.
While running tests to write this content, I actually found multiple bugs where we're not currently cleaning up after ourselves the way we should (brimdata/brimcap#136, brimdata/zed#2983). Normally in these situations I'd confess these issues in the article, link to them, and then update the article once they get fixed. However, I have high hopes we'll get these fixed pretty quick, so for now I've kept it tidy.