Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix for metadata caching #23

Merged
merged 1 commit into from
Jan 7, 2015
Merged

Conversation

jrfnl
Copy link
Contributor

@jrfnl jrfnl commented Jan 7, 2015

Sending this in as a separate PR as I want to keep it out of the discussion on PR #21.

I was wondering why the metadata cache files kept being updated (new save date/time) even when the cache shouldn't have expired yet.

That's when I found this. Basically any non-expired cache was being retrieved and then resaved with a new expiration timestamp, which - depending on the cache time chosen and how often the metadata gets retrieved - would mean that the cache effectively would never get refreshed.

@YahnisElsts
Copy link
Owner

Good catch. I can't believe I missed something like this.

Basically any non-expired cache was being retrieved and then resaved with a new expiration timestamp, which - depending on the cache time chosen and how often the metadata gets retrieved - would mean that the cache effectively would never get refreshed.

You mean it will be refreshed on every request, right?

@jrfnl
Copy link
Contributor Author

jrfnl commented Jan 7, 2015

I can't believe I missed something like this.

I missed it too until now....

You mean it will be refreshed on every request, right?

Actually, the way it was, the cache file would be resaved with a new expiration time, but with the old data. So except for the first time when the cache was created or in the unlikely case that no requests at all were made between the last save and the expiration time (1 week), the zip file would never be read out again for the metadata.

Now in a lot of ways this might not be a problem as when a new version of the file gets uploaded, it would have a new timestamp/filesize and therefore the cache would be invalid and a new one created, so data-wise there wasn't much of a problem.

However, what's the point of setting a cache with an expiration date if that expiration date is then effectively being overruled on every request/set to unlimited ?
Also: it meant effectively that no matter whether a cache was found or not we're writing/updating the cache file (not the data, just the file). Writing to disk on every request kind of defies the point of having a cache.

@YahnisElsts
Copy link
Owner

Actually, the way it was, the cache file would be resaved with a new expiration time, but with the old data.

In this case, the "old data" would be completely identical to the "new data" since the file hasn't changed. But yes, it's still a bug, and it leads to lots of superfluous disk writes. I'll merge your fix momentarily.

YahnisElsts added a commit that referenced this pull request Jan 7, 2015
Bug fix: Prevent lots of unnecessary metadata cache writes.
@YahnisElsts YahnisElsts merged commit 5068727 into YahnisElsts:master Jan 7, 2015
@jrfnl jrfnl deleted the Bug-fix-cache branch January 7, 2015 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants