Cache: selective resource caching

[krizhanovsky]

We must be able to cache or bypass the cache selectively depending on resource URI. Following configuration must be used to fulfill a request from the cache (also please refer #471 since the extension is required for Web-server mode):

    location prefix "/static/" {
            cache_fulfill suffix ".jpg" ".JPG" ".png" ".PNG";
    }

It's also required to be able to bypass cache for specific URLs:

    location suffix ".php" {
            cache_bypass prefix "/nocache/";
    }

Simple array-based matched should be implemented. Array could be limited by 32 or 64 items.

location isn't mandatory grouping. Rather, it's just a base for grouing HTTP management directives. In this case location just help to match prefix and suffix of URI. So cache_bypass and cache_fulfill can be defined at top level of configuration file.

[krizhanovsky]

Lets consider this example which should give an idea how the operation should work:

    1       cache 1;
    2 
    3       cache_fulfill suffix ".css";
    4       cache_bypass prefix "/scripts/";
    5
    6       location prefix "/static/" {
    7               cache_fulfill suffix ".png";
    8               cache_fulfill suffix ".jpg";
    9               cache_bypass prefix "/static/html";
    10      }

1st line defines overall cache mode: switched on/of. If cache 0, then just ignore any cache_fulfill/cache_bypass directives, Tempesta works as pure HTTP proxy w/o any caching.

Lines 3 and 4 defines default policy. If they don't match, then do not do any caching. This means that cache isn't checked at all for the request and corresponding response also isn't stored in the cache.

Few examples:

1. GET /: firstly check location - it doesn't match, check lines 3 & 4 - no match, so there is no caching for the request.
2. Add cache_fulfill * * at line 5, so this wildcard matches everything, so 1st example will be fulfilled from the cache. In this case we also can safely remove 3rd line since it's overwritten by the new line. Note that sequence of the directives is important, such that if we place the new directive at line 2, then 3rd and 4th lines will have no effect.
3. GET /static/img/1.png: location matches, 7th line matches, so request is fulfilled from the cache or forwarded to upstream if there is not cached entry and received response is cached;
4. GET /scripts/do.pl: location doesn't match, check 3rd line - no match, 4th line patches and we bypass the cache;
5. GET /static/styles/main.css: location matches, but all the lines in the location don't match, so check 3rd line - matches, the request is processed by cache;
6. if we replace 9th line by cache_bypass * *, then 5th example will be handles by the line and request will bypass the cache.

More notes about other caching directives. #494 defines which methods are cacheable and which aren't. Request method is simple integer and there is bitmask for the allowed methods, so it's efficient to check request method before location and any caching directives: if the method shouldn't be cached, then request isn't processed by cache and location and cache_bypass/cahe_fulfill directives aren't checked at all.

Next, there are HTTP cache control headers, e.g. Cache-Control which can contradict cache_bypass/cahe_fulfill directives for particular request (e.g. it can match the directive by URL, but contain Cache-Control: no-cache header). Issue #530 should resolve the question: it provides a configuration option which explicitly say to Tempesta what should be ignored at any particular case.

However, default behavior must be do not cache. This is absolutely strong requirement because caching can break an application (just imagine if some dynamic web chat is cached by some mistake - all the chat users won't see new messages!). Next thing is security - if we cache something sensitive by mistake, we can provide huge security flaw. So by default we should not cache. An administrator must explicitly and very carefully specify what to cache. If he/she want's default policy to cache everything, they should explicitly set cache_fulfill * *.

[keshonok]

Implemented in 28889de (#543).