Skip to content

Commit

Permalink
Document --cache-by-content-type flag usage (#1251)
Browse files Browse the repository at this point in the history
* Document `--cache-by-content-type` flag usage

* Add doc about managing and passing plugin states

* Anchor gives issue 🙆‍♂️
  • Loading branch information
abhinavsingh authored Aug 11, 2022
1 parent bb55a04 commit be771d4
Showing 1 changed file with 47 additions and 0 deletions.
47 changes: 47 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
- [Redirect To Custom Server Plugin](#redirecttocustomserverplugin)
- [Filter By Upstream Host Plugin](#filterbyupstreamhostplugin)
- [Cache Responses Plugin](#cacheresponsesplugin)
- [Cache By Response Type](#cachebyresponsetype)
- [Man-In-The-Middle Plugin](#maninthemiddleplugin)
- [Proxy Pool Plugin](#proxypoolplugin)
- [Filter By Client IP Plugin](#filterbyclientipplugin)
Expand Down Expand Up @@ -116,6 +117,8 @@
- [Plugin Developer and Contributor Guide](#plugin-developer-and-contributor-guide)
- [High level architecture](#high-level-architecture)
- [Everything is a plugin](#everything-is-a-plugin)
- [Managing states for your stateless plugins](#managing-states-for-your-stateless-plugins)
- [Passing processing context between plugins](#passing-processing-context-between-plugins)
- [Internal Documentation](#internal-documentation)
- [Read The Doc](#read-the-doc)
- [pydoc](#pydoc)
Expand Down Expand Up @@ -778,6 +781,28 @@ Connection: keep-alive
}
```

### CacheByResponseType

`CacheResponsesPlugin` plugin can also automatically cache responses by `content-type`.
To try this, you must be running under [TLS Interception](#tls-interception) mode
and then pass `--cache-by-content-type` flag. Example:

```console
proxy \
--plugins proxy.plugin.CacheResponsesPlugin \
--cache-by-content-type \
--ca-key-file ca-key.pem \
--ca-cert-file ca-cert.pem \
--ca-signing-key ca-signing-key.pem
```

Make a few requests to the proxy server and you shall see data under `~/.proxy/cache` directory.

You should see 2 folders:

- `content`: Contains parsed `jpg`, `css`, `js`, `html`, `pdf` etc by content type
- `responses`: Contains raw responses as received _(of-course decrypted because of interception)_

### ManInTheMiddlePlugin

Modifies upstream server responses.
Expand Down Expand Up @@ -2218,6 +2243,28 @@ Within `proxy.py` everything is a plugin.
Use this flag with `--enable-web-server` flag to run `proxy.py` as a programmable
http(s) server.

## Managing states for your stateless plugins

Plugin class instances are created per-request. Most importantly,
plugin instances are created within CPU core context where the request
was received.

For above reason, global variables in your plugins may work as expected.
Your plugin code by design must be **stateless**.

To manage global states, you have a couple of options:
1) Make use of Python's [multiprocessing safe data structures](https://python.readthedocs.io/en/latest/library/multiprocessing.html#sharing-state-between-processes)
2) Make use of `proxy.py` in-built [eventing mechanism](https://github.com/abhinavsingh/proxy.py/blob/develop/tutorial/eventing.ipynb)

## Passing processing context between plugins

Sometimes, a plugin may need to pass additional context to other plugins after them in the processing chain. Example, this additional
context can also be dumped as part of access logs.

To pass processing context, make use of plugin's `on_access_log` method. See how [Program Name](https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/plugin/program_name.py) plugin modifies default `client_ip` key in the context and updates it to detected program name.

As a result, when we enable [Program Name Plugin](#programnameplugin), we see local client program name instead of IP address in the access logs.

## Development Guide

### Setup Local Environment
Expand Down

0 comments on commit be771d4

Please sign in to comment.