Skip to content

Commit

Permalink
Merge pull request #660 from tempesta-tech/ab-msg-seq
Browse files Browse the repository at this point in the history 
Enforce the correct order of responses. Handle non-idempotent requests.
  • Loading branch information
@keshonok
keshonok committed Mar 3, 2017
2 parents 56b0ecb + 5d5983f commit c40924b
Show file tree
Hide file tree
Showing 40 changed files with 3,279 additions and 1,093 deletions.
162 changes: 135 additions & 27 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ ssl_certificate_key /path/to/tfw-root.key;

Also, `proto=https` option is needed for the `listen` directive.

#### Self-signed certificate genration
#### Self-signed certificate generation

In case of using a self-signed certificate with Tempesta, it's
convenient to use OpenSSL to generate a key and a certificate. The
Expand Down Expand Up @@ -301,6 +301,31 @@ just one example:
curl -X PURGE http://192.168.10.10/
```

#### Non-Idempotent Requests

The consideration of whether a request is considered non-idempotent may
depend on specific application, server, and/or service. A special directive
allows the definition of a request that will be considered non-idempotent:
```
nonidempotent <METHOD> <OP> <ARG>;
```
`METHOD` is one of supported HTTP methods, such as GET, HEAD, POST, etc.
`OP` is a string matching operator, such as `eq`, `prefix`, etc.
`ARG` is an argument for `OP`, such as `/foo/bar.html`, `example.com`, etc.

One or more of this directive may be specified. The directives apply to one
or more locations as defined below in the [Locations](#Locations) section.

If this directive is not specified, then a non-idempotent request in defined
as a request that has an unsafe method.

Below are examples of this directive:
```
nonidempotent GET prefix "/users/";
nonidempotent POST prefix "/users/";
nonidempotent GET suffix "/data";
```

### Locations

Location is a way of grouping certain directives that are applied only
Expand All @@ -321,9 +346,10 @@ location <OP> "<string>" {
Multiple locations may be defined. Location directives are processed
strictly in the order they are defined in the configuration file.

Only caching policy directives may currently be grouped by the location
directive. Caching policy directives defined outside of any specific
location are considered the default policy for all locations.
Only caching policy directives and the `nonidempotent` directive may
currently be grouped by the location directive. The directives defined
outside of any specific location are considered the default policy for
all locations.

When locations are defined in the configuration, the URL of each request
is matched against strings specified in the location directives and using
Expand Down Expand Up @@ -351,6 +377,7 @@ location prefix "/society/" {
cache_bypass prefix "/society/breaking_news/";
cache_fulfill suffix ".jpg" ".png";
cache_fulfill suffix ".css";
nonidempotent GET prefix "/society/users/";
}
```

Expand All @@ -367,14 +394,72 @@ server <IPADDR>[:<PORT>] [conns_n=<N>];
IPv6 address must be enclosed in square brackets (e.g. "[::0]" but not "::0").
`PORT` defaults to 80 if not specified.
`conns_n=<N>` is the number of parallel connections to the server.
`N` defaults to 4 if not specified.
`N` defaults to 32 if not specified.

Multiple back end servers may be defined. For example:
```
server 10.1.0.1;
server [fc00::1]:80;
```

if a connection with a server is terminated for any reason, an effort is made
to restore the connection. Sometimes the effort is futile. The directive
`server_connect_retries` sets the maximum number of re-connect attempts after
which the server connection is considered dead. It is defined as follows:
```
server_connect_retries <N>;
```
If this directive is not defined, then the number of re-connect attempts
defaults to 10. The value of zero specified for `N` means unlimited number
of attempts.

This is an important directive which controls how Tempesta deals with
outstanding requests in a failed connection. If the connection is restored
within the specified number of attempts, then all outstanding requests are
re-forwarded to the server. However if it's not restored, then the server
connection is considered dead, and all outstanding requests are re-scheduled
to other servers and/or connections.

If a server connection fails intermittenly, then requests may sit in the
connection's forwarding queue for some time. The following directives set
certain allowed limits before these requests are considered failed:
```
server_forward_retries <N>;
server_forward_timeout <N>;
```

`server_forward_retries` sets the maximum number of attempts to re-forward
a request to a server. If not defined, the default number of attempts is 5.
The value of zero specified for `N` means unlimited number of attempts.

`server_forward_timeout` set the maximum time frame in seconds within which
a request may still be forwarded. If not defined, the default time frame
is 60 seconds. The value of zero specified for `N` means unlimited timeout.

When one or both of these limits is exceeded for a request, the request is
evicted and an error is returned to a client.

Note that while requests in a connection are re-forwarded or re-scheduled,
that connection is not schedulable, which means it's not available to
schedulers for new incoming requests.

When re-forwarding or re-scheduling requests in a failed server connection,
a special consideration is given to non-idempotent requests. Usually
a non-idempotent request is not re-forwarded or re-scheduled. That may be
changed with the following directive that doesn't have arguments:
```
server_retry_nonidempotent;
```

Each server connection has a queue of forwarded requests. The size of the
queue is limited with `server_queue_size` directive as follows:
```
server_queue_size <N>;
```
Each connection to the server has the same limit on the queue size set
with this directive. If not specified, the queue size is set to 1000.


#### Server Groups

Back end servers can be grouped together into a single unit for the purpose of
Expand All @@ -383,33 +468,39 @@ The load is distributed evenly among servers within a group.
If a server goes offline, other servers in a group take the load.
The full syntax is as follows:
```
srv_group <NAME> [sched=<SCHED_NAME>] {
srv_group <NAME> {
server <IPADDR>[:<PORT>] [conns_n=<N>];
...
}
```
`NAME` is a unique identifier of the group that may be used to refer to it
later.
`SCHED_NAME` is the name of scheduler module that distributes load among
servers within the group. Default scheduler is used if `sched` parameter is
not specified.

Servers that are defined outside of any group implicitly form a special group
called `default`.

All server-related directives listed in [Servers](#Servers) section above
are applicable for definition for a server group. Also, a scheduler may be
speficied for a group.

Below is an example of server group definition:
```
srv_group static_storage sched=hash {
srv_group static_storage {
sched hash;
server 10.10.0.1:8080;
server 10.10.0.2:8080;
server [fc00::3]:8081 conns_n=1;
server_queue_size 500;
server_forward_timeout 30;
server_connect_retries 15;
}
```

#### Schedulers

Scheduler is used to distribute load among known servers. The syntax is as
follows:
Scheduler is used to distribute load among servers within a group. The group
can be either explicit, defined with `srv_group` directive, or implicit.
The syntax is as follows:
```
sched <SCHED_NAME>;
```
Expand All @@ -423,14 +514,22 @@ scheduler.
Requests are distributed uniformly, and requests with the same URI/Host are
always sent to the same server.

If no scheduler is defined, then scheduler defaults to `round-robin`.
The round-robin scheduler is the fastest scheduler. However, the presence
of a non-idempotent request in a connection means that subsequent requests
may not be sent out until a response is received to the non-idempotent
request. With that in mind, an attempt is made to put new requests to
connections that don't currently have non-idempotent requests. If all
connections have a non-idempotent request in them, then such a connection
is used as there's no other choice.

Only one `sched` directive is allowed per explicit or implicit group.
A scheduler defined for the implicit group becomes the scheduler for an
explicit group defined with `srv_group` directive if the explicit group
is missing the `sched` directive.

The defined scheduler affects all server definitions that are missing a
scheduler definition. If `srv_group` is missing a scheduler definition,
and there is a scheduler defined, then that scheduler is set for the group.
If no scheduler is defined for a group, then scheduler defaults
to `round-robin`.

Multiple `sched` directives may be defined in the configuration file.
Each directive affects server groups that follow it.

#### HTTP Scheduler

Expand Down Expand Up @@ -713,22 +812,31 @@ and running. Below is an example of the command to show the statistics,
and the output:
```
$ cat /proc/tempesta/perfstat
Client messages received : 450
Client messages forwarded : 450
SS pfl hits : 5836412
SS pfl misses : 5836412
Cache hits : 0
Cache misses : 0
Client messages received : 2918206
Client messages forwarded : 2918206
Client messages served from cache : 0
Client messages parsing errors : 0
Client messages filtered out : 0
Client messages other errors : 0
Client connections total : 30
Clients online : 0
Client connection attempts : 2048
Client established connections : 2048
Client connections active : 0
Client RX bytes : 47700
Server messages received : 447
Server messages forwarded : 447
Client RX bytes : 309329836
Server messages received : 2918206
Server messages forwarded : 2918206
Server messages parsing errors : 0
Server messages filtered out : 0
Server messages other errors : 0
Server connections total : 2220
Server connections active : 4
Server RX bytes : 153145
Server connection attempts : 8896
Server established connections : 8896
Server connections active : 32
Server connections schedulable : 32
Server RX bytes : 11494813434
```

Also, there's Application Performance Monitoring statistics. These stats show
Expand Down
Loading

0 comments on commit c40924b

Please sign in to comment.