Skip to content

Commit

Permalink
Documentation updates for UDP, graphite, collectd listeners
Browse files Browse the repository at this point in the history
  • Loading branch information
sparrc committed Nov 5, 2015
1 parent e0bcb57 commit d219361
Show file tree
Hide file tree
Showing 5 changed files with 90 additions and 6 deletions.
15 changes: 15 additions & 0 deletions services/collectd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,18 @@ The path to the collectd types database file may also be set
## Large UDP packets

Please note that UDP packages larger than the standard size of 1452 are dropped at the time of ingestion, so be sure to set `MaxPacketSize` to 1452 in the collectd configuration.

## Config Example

```
[collectd]
enabled = false
bind-address = ":25826" # the bind address
database = "collectd" # Name of the database that will be written to
retention-policy = ""
batch-size = 5000 # will flush if this many points get buffered
batch-pending = 10 # number of batches that may be pending in memory
batch-timeout = "10s"
read-buffer = 8388608 # (8*1024*1024) UDP read buffer size
typesdb = "/usr/share/collectd/types.db"
```
3 changes: 2 additions & 1 deletion services/collectd/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ const (
DefaultTypesDB = "/usr/share/collectd/types.db"

// DefaultUDPReadBuffer is the default UDP read buffer
// increasing this increases the number of UDP packets that can be handled.
// Sets the size of the operating system's receive buffer associated with
// the UDP traffic
DefaultReadBuffer = 8 * 1024 * 1024
)

Expand Down
20 changes: 19 additions & 1 deletion services/graphite/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ If you need to add the same set of tags to all metrics, you can define them glob
#]
```

## Customized Config
## Customized Config
```
[[graphite]]
enabled = true
Expand All @@ -167,3 +167,21 @@ If you need to add the same set of tags to all metrics, you can define them glob
".measurement*",
]
```

## Two graphite listener, UDP & TCP, Config

```
[[graphite]]
enabled = true
bind-address = ":2003"
protocol = "tcp"
# consistency-level = "one"
[[graphite]]
enabled = true
bind-address = ":2004" # the bind address
protocol = "udp" # protocol to read via
udp-read-buffer = 8388608 # (8*1024*1024) UDP read buffer size
```


3 changes: 2 additions & 1 deletion services/graphite/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ const (
DefaultBatchTimeout = time.Second

// DefaultUDPReadBuffer is the default UDP read buffer
// increasing this increases the number of UDP packets that can be handled.
// Sets the size of the operating system's receive buffer associated with
// the UDP traffic
DefaultUDPReadBuffer = 8 * 1024 * 1024
)

Expand Down
55 changes: 52 additions & 3 deletions services/udp/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,62 @@
# Configuration
# The UDP Input

## Configuration

Each UDP input allows the binding address, target database, and target retention policy to be set. If the database does not exist, it will be created automatically when the input is initialized. If the retention policy is not configured, then the default retention policy for the database is used. However if the retention policy is set, the retention policy must be explicitly created. The input will not automatically create it.

Each UDP input also performs internal batching of the points it receives, as batched writes to the database are more efficient. The default _batch size_ is 1000, _pending batch_ factor is 5, with a _batch timeout_ of 1 second. This means the input will write batches of maximum size 1000, but if a batch has not reached 1000 points within 1 second of the first point being added to a batch, it will emit that batch regardless of size. The pending batch factor controls how many batches can be in memory at once, allowing the input to transmit a batch, while still building other batches.

# Processing
## Processing

The UDP input can receive up to 64KB per read, and splits the received data by newline. Each part is then interpreted as line-protocol encoded points, and parsed accordingly.

# UDP is connectionless
## UDP is connectionless

Since UDP is a connectionless protocol there is no way to signal to the data source if any error occurs, and if data has even been successfully indexed. This should be kept in mind when deciding if and when to use the UDP input. The built-in UDP statistics are useful for monitoring the UDP inputs.

## Config Examples

One UDP listener

```
# influxd.conf
...
[[udp]]
enabled = true
bind-address = ":8089" # the bind address
database = "telegraf" # Name of the database that will be written to
batch-size = 5000 # will flush if this many points get buffered
batch-timeout = "1s" # will flush at least this often even if the batch-size is not reached
batch-pending = 10 # number of batches that may be pending in memory
read-buffer = 8388608 # (8*1024*1024) UDP read buffer size
...
```

Multiple UDP listeners

```
# influxd.conf
...
[[udp]]
# Default UDP for Telegraf
enabled = true
bind-address = ":8089" # the bind address
database = "telegraf" # Name of the database that will be written to
batch-size = 5000 # will flush if this many points get buffered
batch-timeout = "1s" # will flush at least this often even if the batch-size is not reached
batch-pending = 10 # number of batches that may be pending in memory
read-buffer = 8388608 # (8*1024*1024) UDP read buffer size
[[udp]]
# High-traffic UDP
enabled = true
bind-address = ":80891" # the bind address
database = "mymetrics" # Name of the database that will be written to
batch-size = 5000 # will flush if this many points get buffered
batch-timeout = "1s" # will flush at least this often even if the batch-size is not reached
batch-pending = 100 # number of batches that may be pending in memory
read-buffer = 33554432 # (32*1024*1024) UDP read buffer size
...
```


0 comments on commit d219361

Please sign in to comment.