statsdcc

---

Statsdcc is a Statsd-compatible high performance multi-threaded network daemon written in C++. It aggregates stats and sends aggregates to backends.

Dependencies

---

• openssl-devel-1.0.1e - libcrypto for MD4
• boost-devel-1.58.0 - lockfree queue
• boost-regex - regex
• gperftools-devel-2.0 - tcmalloc
• libmicrohttpd-0.9.33 - http server
• jsoncpp-1.6.1 - json parser
• googletest-1.7.0 - unit tests

Compilation

---

git clone https://github.com/wayfair/statsdcc.git
cd statsdcc
git submodule init
git submodule update
mkdir build && cd build
cmake -D test=true ..
make
ctest

Usage

---

To run proxy

./src/proxy <config file>

To run aggregator

./src/statsdcc <config file>

Configuration

---

Example configuration files are in statsdcc/etc

Common Configuration Variables

• servers: A list of server threads

  • udp: This variable is optional if tcp server is configured. To configure more than one threads to listen for metrics set variable threads. To override the size of udp recv buffer set recv_buffer variable. If recv_buffer not specified defaults to 8388608 bytes.

    "udp": {
    	"port": 9090,
    	"threads": 3,
    	"recv_buffer": 33554431
    }
    

  • tcp: This variable is optional if udp server is configured.

    "tcp": {
    	"port": 9090
    }
    

  • http: Http server listens for status check queries, by default uses port 8080.

    "http": {
    	"port": 8000
    }
    

  • ros: This server subscribes to topics with message type pal_statistics_msgs/Statistics. For more information on how to publish such topics see the PAL Statistics Framework . The topics' configuration needs to be setup on the ROS parameter server. Check the Proxy Configuration Variables section.

    "ros": {
        "node_name": "statsdcc"
    },
    

• workers: Number of worker threads to compute aggregations. If not specified defaults to 1.

• log_level: Possible values "debug"/"info"/"warn"/"error". If not specified defaults to "warn". If set only logs with the specified level or higher will be logged.

Aggregator Configuration Variables

• name: A custom name to the program. This name will be prefixed to status metrics generated by statsdcc. The default value is "statsdcc".

• prefix: A value that will be appended to all metrics.

• frequency: Interval in seconds to flush metrics to each backend. If not specified defaults to 10.

• percentiles: For time metrics, calculate the Nth percentile(s). If not specified defaults to [90].

  "percentiles": [80, 90]
  

• backends: List of backends to flush aggregations to. At least one of the following backends should be set.

  • stdout: Set to true to dump aggregations to stdout.

  • carbon: List of backend carbon instances.

    "carbon": [
    	{
    		"shard": "1",
    		"host": "localhost",
    		"port": 3101,
    		"vnodes": 1000,
    		"weight": 1
    	},
    	{
    		"shard": "2",
    		"host": "localhost",
    		"port": 3102,
    		"vnodes": 1000,
    		"weight": 1
    	}
    ]
    

    • shard: Key for this node used while building HashRing for consistent hashing.

    • host: Hostname for carbon instance. If not specified defaults to "127.0.0.1".

    • port: Tcp port number on which the carbon instance is listening for metrics. If not specified defaults to 3000.

    • vnodes: The amount of virtual nodes per server. Used for consistent hashing. Larger number gives bigger distribution in the HashRing. If not specified defaults to 1000.

    • weight: Similar to vnodes larger weight gives bigger distribution in the HashRing. If not specified defaults to 1.

• repeaters: Using repeaters you can flush the aggregates to another statsdcc-aggregator or statsdcc-proxy.

  "repeaters": [
  	{
  		"host": "localhost",
  		"port": 8126
  	}
  ]
  

Proxy Configuration Variables

• backends: List of backends to send metrics to, at least one of the following backends should be set.

  • stdout: Set to true to dump aggregations to stdout.

  • aggregator: List of backend aggregator instances.

  "aggregator": [
  	{
  		"host": "localhost",
  		"port": 9090
  	},
  	{
  		"host": "localhost",
  		"port": 9091
  	}
  ]
  

ROS Server Topics' Configuration

A topics parameter needs to be set under the ros server node namespace. topics must contain a list of topics names with a set of rules to indicate which stats should be aggregated.

Here is an example extracted from the standard pal_statsdcc_cfg topics configuration file. Check the rest of the package to get an idea on how the statsdcc is normally launched in the robot.

topics:
    - name: '/motors_statistics'
      stats:
          - name: 'motors.*.mode'
            type: ['g']
          - name: 'motors.*.(current|velocity|position|torque|abs_position|drive_temperature|motor_temperature|voltage)'
            type: ['t']
          - name: 'topic_stats.*.publish_async_(attempts|failures)'
            type: ['g']
          - name: 'topic_stats.*.last_async_pub_duration'
            type: ['t']

stats contains a list of the metrics that should be aggregated. As observed, you can use regex to simplify these rules. The stats can be of type 'gauge' or 'timer'.

• gauge: The result of aggregation will be just the last reported value. Denoted with type 'g'.
• timer: The result of aggregation will be the lower, upper, average and count of all values. Denoted with type 't'.

Developer's References

---

• UNIX Network Programming: W. Richard Stevens
• Advanced Programming in the UNIX Environment: W. Richard Stevens, Stephen A. Rago
• C++ Concurrency in Action: Practical Multithreading: Anthony Williams
• Effective Modern C++: Scott Meyers