Configuration (1.x)

#Use Archaius Turbine uses Archaius for it's configuration management. Archaius is simple to use and can support multiple sources of configuration including a simple config.properties file on your classpath to help you get started quickly. See the Archaius Getting Started Guide for more details.

##Turbine cluster configuration Turbine uses the concept of clusters which is just a group of servers that need to be monitored together. Turbine aggregates all metrics from a cluster of servers together, hence when you request the aggregate feed from Turbine, you need to specify the cluster query param.

Think of clusters as the group of all your app servers deployed to an env such as test or prod. Most folks use a single cluster for a single application in each of their environments. Hence for the app WeatherService, the clusters tend to be weather-svc-prod and weather-svc-test.

When you want metrics for a specific cluster, you can connect to Turbine to get your metrics like so

curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-prod"
curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-test"
curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-canary"

and so on..

Here is how you tell Turbine what clusters are there in the system.

turbine.aggregator.clusterConfig=weather-svc-test,weather-svc-prod,weather-svc-canary

You can also have multiple clusters for the same app in the same environment, and Turbine will monitor each of your clusters individually. Hence you can have a setup like weather-svc-prod-shard1 and weather-svc-prod-shard2.

Fetch these cluster metrics like so

curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-prod-shard1"
curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-prod-shard2"

Note that Turbine will not aggregate metrics from different clusters together into one group, since cluster is the grouping criteria used here. Hence you cannot get a single feed or even a union of feeds by requesting something like so

curl "http://{turbine-hostname}:8080/turbine.stream?cluster=weather-svc-prod-shard1,weather-svc-prod-shard2"

Host Connection URL config

Turbine needs to know how to connect to your servers. You do this using a connection url config property. You can have a global property that applies to all clusters, and you can also have a property for each cluster. The cluster specific property takes precedence over the global property. If it is absent, Turbine will just default to the global property. Examples.

Global Config

turbine.instanceUrlSuffix=:7080/my.custom.stream

Config only for canary cluster

turbine.instanceUrlSuffix.weather-svc-prod=:7080/my.custom.canary.stream

Note that these connection paths must be supported by your servers, else Turbine will not be able to connect to them. Also note the colon and the port number included in the config. Turbine connects to your servers like so ...

    String url =  "http://"  + hostname + urlConfigProperty.get();

Hence if your connection url to your host was http://{weather-svc-hostname}:7080/weatherService/hystrix.stream then your url connection config should be

turbine.instanceUrlSuffix=:7080/weatherService/hystrix.stream

InstanceDiscovery config

InstanceDiscovery component is used to tell Turbine what servers to connect to for each of your configured clusters. The InstanceDiscovery interface is really simple and looks like

public interface InstanceDiscovery {
    
    /**
     * Fetch the collection of Instances.  
     * @return Collection<Instance>
     * @throws Exception
     */
    public Collection<Instance> getInstanceList() throws Exception;
}

Where each Instance object represents each unique host for each of your applications and clusters monitored by Turbine.

And the really important properties of Instance required by Turbine are

    public class Instance implements Comparable<Instance> {

    private final String hostname; 
    private final String cluster;
    private final boolean isUp;
    private final Map<String, String> attributes; 

You can plugin your own InstanceDiscovery impl class, or use one of the simple implementations that are provided as defaults. See section below for using an existing impl of injecting your own. When writing your own, note the cluster param for the Instance obj here. This tells Turbine what cluster the host belongs to and hence Turbine can ensure that all metrics from this host get aggregated with all metrics from hosts that belong to the same cluster.

Hence the cluster param here must match one of the clusters that you configure for Turbine monitoring. See the above section on how to configure Turbine clusters.

Plugging in your own InstanceDiscovery impl

Use the config property InstanceDiscovery.impl Example

InstanceDiscovery.impl=com.foo.bar.MyInstanceDiscovery.class

If this property is not set, then Turbine uses the ConfigPropertyBasedDiscovery impl described below. Turbine also has a simple FileBasedInstanceDiscovery impl described below.

Important reminder!!

Again, please note that if you provide your own impl, you need to ensure that you set the right cluster param when creating and returning a collection of Instance objects.

Provided implementations for InstanceDiscovery

ConfigPropertyBasedDiscovery

By default Turbine looks for turbine.ConfigPropertyBasedDiscovery.default.instances to be set to the comma separated list of host names.

Hence, use this for the default cluster

turbine.ConfigPropertyBasedDiscovery.default.instances=[ comma separated list ]

If you have your own clusters defined using the turbine.aggregator.clusterConfig property, then you need to tell Turbine what hosts map to what cluster like so.

turbine.ConfigPropertyBasedDiscovery.[ clusterName ].instances= [list of instances]

Examples

turbine.ConfigPropertyBasedDiscovery.weather-svc-prod.instances = [list of prod instances]
turbine.ConfigPropertyBasedDiscovery.weather-svc-canary.instances = [list of canary instances]
turbine.ConfigPropertyBasedDiscovery.weather-svc-test.instances = [list of test instances]

FileBasedInstanceDiscovery impl

Turbine provides a utility for providing your host names using a file as well. First tell Turbine where to find the file

turbine.FileBasedInstanceDiscovery.filePath= [your file path]

and then tell Turbine to load the FileBasedInstanceDiscovery implementation

InstanceDiscovery.impl=com.netflix.turbine.discovery.FileBasedInstanceDiscovery

Your file format must look like this

{weather-svc-hostname1},weather-svc-prod,up
{weather-svc-hostname2},weather-svc-prod,down
{weather-svc-hostname3},weather-svc-prod,up
{weather-svc-hostname4},weather-svc-test,up
{weather-svc-hostname5},weather-svc-canary,up
    ...

Note the up/down status, this is used by Turbine to understand whether the host is available for streaming metrics or not. Turbine will disconnect from (or not connect to) instances which have status marked as down

EurekaInstanceDiscovery impl

Turbine provides a Eureka based implementation for InstanceDiscovery. See class EurekaInstanceDiscovery

First tell Turbine to load the EurekaInstanceDiscovery implementation

InstanceDiscovery.impl=com.netflix.turbine.discovery.EurekaInstanceDiscovery.class

Important note

This impl uses the cluster name from the asg name retrieved from Eureka. Hence if you use this impl, then you need to ensure one of 2 things.

• The asg name or your application vended by the Eureka client matches your configured Turbine cluster.
• OR you can override the getCluster() method in the class to provide your own mechanism to do so.

    protected String getClusterName(InstanceInfo iInfo) {
        return iInfo.getASGName();
    }