Address comments

README.md

@@ -70,7 +70,7 @@ The key features of Snap are:
   * **Collectors** - Collectors consume telemetry data. Collectors are plugins for leveraging existing telemetry solutions (Facter, CollectD, Ohai) as well as specific plugins for consuming Intel telemetry (Node, DCM, NIC, Disk) and can reach into new architectures through additional plugins (see [Plugin Authoring below](#author-a-plugin)). Telemetry data is organized into a dynamically generated catalog of available data points.
   * **Processors** - Extensible workflow injection. Convert telemetry into another data model for consumption by existing systems. Allows encryption of all or part of the telemetry payload before publishing. Inject remote queries into workflow for tokens, filtering, or other external calls. Implement filtering at an agent level reducing injection load on telemetry consumer.
   * **Publishers** - Store telemetry into a wide array of systems. Snap decouples the collection of telemetry from the implementation of where to send it. Snap comes with a large library of publisher plugins that allow exposure to telemetry analytics systems both custom and common. This flexibility allows Snap to be valuable to open source and commercial ecosystems alike by writing a publisher for their architectures.
-  * **Streaming Collectors** - Streaming collectors act just like collectors, but rather than waiting for a specified duration before sending the collected data, they send it immediatly over a grpc stream to snaptel. Check out [STREAMING.md](/docs/STREAMING.md) for more details on the differences between collectors and streaming collectors. 
+  * **Streaming Collectors** - Streaming collectors act just like collectors, but rather than waiting for a specified duration before sending the collected data, they send it immediately over a grpc stream to snaptel. Check out [STREAMING.md](/docs/STREAMING.md) for more details on the differences between collectors and streaming collectors. 
 
 * **Dynamic Updates**: Snap is designed to evolve. Each scheduled workflow automatically uses the most mature plugin for that step, unless the collection is pinned to a specific version (e.g. get `/intel/psutil/load/load1/v1`). Loading a new plugin automatically upgrades running workflows in tasks. Load plugins dynamically, without a restart to the service or server. This dynamically extends the metric catalog when loaded, giving access to new measurements immediately. Swapping a newer version plugin for an old one in a safe transaction. All of these behaviors allow for simple and secure bug fixes, security patching, and improving accuracy in production.
 

docs/PLUGIN_AUTHORING.md

@@ -41,10 +41,10 @@ Before writing a new Snap plugin, please check out the [Plugin Catalog](./PLUGIN
 
 Snap supports four types of plugins:
 
-* collector: gathering metrics
+* collector: gathering metrics based on the specified interval
 * processor: transforming metrics
 * publisher: publishing metrics
-* streaming collector: streaming gathered metrics
+* streaming collector: gathering metrics when they are available 
 
 ### Plugin Name
 

docs/STREAMING.md

@@ -1,13 +1,13 @@
 # Streaming 
 
 
-Uses grpc streams to allow the plugin to send data immediately or after a certain period of time instead of on an interval governed by Snap.
+Streaming plugins use grpc streams to allow the plugin to send data immediately or after a certain period of time instead of on an interval governed by Snap.
 Streaming by snap enables - 
 * Improved performance by enabling event based data flows
-* Supports runtime configuration controlling event throughput
-* Buffer configurations enable dispatching events after a given duration and/or when a given event count has been reached
+* Runtime configuration controlling event throughput
+* Buffer configurations which dispatch events after a given duration and/or when a given event count has been reached
 
-Currently there are two plugins that support streaming - [snap relay] (https://github.com/intelsdi-x/snap-relay) and [snap-plugin-collector-rand-streaming] (https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/examples/snap-plugin-collector-rand-streaming). 
+Currently there are two plugins that support streaming - [snap relay](https://github.com/intelsdi-x/snap-relay) and [snap-plugin-collector-rand-streaming](https://github.com/intelsdi-x/snap-plugin-lib-go/tree/master/examples/snap-plugin-collector-rand-streaming). 
 
 # Configuration options 
 MaxCollectDuration and MaxMetricsBuffer are two configuration options that can be set through streaming task manifest or flags. 
@@ -31,37 +31,78 @@ MaxCollectDuration and MaxMetricsBuffer are two configuration options that can b
 ```
 
 # Streaming configuration flags
-Below is an example of the how to run the streaming rand collector using the configurable flags. 
+Below is an example of the how to run the snap-relay using the configurable flags. 
 1. Start the Snap daemon:
 * Run
 ```
 $ snapteld -l 1 -t 0
 ```
 The option "-l 1" is for setting the debugging log level and "-t 0" is for disabling plugin signing.
 
-2. Start the plugin using flags: 
-* Run snap-plugin-collector-rand-streaming plugin
+2. Start the relay using flags: 
+* Run snap-relay plugin
 ```
-go run main.go --stand-alone --max-collect-duration 10s --max-metrics-buffer 50
+$ go run main.go --stand-alone --stand-alone-port 8182 --log-level 5 --max-collect-duration 10s --max-metrics-buffer 50
+
+Output: Preamble URL: [::]:8182
+```
+
+3. Obtain meta information:
+* Curl preamble
+```
+$ curl localhost:8182
+{"Meta":{"Type":3,"Name":"plugin-relay","Version":1,"RPCType":3,"RPCVersion":1,"ConcurrencyCount":5,"Exclusive":false,"Unsecure":true,"CacheTTL":0,"RoutingStrategy":0,"CertPath":"","KeyPath":"","TLSEnabled":false,"RootCertPaths":""},"ListenAddress":"127.0.0.1:54541","PprofAddress":"0","Type":3,"State":0,"ErrorMessage":""}
+```
+
+4. Start client in snap-relay:
+* Run client (from a different terminal) using the listen address obtained from the previous step
+```
+$ go run client/main.go 127.0.0.1:54541
 ```
 
-3. Load the plugin:
-Run
+5. Load plugin:
 ```
 $ snaptel plugin load http://localhost:8182
+Plugin loaded
+Name: plugin-relay
+Version: 1
+Type: streaming-collector
+Signed: false
+Loaded Time: Tue, 05 Sep 2017 17:41:42 PDT
 ```
 
 * List the metric catalog by running:
 ```
 $ snaptel metric list
-NAMESPACE 		 VERSIONS
-/random/float 		 1
-/random/integer 	 1
-/random/string 		 1
+NAMESPACE 		           VERSIONS
+/intel/relay/collectd 	           1
+/intel/relay/statsd 	           1
+```
+
+6. Create a task manifest for snap-relay (see [exemplary files](https://github.com/intelsdi-x/snap-relay/tree/master/examples/tasks)):
+```
+---
+  version: 1
+  schedule:
+    type: "streaming"
+  workflow:
+    collect:
+      metrics:
+       /intel/relay/collectd: {}
+```
+
+7. Create a task:
+```
+$ snaptel task create -t collectd.yaml
+Using task manifest to create task
+Task created
+ID: c168b992-8aaf-4eec-8e3c-883510962789
+Name: Task-c168b992-8aaf-4eec-8e3c-883510962789
+State: Running
 ```
 
 # Metrics exposed by streaming collectors
-Below are some of the metrics collected by the streaming plugins currently. 
+Below are some of the metrics collected by the streaming plugins currently: 
 ```
 /intel/relay/collectd
 /intel/relay/statsd

docs/TASKS.md

@@ -184,6 +184,7 @@ Snap is designed in a way where custom schedulers can easily be dropped in. If a
   schedule:
     type: "streaming"
 ```
+The streaming schedule doesn't support fields such as `interval` and `count`. If those fields are provided as part of the schedule, they will simply be skipped. 
 For more details on streaming, visit [STREAMING.md](STREAMING.md)
 
 #### Max-Failures

examples/README.md

@@ -33,5 +33,4 @@ For additional examples of using Snap, checkout the examples in these repositori
  - [snap-plugin-publisher-influxdb](https://github.com/intelsdi-x/snap-plugin-publisher-influxdb)
  - [snap-plugin-publisher-graphite](https://github.com/intelsdi-x/snap-plugin-publisher-graphite)
  - [snap-plugin-publisher-file](https://github.com/intelsdi-x/snap-plugin-publisher-file)
- - [snap-relay](https://github.com/intelsdi-x/snap-relay) (streaming collector plugin)
- - [snap-plugin-collector-rand-streaming](https://github.com/intelsdi-x/snap-plugin-lib-go/blob/master/examples/snap-plugin-collector-rand-streaming) (streaming collector plugin)
+ - [snap-relay](https://github.com/intelsdi-x/snap-relay) (streaming collector plugin to retrieve data from collectd or statsd and include them into Snap workflow)