Add documentation

README.md

@@ -298,6 +298,7 @@ Documentation for Snap will be kept in this repository for now with an emphasis
 * [plugin life cycle](docs/PLUGIN_LIFECYCLE.md)
 * [plugin signing](docs/PLUGIN_SIGNING.md)
 * [tribe](docs/TRIBE.md)
+* [secure plugin communication](docs/SECURE_PLUGIN_COMMUNICATION.md)
 
 To learn more about Snap and how others are using it, check out our [blog](https://medium.com/intel-sdi). A good first post to read is [My How-to for the Snap Telemetry Framework](https://medium.com/intel-sdi/my-how-to-for-the-snap-telemetry-framework-e3bb641bc740#.6f5nk543t) by @mjbrender.
 

docs/SECURE_PLUGIN_COMMUNICATION.md

@@ -0,0 +1,124 @@
+<!--
+http://www.apache.org/licenses/LICENSE-2.0.txt
+
+
+Copyright 2017 Intel Corporation
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Secure Plugin Communication
+
+Snap communicates with plugins over gRPC protocol, which in general transfers data in plaintext.
+Snap allows securing communication with plugins by opening TLS channels and using certificates to authenticate plugins and framework.
+
+## Usage 
+
+This walkthrough assumes you have downloaded a Snap release as described in [Getting Started](../README.md#getting-started).
+
+### Shortest guide
+
+Assuming all the test files are available, the following steps will result in secure plugin communication:
+
+```
+snapteld --log-level 1 --plugin-trust 0 --tls-cert /tmp/snaptest-cli.crt --tls-key /tmp/snaptest-cli.key --ca-cert-paths /tmp/snaptest-ca.crt
+## (in another terminal)
+## Load each plugin
+snaptel plugin load --plugin-cert /tmp/snaptest-srv.crt --plugin-key /tmp/snaptest-srv.key --plugin-ca-certs /tmp/snaptest-ca.crt plugins/snap-plugin-collector-rand
+## Start a sample task
+snaptel task create -t sample-task.json
+```
+
+### Detailed preparation
+
+Starting secure communication requires following steps:
+1. Obtain TLS certificate and private key for framework.
+    * Please note that this certificate should allow usage for TLS web client authentication (as specified in RFC 3280)
+1. Obtain TLS certificate and private key for each plugin or group of plugins.
+    * Please note that this certificate should allow usage for TLS web server authentication (as specified in RFC 3280)
+1. Obtain and locate the CA certificates that are necessary to authenticate framework and plugin certificates.
+
+Process of acquiring a TLS certificate is a complex one. Every organization has its specific rules on security, thus the details are not given here.
+
+We do provide a short guide on obtaining self-signed certificates that may be used for tests outside production environment; see [Obtaining self-signed TLS certificates for tests](#obtaining-self-signed-tls-certificates-for-tests).
+
+### Enabling secure communication
+
+Secure communication is enabled by passing the required paths to programs: `snapteld`, and plugin (via `snaptel`). The minimum paths necessary are:
+* for `snapteld`: `--tls-cert`, `--tls-key`,
+* for plugins (`snaptel`): `--plugin-cert`, `--plugin-key`.
+
+The required paths are sufficient and necessary to enable TLS. Daemon (`snapteld`) and plugins (via `snaptel`) will refuse to start if certificate or key file argument is missing.
+
+### Using system-installed CA certificates
+
+Framework and plugins need CA certificates to validate each other's certificate. The CA certificates may be obtained in two ways:
+* by passing a list of CA certificate paths directly as a parameter, e.g.: `--ca-cert-paths=/tmp/small-setup-ca.crt:/tmp/medium-setup-ca.crt:/tmp/ca-certs/`, `--plugin-ca-certs`
+    * plugin as well as framework will examine each path in the list and either load a file directly or list directory contents and load the enumerated files (e.g.: the files in `/tmp/ca-certs/` folder) 
+* by relying on default CA certificate discovery mechanism
+    * plugin and framework will by default load certificates from system (if no paths were given as parameter). Each OS has its own specific locations, e.g.: `/etc/ssl/certs` on Ubuntu. This mechanism is provided by Go language, and is only available on selected OSes.     
+    System CA certificates may also be loaded explicitly by listing system locations explicitly, e.g.: `--ca-cert-paths /etc/ssl/certs:/tmp/snaptest-ca.crt`
+
+## More information
+
+### Exclusive security
+
+It's important to note that once secure plugin communication is enabled in framework, only secure connections may be established. 
+
+In other words: attempting to load insecure plugin in framework will result in an error. 
+
+### Relation to other functionalities
+
+Several modes of operation do not fully support secure communication:
+* distributed workflow is not covered by secure communication,
+* tribe doesn't support secure communication; `snapteld` will refuse to start in tribe mode if configured with secure communication,
+* plugin and task autodiscovery doesn't support secure communication; `snapteld` will refuse to start with autodiscovery path and secure communication enabled.
+
+### Obtaining self-signed TLS certificates for tests
+
+The following intstructions will result in TLS certificate files. These files may be used for manual tests.
+1. Install tool [certstrap](https://github.com/square/certstrap) for generating test certificates. Further steps will assume that `certstrap` is available under `$PATH` location.
+1. Generate root CA certificate:
+    ```
+    certstrap init --cn "snaptest-ca" --o "snaptest" --ou "ca" --key-bits 2048 --years 1 --passphrase '
+    ```
+1. **optional** Install root CA certificate in the system:
+    ```
+    sudo cp out/snaptest-ca.crt /usr/local/share/ca-certificates/; sudo update-ca-certificates --verbose --fresh
+    ``` 
+1. Generate server certificate and key to use with plugins:
+    ```
+    certstrap request-cert --cn "snaptest-srv" --ip "127.0.0.1" --domain "localhost" --passphrase '' --key-bits 2048 --o "snaptest" --ou "server"
+    certstrap sign "snaptest-srv" --CA "snaptest-ca" --passphrase '' --years 1
+    ```
+1. Generate client certificate and key to use with `snapteld`:
+    ```
+    certstrap request-cert --cn "snaptest-cli" --ip "127.0.0.1" --domain "localhost" --passphrase '' --key-bits 2048 --o "snaptest" --ou "client"
+    certstrap sign "snaptest-cli" --CA "snaptest-ca" --passphrase '' --years 1
+    ```
+1. Copy server and client certificates into common location, e.g.: `/tmp`:
+    ```
+    for unit in srv cli; do for fname in crt key; do cp out/snaptest-$unit.$fname /tmp; done; done
+    ```
+The following files are relevant for running the tests:
+* `/tmp/snaptest-cli.crt`, `/tmp/snaptest-cli.key` - these are the certificate and private key files for `snapteld`,
+* `/tmp/snaptest-srv.crt`, `/tmp/snaptest-srv.key` - these are the certificate and private key files for plugin,
+* `/tmp/snaptest-ca.crt` - this is the CA certificate that should be used to authenticate framework and plugins.
+
+## More information
+
+* [SNAPTELD_CONFIGURATION.md](SNAPTELD_CONFIGURATION.md)
+* [SNAPTELD](SNAPTELD.md)
+* [SNAPTELD](SNAPTELD.md)
+* [TRIBE.md](TRIBE.md)
+* [DISTRIBUTED_WORKFLOW_ARCHITECTURE](DISTRIBUTED_WORKFLOW_ARCHITECTURE.md)

docs/SNAPTEL.md

@@ -87,9 +87,9 @@ help, h     Shows a list of commands or help for one command
 $ snaptel plugin command [command options] [arguments...]
 ```
 ```
-load        load <plugin_path> [--plugin-cert=<plugin_cert_path> --plugin-key=<plugin_key_path>]
+load        load <plugin_path> [--plugin-cert=<plugin_cert_path> --plugin-key=<plugin_key_path> --plugin-ca-certs=<ca_cert_paths>]
 unload      unload <plugin_type> <plugin_name> <plugin_version>
-swap        swap <load_plugin_path> <unload_plugin_type>:<unload_plugin_name>:<unload_plugin_version> or swap <load_plugin_path> -t <unload_plugin_type> -n <unload_plugin_name> -v <unload_plugin_version> [--plugin-cert=<plugin_cert_path> --plugin-key=<plugin_key_path>
+swap        swap <load_plugin_path> <unload_plugin_type>:<unload_plugin_name>:<unload_plugin_version> or swap <load_plugin_path> -t <unload_plugin_type> -n <unload_plugin_name> -v <unload_plugin_version> [--plugin-cert=<plugin_cert_path> --plugin-key=<plugin_key_path> [--plugin-ca-certs=<ca_cert_paths>] ]
 list        list
 help, h     Shows a list of commands or help for one command
 ```
@@ -205,3 +205,6 @@ $ snaptel plugin unload collector mock <version>
 $ snaptel plugin unload processor passthru <version>
 $ snaptel plugin unload publisher publisher <version>
 ```
+
+## More information
+* [SECURE_PLUGIN_COMMUNICATION](SECURE_PLUGIN_COMMUNICATION.md)

docs/SNAPTELD.md

@@ -44,8 +44,9 @@ $ snapteld [global options] command [command options] [arguments...]
 --control-listen-port value                  Listen port for control RPC server (default: 8082) [$SNAP_CONTROL_LISTEN_PORT]
 --control-listen-addr value                  Listen address for control RPC server [$SNAP_CONTROL_LISTEN_ADDR]
 --temp_dir_path value                        Temporary path for loading plugins [$SNAP_TEMP_DIR_PATH]
---tls-cert value                             A path to PEM-encoded certificate to use for TLS channels
---tls-key value                              A path to PEM-encoded private key file to use for TLS channels
+--tls-cert value                             A path to PEM-encoded certificate for framework to use for securing communication channels to plugins over TLS
+--tls-key value                              A path to PEM-encoded private key file for framework to use for securing communication channels to plugins over TLS
+--ca-cert-paths                              List of paths (directories/files) to CA certificates for validating plugin certificates in secure TLS communication
 --work-manager-queue-size value              Size of the work manager queue (default: 25) [$WORK_MANAGER_QUEUE_SIZE]
 --work-manager-pool-size value               Size of the work manager pool (default: 4) [$WORK_MANAGER_POOL_SIZE]
 --disable-api, -d                            Disable the agent REST API
@@ -74,6 +75,8 @@ $ snapteld --version
 $ snapteld --log-level 4
 $ snapteld --auto-discover /opt/snap/plugins/
 $ snapteld --log-level 1 --plugin-trust 2 --keyring-paths /etc/snap/keyrings
+$ snapteld --log-level 1 --tls-cert /etc/snap/cert/snapteld.crt --tls-key /etc/snap/key/snapteld.key
+--ca-cert-paths /etc/ssl/certs/sample_organization_CA.crt:/etc/snap/ca/
 ```
 
 ### Debug output
@@ -157,3 +160,4 @@ INFO[2017-01-09T12:55:05-08:00] snapteld started
 * [REST_API.md](REST_API.md)
 * [PLUGIN_SIGNING.md](PLUGIN_SIGNING.md)
 * [TRIBE.md](TRIBE.md)
+* [SECURE_PLUGIN_COMMUNICATION](SECURE_PLUGIN_COMMUNICATION.md)

docs/SNAPTELD_CONFIGURATION.md

@@ -108,6 +108,17 @@ control:
   # before failing. Snap will not disable a plugin due to failures when this value is -1.
   max_plugin_restarts: 10
 
+  ## Secure plugin communication optional parameters:
+  # tls_cert_path sets the TLS certificate path to enable secure plugin communication
+  # and authenticate itself to plugins. Requires also: tls_key_path.
+  tls_cert_path: /tmp/snaptest-cli.crt
+  # tls_key_path sets the TLS key path to enable secure plugin communication and
+  # authenticate itself to plugins. Requires also: tls_cert_path.
+  tls_key_path: /tmp/snaptest-cli.key
+  # ca_cert_paths sets the list of filesystem paths (files/directories) to CA certificates
+  # for use in validating
+  ca_cert_paths: /tmp/small-setup-ca.crt:/tmp/medium-setup-ca.crt:/tmp/ca-certs/
+
   # plugins section contains plugin config settings that will be applied for
   # plugins across tasks.
   plugins:

examples/configs/snap-config-sample.json

@@ -15,6 +15,9 @@
         "keyring_paths":"/etc/snap/keyrings",
         "temp_dir_path":"/tmp",
         "plugin_trust_level":0,
+        "tls_cert_path": "/tmp/snaptest-cli.crt",
+        "tls_key_path": "/tmp/snaptest-cli.key",
+        "ca_cert_paths": "/tmp/small-setup-ca.crt:/tmp/medium-setup-ca.crt:/tmp/ca-certs/",
         "plugins":{
             "all":{
                 "password":"p@ssw0rd"

examples/configs/snap-config-sample.yaml

@@ -72,6 +72,17 @@ control:
   # before failing. Snap will not disable a plugin due to failures when this value is -1.
   max_plugin_restarts: 10
 
+  ## Secure plugin communication optional parameters:
+  # tls_cert_path sets the TLS certificate path to enable secure plugin communication
+  # and authenticate itself to plugins. Requires also: tls_key_path.
+  tls_cert_path: /tmp/snaptest-cli.crt
+  # tls_key_path sets the TLS key path to enable secure plugin communication and
+  # authenticate itself to plugins. Requires also: tls_cert_path.
+  tls_key_path: /tmp/snaptest-cli.key
+  # ca_cert_paths sets the list of filesystem paths (files/directories) to CA certificates
+  # for use in validating
+  ca_cert_paths: /tmp/small-setup-ca.crt:/tmp/medium-setup-ca.crt:/tmp/ca-certs/
+
   # plugins section contains plugin config settings that will be applied for
   # plugins across tasks.
   plugins: