Merge pull request #288 from nornir-automation/develop

Release 2.0.0

.gitignore

@@ -101,5 +101,6 @@ output/
 .DS_Store
 
 .pytest_cache/
+.mypy_cache/
 
 .vscode

.readthedocs.yml

@@ -0,0 +1,11 @@
+---
+build:
+  image: latest
+
+python:
+  version: 3.6
+  pip_install: true
+
+formats: []
+
+requirements_file: docs/requirements.txt

.travis.yml

@@ -6,18 +6,21 @@ addons:
   - pandoc
 language: python
 python:
-- 2.7
-- 3.4
-- 3.5
 - 3.6
 matrix:
   include:
+  - python: 3.6
+    env: TOXENV=py36
   - python: 3.6
     env: TOXENV=sphinx
   - python: 3.6
     env: TOXENV=black
   - python: 3.6
     env: TOXENV=pylama
+  - python: 3.6
+    env: TOXENV=mypy
+  - python: 3.6
+    env: TOXENV=nbval
 install:
 - pip install tox tox-travis coveralls
 script:

CHANGELOG.rst

@@ -1,3 +1,19 @@
+2.0.0 - December 17 2018
+========================
+
+For details about upgrading to 2.0.0 see the [https://nornir.readthedocs.io/en/2.0.0-beta/upgrading/1_to_2.html](notes).
+
++ [CORE_ENHANCEMENTS] Lots of core enhancements, too many to document
++ [CORE_ENHANCEMENTS] Changes on how the inventory
++ [CORE_ENHANCEMENTS] New ``F`` object for advanced filtering of hosts [docs](file:///Users/dbarroso/workspace/nornir/docs/_build/html/howto/advanced_filtering.html)
++ [CORE_ENHANCEMENTS] Improvements on how to serialize/deserialize user facing data like the configuration and the inventory
++ [CORE_ENHANCEMENTS] Connections are now their own type of plugin
++ [CORE_ENHANCEMENTS] Ability to handle connections manually [docs](file:///Users/dbarroso/workspace/nornir/docs/_build/html/howto/handling_connections.html)
++ [CORE_BUGFIX] Lots
++ [PLUGIN_BUGFIX] Lots
++ [PLUGIN_NEW] netmiko_save_config
++ [PLUGIN_NEW] echo_data
+
 1.1.0 - July 12 2018
 ====================
 
@@ -6,7 +22,6 @@
 + [TESTS] replace nsot container with requests-mock #172 
 + [PLUGIN_IMPROVEMENT] Support SSH Agent forwarding for paramiko SSH connections #159
 + [PLUGIN_IMPROVEMENT] allow passing options to napalm getters #156 
-+ [CORE_NEW_FEATURE] Filter object #155
 + [PLUGIN_BUGFIX] Fix for SSH and API port mapping issues #154
 + [CORE_NEW_FEATURE] add to_dict function so the inventory is serializable #146
 + [CORE_BUGFIX] Fix issues with using built-in and overwriting variable with loop variable #144 

CONTRIBUTING.rst

@@ -70,7 +70,7 @@ In order to run tests locally you need to have `Docker <https://docs.docker.com/
 
 .. code-block:: bash
 
-	pip -r requirements-dev.txt
+	pip install -r requirements-dev.txt
 
 Running tests
 -------------

README.md

@@ -3,7 +3,7 @@
 
 Nornir
 =======
-Nornir is a pure Python automation framework intented to be used directly from Python. While most automation frameworks use their own DSL which you use to describe what you want to have done, Nornir lets you control everything from Python.
+Nornir is a pure Python automation framework intented to be used directly from Python. While most automation frameworks use their own Domain Specific Language (DSL) which you use to describe what you want to have done, Nornir lets you control everything from Python.
 
 One of the benefits we want to highlight with this approach is the ease of troubleshooting, if something goes wrong you can just use your existing debug tools directly from Python (just add a line of `import pdb` & `pdb.set_trace()` and you're good to go). Doing the same using a DSL can be quite time consuming.
 

docs/_data_templates/configuration-parameters.j2

@@ -1,29 +1,29 @@
-The configuration parameters will be set by the :doc:`Nornir.core.configuration.Config </ref/api/configuration>` class.
+{% macro document_section(section, schema) %}
+{{ section }}
+{{ "-" * section|length }}
 
-{% for k, v in params|dictsort %}
-----------
+{% for k, v in schema["properties"].items() -%}
 
-{{ k }}
-----------------------------------
+``{{ k }}``
+{{ "_" * (k|length + 4) }}
 
+.. list-table::
+   :widths: 15 85
 
-.. raw:: html
+   * - **Description**
+     - {{ v["description"] }}
+   * - **Type**
+     - ``{{ v["type"] }}``
+   * - **Default**
+     - {{ "``{}``".format(v["default"]) if v["default"] else "" }}
+   * - **Required**
+     - ``{{ v["required"] or false }}``
+   * - **Environment Variable**
+     - ``{{ "NORNIR_{}_{}".format(section, k).upper() }}``
 
-	<table border="1" class="docutils">
-		<th>Environment variable</th>
-		<th>Type</th>
-		<th>Default</th>
-			<tr>
-			{% if v['type'] in ('str', 'int', 'bool') %}
-				<td>{{ v['env'] or 'BRIGADE_' + k|upper }}</td>
-			{% else %}
-				<td>N/A</td>
-			{% endif %}
-				<td>{{ v['type'] }}</td>
-				<td>{{ v['default_doc'] or v['default'] }}</td>
-			</tr>
-	</table>
-
-{{ v['description'] }}
+{% endfor %}
+{%- endmacro %}
 
+{% for k, v in schema["properties"].items() if k not in ["user_defined"] %}
+{{ document_section(k, v) }}
 {% endfor %}

docs/conf.py

@@ -20,16 +20,15 @@
 import os
 import sys
 
-from jinja2 import Environment
-from jinja2 import FileSystemLoader
+from jinja2 import Environment, FileSystemLoader
 
 sys.path.insert(0, os.path.abspath("../"))
 
+from nornir.core.deserializer.configuration import Config  # noqa
 
-from nornir.core.configuration import CONF  # noqa
 
 # -- General configuration ------------------------------------------------
-BASEPATH = os.path.dirname(__file__)
+BASEPATH = os.path.abspath(os.path.dirname(__file__))
 
 # If your documentation needs a minimal Sphinx version, state it here.
 #
@@ -54,7 +53,7 @@
 
 # General information about the project.
 project = "nornir"
-copyright = "2017, David Barroso"
+copyright = "2018, David Barroso"
 author = "David Barroso"
 
 # The version info for the project you're documenting, acts as replacement for
@@ -173,13 +172,18 @@
 ]
 
 
+def skip_slots(app, what, name, obj, skip, options):
+    if obj.__class__.__name__ == "member_descriptor":
+        return True
+    return None
+
+
 def build_configuration_parameters(app):
     """Create documentation for configuration parameters."""
-
     env = Environment(loader=FileSystemLoader("{0}/_data_templates".format(BASEPATH)))
     template_file = env.get_template("configuration-parameters.j2")
     data = {}
-    data["params"] = CONF
+    data["schema"] = Config.schema()
     rendered_template = template_file.render(**data)
     output_dir = "{0}/configuration/generated".format(BASEPATH)
     with open("{}/parameters.rst".format(output_dir), "w") as f:
@@ -189,7 +193,5 @@ def build_configuration_parameters(app):
 def setup(app):
     """Map methods to states of the documentation build."""
     app.connect("builder-inited", build_configuration_parameters)
+    app.connect("autodoc-skip-member", skip_slots)
     app.add_stylesheet("css/custom.css")
-
-
-build_configuration_parameters(None)

docs/configuration/index.rst

@@ -1,10 +1,25 @@
 Configuration
 =============
 
-Each configuration parameter are applied in the following order:
+The configuration is comprised of a set of sections and parameters for those sections. You can set the configuration programmatically using nornir by passing a dictionary of options for each section, by using a YAML file, by setting the corresponding environment variables or by a combination of the three. The order of preference from less to more preferred is "configuration file" -> "env variable" -> "code".
 
-1. Environment variable
-2. Parameter in configuration file / object
-3. Default value
+An example using ``InitNornir`` would be::
+
+    nr = InitNornir(
+        core={"num_workers": 20},
+        logging={"file": "mylogs", "level": "debug"}
+    )
+
+A similar example using a ``yaml`` file:
+
+.. include:: ../howto/advanced_filtering/config.yaml
+   :code: yaml
+
+Next, you can find each section and their corresponding options.
 
 .. include:: generated/parameters.rst
+
+user_defined
+------------
+
+You can set any ``<k, v>`` pair you want here and you will have it available under your configuration object, i.e. ``nr.config.user_defined.my_app_option``.