Merge branch 'develop' into add_flatpak

Gemfile

@@ -25,3 +25,9 @@ group :vagrant do
   gem 'vagrant-wrapper'
   gem 'kitchen-vagrant'
 end
+
+group :macos do
+  gem 'rbnacl', '< 5.0', :require => false
+  gem 'rbnacl-libsodium', :require => false
+  gem 'bcrypt_pbkdf', '< 2.0', :require => false
+end

doc/ref/states/requisites.rst

@@ -155,17 +155,17 @@ module they are using.
 Requisites Types
 ----------------
 
-All requisite types have a corresponding `<requisite>_in <requisites-in>` form:
+All requisite types have a corresponding :ref:`<requisite>_in <requisites-in>` form:
 
-* `require <requisites-require>`: Requires that a list of target states succeed before execution
-* `onchanges <requisites-onchanges>`: Execute if any target states succeed with changes
-* `watch <requisites-watch>`: Similar to ``onchanges``; modifies state behavior using ``mod_watch``
-* `listen <requisites-listen>`: Similar to ``onchanges``; delays execution to end of state run using ``mod_wait``
-* `prereq <requisites-prereq>`: Execute prior to target state if target state expects to produce changes
-* `onfail <requisites-onfail>`: Execute only if a target state fails
-* `use <requisites-use>`: Copy arguments from another state
+* :ref:`require <requisites-require>`: Requires that a list of target states succeed before execution
+* :ref:`onchanges <requisites-onchanges>`: Execute if any target states succeed with changes
+* :ref:`watch <requisites-watch>`: Similar to ``onchanges``; modifies state behavior using ``mod_watch``
+* :ref:`listen <requisites-listen>`: Similar to ``onchanges``; delays execution to end of state run using ``mod_wait``
+* :ref:`prereq <requisites-prereq>`: Execute prior to target state if target state expects to produce changes
+* :ref:`onfail <requisites-onfail>`: Execute only if a target state fails
+* :ref:`use <requisites-use>`: Copy arguments from another state
 
-Several requisite types have a corresponding `requisite_any <requisites-any>` form:
+Several requisite types have a corresponding :ref:`requisite_any <requisites-any>` form:
 
 * ``require_any``
 * ``watch_any``
@@ -177,7 +177,7 @@ logic is desired instead of the default `OR` logic of onfail/onfail_any (which
 are equivalent).
 
 All requisites define specific relationships and always work with the dependency
-logic defined `above <requisites-matching>`.
+logic defined :ref:`above <requisites-matching>`.
 
 .. _requisites-require:
 

doc/topics/blackout/index.rst

@@ -10,17 +10,24 @@ Salt supports minion blackouts. When a minion is in blackout mode, all remote
 execution commands are disabled. This allows production minions to be put
 "on hold", eliminating the risk of an untimely configuration change.
 
-Minion blackouts are configured via a special pillar key, ``minion_blackout``.
+Minion blackouts are configured two ways either via a special pillar key, ``minion_blackout``.
+or a special grains key ``minion_blackout``.
 If this key is set to ``True``, then the minion will reject all incoming
 commands, except for ``saltutil.refresh_pillar``. (The exception is important,
 so minions can be brought out of blackout mode)
 
 Salt also supports an explicit whitelist of additional functions that will be
-allowed during blackout. This is configured with the special pillar key
-``minion_blackout_whitelist``, which is formed as a list:
+allowed during blackout. This is configured two ways as well. Either with the special pillar key
+or the special grains key ``minion_blackout_whitelist``, which is formed as a list:
 
 .. code-block:: yaml
 
     minion_blackout_whitelist:
       - test.ping
       - pillar.get
+
+When use a special pillar key ``minion_blackout`` then salt will get ``minion_blackout_whitelist`` from the
+pillar keys. And will get it from the grains when use ``minion_blackout`` as a special grains key.
+You therefore can strictly control ``minion_blackout`` status and ``minion_blackout_whitelist`` content by a minion side
+when you use a special grains key. A special grains key ``blackout_mode`` has higher priority
+than a special pillar key ``blackout_mode```

doc/topics/cloud/vmware.rst

@@ -227,6 +227,13 @@ Set up an initial profile at ``/etc/salt/cloud.profiles`` or
     Enter the name of the VM/template to clone from. If not specified, the VM will be created
     without cloning.
 
+``clonefrom_datacenter``
+    If the VM/template to clone exists in a different datacenter than the destination
+    datacenter, supply the source VM/template's datacenter here.
+    This defaults to the same value as ``datacenter``.
+
+    .. versionadded:: neon
+
 ``num_cpus``
     Enter the number of vCPUS that you want the VM/template to have. If not specified,
     the current VM/template\'s vCPU count is used.
@@ -525,7 +532,7 @@ Set up an initial profile at ``/etc/salt/cloud.profiles`` or
 
     .. note::
 
-    	Windows template should have "administrator" account.
+        Windows template should have "administrator" account.
 
 ``win_password``
     Specify windows vm administrator account password.
@@ -547,10 +554,10 @@ Set up an initial profile at ``/etc/salt/cloud.profiles`` or
     https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.UserData.html
 
 ``plain_text``
-	Flag to specify whether or not the password is in plain text, rather than encrypted.
-	VMware vSphere documentation:
+    Flag to specify whether or not the password is in plain text, rather than encrypted.
+    VMware vSphere documentation:
 
-	https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.Password.html
+    https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.Password.html
 
 ``win_installer``
     Specify windows minion client installer path
@@ -560,6 +567,54 @@ Set up an initial profile at ``/etc/salt/cloud.profiles`` or
 
     https://www.vmware.com/support/developer/vc-sdk/visdk25pubs/ReferenceGuide/vim.vm.customization.GuiRunOnce.html
 
+``win_ad_domain``
+    Specify the AD domain to join during customization.  ``win_ad_user`` and ``win_ad_password``
+    must also be specified.
+
+    Default is not set.
+
+    .. versionadded:: neon
+
+``win_ad_user``
+    Specify the user from ``win_ad_domain`` that will be used to join the computer to the domain
+    during customization.
+
+    Default is not set.
+
+    .. versionadded:: neon
+
+``win_ad_password``
+    Specify the password for the ``win_ad_user``.
+
+    Default is not set.
+
+    .. versionadded:: neon
+
+``win_autologon``
+    Specify if the local "Administrator" account should be logged in to the Windows machine
+    after the cloning process.
+
+    Defaults to 'True', must be 'True' for ``win_run_once`` to be executed.
+
+    .. versionadded:: neon
+
+``timezone``
+    Specify the timezone to apply to the VM during customization.
+
+    See https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.vm.customization.LinuxPrep.html for Linux timezone information.
+    See https://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.vm.customization.GuiUnattended.html for Windows timezone information.
+
+    Default is not set.
+
+    .. versionadded:: neon
+
+``hw_clock_utc``
+    Specify whether the hardware clock is in UTC or local time.
+
+    Default is not set.
+
+    .. versionadded:: neon
+
 Cloning a VM
 ============
 

doc/topics/development/modules/index.rst

@@ -340,7 +340,7 @@ SDB
 
 * :ref:`Writing SDB Modules <sdb-writing-modules>`
 
-SDB is a way to store data that's not associated with a minion. See 
+SDB is a way to store data that's not associated with a minion. See
 :ref:`Storing Data in Other Databases <sdb>`.
 
 Serializer
@@ -384,6 +384,12 @@ pkgfiles modules handle the actual installation.
 SSH Wrapper
 -----------
 
+.. toctree::
+    :maxdepth: 1
+    :glob:
+
+    ssh_wrapper
+
 Replacement execution modules for :ref:`Salt SSH <salt-ssh>`.
 
 Thorium

doc/topics/development/modules/ssh_wrapper.rst

@@ -0,0 +1,63 @@
+.. _ssh-wrapper:
+
+===========
+SSH Wrapper
+===========
+
+Salt-SSH Background
+===================
+
+Salt-SSH works by creating a tar ball of salt, a bunch of python modules, and a generated
+short minion config. It then copies this onto the destination host over ssh, then
+uses that host's local python install to run ``salt-client --local`` with any requested modules.
+It does not automatically copy over states or cache files and since it is uses a local file_client,
+modules that rely on :py:func:`cp.cache* <salt.modules.cp>` functionality do not work.
+
+SSH Wrapper modules
+===================
+
+To support cp modules or other functionality which might not otherwise work in the remote environment,
+a wrapper module can be created. These modules are run from the salt-master initiating the salt-ssh
+command and can include logic to support the needed functionality. SSH Wrapper modules are located in
+/salt/client/ssh/wrapper/ and are named the same as the execution module being extended. Any functions
+defined inside of the wrapper module are called from the ``salt-ssh module.function argument``
+command rather than executing on the minion.
+
+State Module example
+--------------------
+
+Running salt states on an salt-ssh minion, obviously requires the state files themselves. To support this,
+a state module wrapper script exists at salt/client/ssh/wrapper/state.py, and includes standard state
+functions like :py:func:`apply <salt.modules.state.apply>`, :py:func:`sls <salt.modules.state.sls>`,
+and :py:func:`highstate <salt.modules.state.highstate>`. When executing ``salt-ssh minion state.highstate``,
+these wrapper functions are used and include the logic to walk the low_state output for that minion to
+determine files used, gather needed files, tar them together, transfer the tar file to the minion over
+ssh, and run a state on the ssh minion. This state then extracts the tar file, applies the needed states
+and data, and cleans up the transferred files.
+
+Wrapper Handling
+----------------
+
+From the wrapper script any invocations of ``__salt__['some.module']()`` do not run on the master
+which is running the wrapper, but instead magically are invoked on the minion over ssh.
+Should the function being called exist in the wrapper, the wrapper function will be
+used instead.
+
+One way of supporting this workflow may be to create a wrapper function which performs the needed file
+copy operations. Now that files are resident on the ssh minion, the next step is to run the original
+execution module function. But since that function name was already overridden by the wrapper, a
+function alias can be created in the original execution module, which can then be called from the
+wrapper.
+
+Example
+```````
+
+The saltcheck module needs sls and tst files on the minion to function. The invocation of
+:py:func:`saltcheck.run_state_tests <salt.modules.saltcheck.run_state_tests>` is run from
+the wrapper module, and is responsible for performing the needed file copy. The
+:py:func:`saltcheck <salt.modules.saltcheck>` execution module includes an alias line of
+``run_state_tests_ssh = salt.utils.functools.alias_function(run_state_tests, 'run_state_tests_ssh')``
+which creates an alias of ``run_state_tests`` with the name ``run_state_tests_ssh``. At the end of
+the ``run_state_tests`` function in the wrapper module, it then calls
+``__salt__['saltcheck.run_state_tests_ssh']()``. Since this function does not exist in the wrapper script,
+the call is made on the remote minion, which then having the needed files, runs as expected.