spot.1

.TH "SPOT" 1.13  20240201T204425 spot manual
.\" Automatically generated by Pandoc 3.1.11.1
.\"
.SH Spot
Spot is a powerful and easy\-to\-use tool for effortless deployment and
configuration management.
It allows users to define a playbook with a list of tasks and targets,
where each task consists of a series of commands that can be executed on
remote hosts concurrently.
Spot supports running scripts, copying files, syncing directories, and
deleting files or directories, as well as custom inventory files or
inventory URLs.
.SS Getting Started
.IP \[bu] 2
Install Spot by download the latest release from the \c
.UR https://github.com/umputun/spot/releases
Releases
.UE \c
\ page.
.IP \[bu] 2
Create a configuration file, as shown in the example below, and save it
as \f[CR]spot.yml\f[R].
.IP \[bu] 2
Run Spot using the following command: \f[CR]spot\f[R].
This will execute all the tasks defined in the default
\f[CR]spot.yml\f[R] file for the \f[CR]default\f[R] target with a
concurrency of 1.
.IP \[bu] 2
To execute a specific task, use the \f[CR]\-\-task\f[R] flag:
\f[CR]spot \-\-task=deploy\-things\f[R].
This will execute only the \f[CR]deploy\-things\f[R] task.
.IP \[bu] 2
To execute a specific task for a specific target, use the
\f[CR]\-\-task\f[R] and \f[CR]\-\-target\f[R] flags:
\f[CR]spot \-\-task=deploy\-things \-\-target=prod\f[R].
This will execute only the \f[CR]deploy\-things\f[R] task for the
\f[CR]prod\f[R] target.
.RS 2
.RE
.PP
\f[B]Install from homebrew (macOS)\f[R]
.IP
.EX
brew tap umputun/apps
brew install umputun/apps/spot
.EE
.PP
\f[I]This will install both \f[CI]spot\f[I] and \f[CI]spot\-secrets\f[I]
binaries.\f[R]
.PP
\f[B]Install from deb package (Ubuntu/Debian)\f[R]
.IP "1." 3
Download the latest version of the package by running:
\f[CR]wget https://github.com/umputun/spot/releases/download/<versiom>/spot_<version>_linux_<arch>.deb\f[R]
(replace \f[CR]<version>\f[R] and \f[CR]<arch>\f[R] with the actual
values).
.IP "2." 3
Install the package by running:
\f[CR]sudo dpkg \-i spot_<version>_linux_<arch>.deb\f[R]
.PP
Example for the version 0.14.6 and amd64 architecture:
.IP
.EX
wget https://github.com/umputun/spot/releases/download/v0.14.6/spot_v0.14.6_linux_<arch>.deb
sudo dpkg \-i spot_v0.14.6_linux_<arch>.deb
.EE
.PP
\f[B]Install from rpm package (CentOS/RHEL/Fedora/AWS Linux)\f[R]
.IP
.EX
wget https://github.com/umputun/spot/releases/download/v<version>/spot_v<version>_linux_<arch>.rpm
sudo rpm \-i spot_v<version>_linux_<arch>.rpm
.EE
.PP
\f[B]Install from apk package (Alpine)\f[R]
.IP
.EX
wget https://github.com/umputun/spot/releases/download/<versiom>/spot_<version>_linux_<arch>.apk
sudo apk add spot_<version>_linux_<arch>.apk
.EE
.PP
\f[B]Universal installation for Linux and macOS\f[R]
.PP
Spot provides a universal installation script that can be used to
install the latest version of the tool on Linux and macOS.
.IP "1." 3
Download the installation script:
\f[CR]wget https://raw.githubusercontent.com/umputun/spot/master/install.sh\f[R]
.IP "2." 3
Carefully review the script to make sure it is safe.
.IP "3." 3
Run the script: \f[CR]sudo sh install.sh\f[R]
.PP
The script will detect the OS and architecture and download the correct
binary for the latest version of Spot.
.PP
If you are brave enough, you can run the script directly from the web,
but I\[aq]d recommend downloading it first and reviewing it:
.IP
.EX
curl \-sSfL https://raw.githubusercontent.com/umputun/spot/master/install.sh | sudo sh
.EE
.PP
\f[B]Install with go install\f[R]
.PP
This method requires \c
.UR https://golang.org/
Go
.UE \c
\ to be installed on your system.
.IP
.EX
go install github.com/umputun/spot/cmd/spot\[at]latest
go install github.com/umputun/spot/cmd/spot\-secrets\[at]latest
.EE
.SS Options
Spot supports the following command\-line options:
.IP \[bu] 2
\f[CR]\-p\f[R], \f[CR]\-\-playbook=\f[R]: Specifies the playbook file
for use.
Defaults to \f[CR]spot.yml\f[R].
You can also set the environment variable \f[CR]$SPOT_PLAYBOOK\f[R] to
define the playbook file path.
.IP \[bu] 2
\f[CR]\-n\f[R], \f[CR]\-\-task=\f[R]: Specifies task names to execute.
The task should be defined in the playbook file.
Several tasks can be executed by providing the \f[CR]\-\-task\f[R] flag
multiple times, e.g., \f[CR]\-n copy_files \-n warmup_cache\f[R].
If not specified all the tasks will be executed.
.IP \[bu] 2
\f[CR]\-t\f[R], \f[CR]\-\-target=\f[R]: Specifies the target name to use
for the task execution.
The target should be defined in the playbook file and can represent
remote hosts, inventory files, or inventory URLs.
If not specified, the \f[CR]default\f[R] target will be used.
User can pass a hostname, group name, tag or IP instead of the target
name for a quick override.
Providing the \f[CR]\-t\f[R], \f[CR]\-\-target\f[R] flag multiple times
with different targets sets multiple destination targets or multiple
hosts, e.g., \f[CR]\-t prod \-t dev\f[R] or
\f[CR]\-t example1.com \-t example2.com\f[R].
.IP \[bu] 2
\f[CR]\-c\f[R], \f[CR]\-\-concurrent=\f[R]: Sets the number of
concurrent hosts to execute tasks.
Defaults to \f[CR]1\f[R], which means hosts will be handled
sequentially.
.IP \[bu] 2
\f[CR]\-\-timeout\f[R]: Sets the SSH timeout.
Defaults to \f[CR]30s\f[R].
User can also set the environment variable \f[CR]$SPOT_TIMEOUT\f[R] to
define the SSH timeout.
.IP \[bu] 2
\f[CR]\-\-ssh\-agent\f[R]: Enables using the SSH agent for
authentication.
Defaults to \f[CR]false\f[R].
Users can also set the environment variable \f[CR]SPOT_SSH_AGENT\f[R] to
define the value.
.IP \[bu] 2
\f[CR]\-\-shell\f[R] \- shell for remote ssh execution, default is
\f[CR]/bin/sh\f[R].
Users can also set the environment variable \f[CR]SPOT_SHELL\f[R] to
define the value.
.IP \[bu] 2
\f[CR]\-i\f[R], \f[CR]\-\-inventory=\f[R]: Specifies the inventory file
or URL to use for the task execution.
Overrides the inventory file defined in the playbook file.
Users can also set the environment variable \f[CR]$SPOT_INVENTORY\f[R]
to define the default inventory file path or url.
.IP \[bu] 2
\f[CR]\-u\f[R], \f[CR]\-\-user=\f[R]: Specifies the SSH user to use when
connecting to remote hosts.
Overrides the user defined in the playbook file .
.IP \[bu] 2
\f[CR]\-k\f[R], \f[CR]\-\-key=\f[R]: Specifies the SSH key for
connecting to remote hosts.
Overrides the key defined in the playbook file.
.IP \[bu] 2
\f[CR]\-s\f[R], \f[CR]\-\-skip=\f[R]: Skips the specified commands
during the task execution.
Providing the \f[CR]\-s\f[R] flag multiple times with different command
names skips multiple commands.
.IP \[bu] 2
\f[CR]\-o\f[R], \f[CR]\-\-only=\f[R]: Runs only the specified commands
during the task execution.
Providing the \f[CR]\-o\f[R] flag multiple times with different command
names runs only multiple commands.
.IP \[bu] 2
\f[CR]\-e\f[R], \f[CR]\-\-env=\f[R]: Sets the environment variables to
be used during the task execution.
Providing the \f[CR]\-e\f[R] flag multiple times with different
environment variables sets multiple environment variables, e.g.,
\f[CR]\-e VAR1:VALUE1 \-e VAR2:VALUE2\f[R].
.IP \[bu] 2
\f[CR]\-E\f[R], \f[CR]\-\-env\-file=\f[R]: Sets the environment
variables from the file to be used during the task execution.
The default is env.yml.
Can also be set with the environment variable \f[CR]SPOT_ENV_FILE\f[R].
.IP \[bu] 2
\f[CR]\-\-no\-color\f[R]: disable the colorized output.
It can also be set with the environment variable
\f[CR]SPOT_NO_COLOR\f[R].
.IP \[bu] 2
\f[CR]\-\-dry\f[R]: Enables dry\-run mode, which prints out the commands
to be executed without actually executing them.
.IP \[bu] 2
\f[CR]\-v\f[R], \f[CR]\-\-verbose\f[R]: Enables verbose mode, providing
more detailed output and error messages during the task execution.
.IP \[bu] 2
\f[CR]\-\-dbg\f[R]: Enables debug mode, providing even more detailed
output and error messages during the task execution and diagnostic
messages.
.IP \[bu] 2
\f[CR]\-h\f[R] \f[CR]\-\-help\f[R]: Displays the help message, listing
all available command\-line options.
.SS Basic Concepts
.IP \[bu] 2
\f[B]Playbook\f[R] is a YAML or TOML file that defines a list of tasks
to be executed on one or more target hosts.
Each task consists of a series of commands that can be executed on the
target hosts.
Playbooks can be used to automate deployment and configuration
management tasks.
.IP \[bu] 2
\f[B]Task\f[R] is a named set of commands that can be executed on one or
more target hosts.
Tasks can be defined in a playbook and executed concurrently on multiple
hosts.
.IP \[bu] 2
\f[B]Command\f[R] is an action that can be executed on a target host.
Spot supports several built\-in commands, including copy, sync, delete,
script, echo, and wait.
.IP \[bu] 2
\f[B]Target\f[R] is a host or group of hosts on which a task can be
executed.
Targets can be specified directly in a playbook or defined in an
inventory file.
Spot supports several inventory file formats.
.IP \[bu] 2
\f[B]Inventory\f[R] is a list of targets that can be used to define the
hosts and groups of hosts on which a task can be executed.
.SS Playbooks
.SS Full playbook example
.IP
.EX
user: umputun                       # default ssh user. Can be overridden by \-u flag or by inventory or host definition
ssh_key: keys/id_rsa                # ssh key
ssh_shell: /bin/bash                # shell to use for remote ssh execution, default is /bin/sh
local_shell: /bin/bash              # shell to use for local execution, default is os shell
inventory: /etc/spot/inventory.yml  # default inventory file. Can be overridden by \-\-inventory flag

# list of targets, i.e. hosts, inventory files or inventory URLs
targets:
  prod:
    hosts: # list of hosts, user, name and port optional. 
      \- {host: \[dq]h1.example.com\[dq], user: \[dq]user2\[dq], name: \[dq]h1\[dq]}
      \- {host: \[dq]h2.example.com\[dq], port: 2222}
  staging:
    groups: [\[dq]dev\[dq], \[dq]staging\[dq]] # list of groups from inventory file
  dev:
    names: [\[dq]devbox1\[dq], \[dq]devbox2\[dq]] # list of server names from inventory file
  all\-boxes:
    groups: [\[dq]all\[dq]] # all hosts from all groups from inventory file

# list of tasks, i.e. commands to execute
tasks:
  \- name: deploy\-things
    on_error: \[dq]curl \-s localhost:8080/error?msg={SPOT_ERROR}\[dq] # call hook on error
    commands:
      \- name: wait
        script: sleep 5s
      
      \- name: copy configuration
        copy: {\[dq]src\[dq]: \[dq]testdata/conf.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/conf.yml\[dq], \[dq]mkdir\[dq]: true}

      \- name: copy other files
        copy:
          \- {\[dq]src\[dq]: \[dq]testdata/f1.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things/f1.csv\[dq], \[dq]recur\[dq]: true}
          \- {\[dq]src\[dq]: \[dq]testdata/f2.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things/f2.csv\[dq], \[dq]recur\[dq]: true}

      \- name: sync things
        sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}
      
      \- name: some command
        script: |
          ls \-laR /tmp
          du \-hcs /srv
          cat /tmp/conf.yml
          echo all good, 123
      
      \- name: delete things
        delete: {\[dq]path\[dq]: \[dq]/tmp/things\[dq], \[dq]recur\[dq]: true}
      
      \- name: show content
        script: ls \-laR /tmp

  \- name: docker
    commands:
      \- name: docker pull and start
        script: |
          docker pull umputun/remark42:latest
          docker stop remark42 || true
          docker rm remark42 || true
          docker run \-d \-\-name remark42 \-p 8080:8080 umputun/remark42:latest
        env: {FOO: bar, BAR: qux} # set environment variables for the command
      \- wait: {cmd: \[dq]curl \-s localhost:8080/health\[dq], timeout: \[dq]10s\[dq], interval: \[dq]1s\[dq]} # wait for health check to pass
.EE
.PP
\f[I]Alternatively, the playbook can be represented using the TOML
format.\f[R]
.SS Simplified playbook example
In some cases, the rich syntax of the full playbook is not needed and
can feel over\-engineered and even overwhelming.
For those situations, Spot supports a simplified playbook format, which
is easier to read and write, but also more limited in its capabilities.
.IP
.EX
user: umputun                       # default ssh user. Can be overridden by \-u flag or by inventory or host definition
ssh_key: keys/id_rsa                # ssh key
ssh_shell: /bin/bash                # shell to use for remote ssh execution, default is /bin/sh
local_shell: /bin/bash              # shell to use for local execution, default is os shell
inventory: /etc/spot/inventory.yml  # default inventory file. Can be overridden by \-\-inventory flag

targets: [\[dq]devbox1\[dq], \[dq]devbox2\[dq], \[dq]h1.example.com:2222\[dq], \[dq]h2.example.com\[dq]] # list of host names from inventory and direct host ips

# the actual list of commands to execute
task:
  \- name: wait
    script: sleep 5s
  
  \- name: copy configuration
    copy: {\[dq]src\[dq]: \[dq]testdata/conf.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/conf.yml\[dq], \[dq]mkdir\[dq]: true}
  
  \- name: copy other files
    copy: 
      \- {\[dq]src\[dq]: \[dq]testdata/f1.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things/f1.csv\[dq], \[dq]recur\[dq]: true}
      \- {\[dq]src\[dq]: \[dq]testdata/f2.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things/f2.csv\[dq], \[dq]recur\[dq]: true}
  
  \- name: sync things
    sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}
  
  \- name: some command
    script: |
      ls \-laR /tmp
      du \-hcs /srv
      cat /tmp/conf.yml
      echo all good, 123
  
  \- name: delete things
    delete: {\[dq]path\[dq]: \[dq]/tmp/things\[dq], \[dq]recur\[dq]: true}
  
  \- name: show content
    script: ls \-laR /tmp

  \- name: docker pull and start
    script: |
      docker pull umputun/remark42:latest
      docker stop remark42 || true
      docker rm remark42 || true
      docker run \-d \-\-name remark42 \-p 8080:8080 umputun/remark42:latest
    env: {FOO: bar, BAR: qux} # set environment variables for the command
    
  \- wait: {cmd: \[dq]curl \-s localhost:8080/health\[dq], timeout: \[dq]10s\[dq], interval: \[dq]1s\[dq]} # wait for health check to pass
.EE
.PP
\f[B]For more examples see \c
.UR https://github.com/umputun/spot/tree/master/.examples
\&.examples
.UE \c
\ directory.\f[R]
.SS Playbook Types
Spot supports two types of playbooks: full and simplified.
Both can be represented in either YAML or TOML format.
The full playbook is more powerful and flexible but also more verbose
and complex.
The simplified playbook, on the other hand, is easier to read and write
but has more limited capabilities.
.PP
Here are the main differences between the two types of playbooks:
.IP \[bu] 2
The full playbook supports multiple target sets, while the simplified
playbook only supports a single target set.
In other words, the full playbook can execute the same set of commands
on multiple environments, with each environment defined as a separate
target set.
The simplified playbook can execute the same set of commands in just one
environment.
.IP \[bu] 2
The full playbook supports multiple tasks, while the simplified playbook
only supports a single task.
This means that the full playbook can execute multiple sets of commands,
whereas the simplified playbook can only execute one set of commands.
.IP \[bu] 2
The full playbook supports various target types, such as
\f[CR]hosts\f[R], \f[CR]groups\f[R], and \f[CR]names\f[R], while the
simplified playbook only supports a single type, which is a list of
names or host addresses.
See the Targets section for more details.
.IP \[bu] 2
The simplified playbook does not support task\-level
\f[CR]on_error\f[R], \f[CR]user\f[R], and \f[CR]ssh_key\f[R] fields,
while the full playbook does.
See the Task details section for more information.
.IP \[bu] 2
The simplified playbook also has \f[CR]target\f[R] field (in addition to
\f[CR]targets\f[R]) that allows setting a single host/name only.
This is useful when users want to run the playbook on a single host
only.
The full playbook does not have this field.
.PP
Both types of playbooks support the remaining fields and options.
.SS Tasks and Commands
Each task consists of a list of commands that will be executed on the
remote host(s).
The task can also define the following optional fields:
.IP \[bu] 2
\f[CR]on_error\f[R]: specifies the command to execute on the local host
(the one running the \f[CR]spot\f[R] command) in case of an error.
The command can use the \f[CR]{SPOT_ERROR}\f[R] variable to access the
last error message.
Example:
\f[CR]on_error: \[dq]curl \-s localhost:8080/error?msg={SPOT_ERROR}\[dq]\f[R]
.IP \[bu] 2
\f[CR]user\f[R]: specifies the SSH user to use when connecting to remote
hosts.
Overrides the user defined in the top section of the playbook file for
the specified task.
.IP \[bu] 2
\f[CR]targets\f[R] \- list of target names, groups, tags, or host
addresses to execute the task on.
Command line \f[CR]\-t\f[R] flag can be used to override this field.
The \f[CR]targets\f[R] field may include variables.
For more details see Dynamic targets section.
.PP
\f[I]Note: these fields are supported in the full playbook type
only\f[R]
.PP
All tasks are executed sequentially on a given host, one after another.
If a task fails, the execution of the playbook will stop and the
\f[CR]on_error\f[R] command will be executed on the local host, if
defined.
Every task has to have \f[CR]name\f[R] field defined, which is used to
identify the task everywhere.
Playbook with a missing \f[CR]name\f[R] field will fail to execute
immediately.
Duplicate task names are not allowed either.
.SS Relative paths resolution
Relative path resolution is a frequent issue in systems that involve
file references or inclusion.
Different systems handle this in various ways.
Spot uses a widely\-adopted method of resolving relative paths based on
the current working directory of the process.
This means that if you run Spot from different directories, the way
relative paths are resolved will change.
In simpler terms, Spot doesn\[aq]t resolve relative paths according to
the location of the playbook file itself.
.PP
This approach is intentional to prevent confusion and make it easier to
comprehend relative path resolution.
Generally, it\[aq]s a good practice to run Spot from the same directory
where the playbook file is located when using relative paths.
Alternatively, you can use absolute paths for even better results.
.SS Command Types
Spot supports the following command types:
.SS \f[CR]script\f[R]
Can be any valid shell script.
The script will be executed on the remote host(s) using SSH, inside a
shell.
.IP
.EX
script: |
  ls \-laR /tmp
  du \-hcs /srv
  cat /tmp/conf.yml
  echo all good, 123
.EE
.PP
Read more about YAML multiline string formatting on \c
.UR https://yaml-multiline.info/
yaml\-multiline.info
.UE \c
\ and this \c
.UR
https://stackoverflow.com/questions/3790454/how-do-i-break-a-string-in-yaml-over-multiple-lines
stackoverflow post
.UE \c
\&.
.SS \f[CR]copy\f[R]
Copies a file from the local machine to the remote host(s).
If \f[CR]mkdir\f[R] is set to \f[CR]true\f[R] the command will create
the destination directory if it doesn\[aq]t exist, the same as
\f[CR]mkdir \-p\f[R] in bash.
The command also supports glob patterns in \f[CR]src\f[R] field.
.PP
Copy command performs a quick check to see if the file already exists on
the remote host(s) with the same size and modification time, and skips
the copy if it does.
This option can be disabled by setting \f[CR]force: true\f[R] flag.
Another option is \f[CR]exclude\f[R] which allows to specify a list of
files to exclude to be copied.
.IP
.EX
\- name: copy file with mkdir
  copy: {\[dq]src\[dq]: \[dq]testdata/conf.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/conf.yml\[dq], \[dq]mkdir\[dq]: true}

\- name: copy files with glob
  copy: {\[dq]src\[dq]: \[dq]testdata/*.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}

\- name: copy files with glob and exclude
  copy: {\[dq]src\[dq]: \[dq]testdata/*.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq], \[dq]exclude\[dq]: [\[dq]conf.dist.yml\[dq]]}

\- name: copy files with force flag
  copy: {\[dq]src\[dq]: \[dq]testdata/*.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq], \[dq]force\[dq]: true}
.EE
.PP
Copy also supports list format to copy multiple files at once:
.IP
.EX
\- name: copy files with glob
  copy:
    \- {\[dq]src\[dq]: \[dq]testdata/*.csv\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}
    \- {\[dq]src\[dq]: \[dq]testdata/*.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}
.EE
.PP
Copy file and making it executable is also supported:
.IP
.EX
\- name: copy file and make it executable
  copy:
    \- {\[dq]src\[dq]: \[dq]testdata/script.sh\[dq], \[dq]dst\[dq]: \[dq]/tmp/script.sh\[dq], \[dq]chmod+x\[dq]: true}
.EE
.SS \f[CR]sync\f[R]
Synchronises directory from the local machine to the remote host(s).
Optionally supports deleting files on the remote host(s) that don\[aq]t
exist locally with \f[CR]\[dq]delete\[dq]: true\f[R] flag.
Another option is \f[CR]exclude\f[R] which allows to specify a list of
files to exclude from the sync.
.IP
.EX
\- name: sync directory
  sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq]}

\- name: sync directory with delete
  sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq], \[dq]delete\[dq]: true}

\- name: sync directory with exclude
  sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/things\[dq], \[dq]exclude\[dq]: [\[dq]*.txt\[dq], \[dq]*.yml\[dq]]}
  
.EE
.PP
Sync also supports list format to sync multiple paths at once.
.SS \f[CR]delete\f[R]
Deletes a file or directory on the remote host(s), optionally can remove
recursively.
.IP
.EX
\- name: delete file
  delete: {\[dq]path\[dq]: \[dq]/tmp/things.csv\[dq]}
  
\- name: delete directory recursively
  delete: {\[dq]path\[dq]: \[dq]/tmp/things\[dq], \[dq]recur\[dq]: true}
  
\- name: delete directory recursively with exclude
  delete: {\[dq]path\[dq]: \[dq]/tmp/things\[dq], \[dq]recur\[dq]: true, \[dq]exclude\[dq]: [\[dq]*.txt\[dq], \[dq]*.yml\[dq]]}
.EE
.PP
Delete also supports a list format to remove multiple paths at once.
.SS \f[CR]wait\f[R]
Waits for the specified command to finish on the remote host(s) with 0
error code.
This command is useful when a user needs to wait for a service to start
before executing the next command.
Allows to specify the timeout as well as check interval.
.IP
.EX
\- name: wait for service to start
  wait: {\[dq]cmd\[dq]: \[dq]curl \-s \-\-fail localhost:8080\[dq], \[dq]timeout\[dq]: \[dq]30s\[dq], \[dq]interval\[dq]: \[dq]1s\[dq]}
.EE
.SS \f[CR]echo\f[R]
Prints the specified message to the console.
This command is useful for debugging purposes and also to print the
value of variables to the console.
.IP
.EX
\- name: print message
  echo: \[dq]hello world\[dq]
\- name: print variable
  echo: $some_var
.EE
.SS Command options
Each command type supports the following options:
.IP \[bu] 2
\f[CR]ignore_errors\f[R]: if set to \f[CR]true\f[R] the command will not
fail the task in case of an error.
.IP \[bu] 2
\f[CR]no_auto\f[R]: if set to \f[CR]true\f[R] the command will not be
executed automatically, but can be executed manually using the
\f[CR]\-\-only\f[R] flag.
.IP \[bu] 2
\f[CR]local\f[R]: if set to \f[CR]true\f[R] the command will be executed
on the local host (the one running the \f[CR]spot\f[R] command) instead
of the remote host(s).
.IP \[bu] 2
\f[CR]sudo\f[R]: if set to \f[CR]true\f[R] the command will be executed
with \f[CR]sudo\f[R] privileges.
This option is not supported for \f[CR]sync\f[R] command type but can be
used with any other command type.
.IP \[bu] 2
\f[CR]only_on\f[R]: allows to set a list of host names or addresses
where the command will be executed.
For example, \f[CR]only_on: [host1, host2]\f[R] will execute a command
on \f[CR]host1\f[R] and \f[CR]host2\f[R] only.
This option also supports reversed conditions, so if a user wants to
execute a command on all hosts except some, \f[CR]!\f[R] prefix can be
used.
For example, \f[CR]only_on: [!host1, !host2]\f[R] will execute a command
on all hosts except \f[CR]host1\f[R] and \f[CR]host2\f[R].
.PP
example setting \f[CR]ignore_errors\f[R], \f[CR]no_auto\f[R] and
\f[CR]only_on\f[R] options:
.IP
.EX
  commands:
      \- name: wait
        script: sleep 5s
        options: {ignore_errors: true, no_auto: true, only_on: [host1, host2]}
.EE
.SS Command conditionals
\f[CR]cond\f[R]: defines a condition for the command to be executed.
The condition is a valid shell command that will be executed on the
remote host(s) and if it returns 0, the primary command will be
executed.
For example, \f[CR]cond: \[dq]test \-f /tmp/foo\[dq]\f[R] will execute
the primary script command only if the file \f[CR]/tmp/foo\f[R] exists.
The condition can be reversed by adding \f[CR]!\f[R] prefix, i.e.
\f[CR]! test \-f /tmp/foo\f[R] will pass only if the file
\f[CR]/tmp/foo\f[R] doesn\[aq]t exist.
.PP
example installing curl package if not installed already:
.IP
.EX
  \- name: \[dq]install curl\[dq]
    script: \[dq]yum install curl \-y\[dq]
    options: {sudo: true}
    cond: \[dq]! command \-v curl\[dq]
.EE
.SS Deferred actions (\f[CR]on_exit\f[R])
Each command may have \f[CR]on_exit\f[R] parameter defined.
It allows executing a command on the remote host after the task with all
commands is completed.
The command is called regardless of the task\[aq]s exit code.
.PP
This is useful in several scenarios:
.IP \[bu] 2
a temporary script copied to the remote host and executed and should be
removed after execution with
\f[CR]on_exit: \[dq]rm \-fv /tmp/script.sh\[dq]\f[R]
.IP \[bu] 2
a service should be restarted after the new version is deployed with
\f[CR]on_exit: \[dq]systemctl restart myservice\[dq]\f[R]
.IP
.EX
  \- name: \[dq]copy script\[dq]
    copy: {src: \[dq]testdata/script.sh\[dq], \[dq]dst\[dq]: \[dq]/tmp/script.sh\[dq], \[dq]chmod+x\[dq]: true}
    on_exit: \[dq]rm \-fv /tmp/script.sh\[dq] # register deferred action to remove script.sh after execution
  \- name: \[dq]run script\[dq]
    script: \[dq]/tmp/script.sh\[dq]
.EE
.PP
In the example above, the \f[CR]script.sh\f[R] is copied to the remote
host, executed, and removed after completion of the task.
.SS Script Execution
Spot allows executing scripts on remote hosts, or locally if
\f[CR]options.local\f[R] is set to true.
Scripts can be executed in two different ways, depending on whether they
are single\-line or multi\-line scripts.
.PP
\f[B]Single\-line Script Execution\f[R]
.PP
For single\-line scripts, they are executed directly inside the shell
with the optional parameters set to the command line.
For example:
.IP
.EX
  commands:
      \- name: some command
        script: ls \-laR /tmp
        env: {FOO: bar, BAR: qux} 
.EE
.PP
this will be executed as:
\f[CR]FOO=\[aq]bar\[aq] BAR=\[aq]qux\[aq]ls \-laR /tmp FOO=bar BAR=qux\f[R]
inside the shell on the remote host(s), i.e.
\f[CR]sh \-c \[dq]FOO=\[aq]bar\[aq] BAR=\[aq]qux\[aq]ls \-laR /tmp FOO=bar BAR=qux\[dq]\f[R].
.PP
\f[B]Multi\-line Script Execution\f[R]
.PP
For multi\-line scripts, Spot creates a temporary script containing all
the commands, uploads it to the remote host (or keeps it locally if
\f[CR]options.local\f[R] is set to true), and executes the script.
Environment variables are set inside the script, allowing the user to
create complex scripts that include setting variables, conditionals,
loops, and other advanced functionality.
Scripts run with \[dq]set \-e\[dq] to fail on error.
For example:
.IP
.EX
commands:
  \- name: multi_line_script
    script: |
      touch /tmp/file1
      echo \[dq]Hello World\[dq] > /tmp/file2
      echo \[dq]Executing loop...\[dq]
      for i in {1..5}; do
        echo \[dq]Iteration $i\[dq]
      done
      echo \[dq]All done! $FOO $BAR
    env: {FOO: bar, BAR: qux}
.EE
.PP
this will create a temporary script on the remote host(s) with the
following content and execute it:
.IP
.EX
#!/bin/sh
set \-e
export FOO=\[aq]bar\[aq]
export BAR=\[aq]qux\[aq]
touch /tmp/file1
echo \[dq]Hello World\[dq] > /tmp/file2
echo \[dq]Executing loop...\[dq]
for i in {1..5}; do
  echo \[dq]Iteration $i\[dq]
done
echo \[dq]All done! $FOO $BAR\[dq]
.EE
.PP
By using this approach, Spot enables users to write and execute more
complex scripts, providing greater flexibility and power in managing
remote hosts or local environments.
.PP
Users can also set any custom shebang for the script by adding
\f[CR]#!\f[R] at the beginning of the script.
For example:
.IP
.EX
commands:
  \- name: multi_line_script
    script: |
      #!/bin/bash
      touch /tmp/file1
      echo \[dq]Hello World\[dq] > /tmp/file2
.EE
.SS Passing variables from one script command to another
The spot allows passing variables from one command to another.
This feature is especially useful when a command, often a script, sets a
variable, and the subsequent command requires this variable.
For instance, if one command creates a file and the file name is needed
in another command.
To pass these variables, a user must use the conventional shell\[aq]s
export directive in the initial script command.
Subsequently, all variables exported in this initial command will be
accessible in the following commands.
.PP
For example:
.IP
.EX
commands:
  \- name: first command
    script: |
      export FILE_NAME=/tmp/file1
      touch $FILE_NAME
  \- name: second command
    script: |
      echo \[dq]File name is $FILE_NAME\[dq]
  \- name: third command
    copy: {src: $FILE_NAME, dest: /tmp/file2}
.EE
.PP
Sometimes, exporting variables is not possible or not desired.
For such cases, Spot allows to register variables explicitly using the
\f[CR]register\f[R] option.
.PP
For example:
.IP
.EX
commands:
  \- name: first command
    script: |
      FILE_NAME=/tmp/file1
      touch $FILE_NAME
      ANOTHER_VAR=foo
    register: [FILE_NAME, ANOTHER_VAR]
      
  \- name: second command
    script: |
      echo \[dq]File name is $FILE_NAME, var is $ANOTHER_VAR\[dq]
      
  \- name: third command
    copy: {src: $FILE_NAME, dest: /tmp/file2}
.EE
.SS Setting environment variables
Environment variables can be set with \f[CR]\-\-env\f[R] /
\f[CR]\-e\f[R] cli option.
For example: \f[CR]\-e VAR1:VALUE1 \-e VAR2:VALUE2\f[R].
Environment variables can also be set in the environment file (default
\f[CR]env.yml\f[R] can be changed with \f[CR]\-\-env\-file\f[R] /
\f[CR]\-E\f[R] cli flag).
For example:
.IP
.EX
vars:  
  VAR1: VALUE1
  VAR2: VALUE2
.EE
.PP
Environment variables can be used in the playbook file with the expected
syntax: \f[CR]$VAR_NAME\f[R] or \f[CR]${VAR_NAME}\f[R].
For example:
.IP
.EX
commands:
  \- name: some command
    script: echo $VAR1
  \- name: another command
    copy: {\[dq]src\[dq]: \[dq]testdata/*.csv\[dq], \[dq]dst\[dq]: \[dq]$VAR2\[dq]}
.EE
.PP
In case of a conflict between environment variables set in the
environment file and the cli, the cli variables will take precedence.
.SS Targets
Targets are used to define the remote hosts to execute the tasks on.
Targets can be defined in the playbook file or passed as a command\-line
argument.
The following target types are supported:
.IP \[bu] 2
\f[CR]hosts\f[R]: a list of destination host names or IP addresses, with
optional port and username, to execute the tasks on.
Example:
\f[CR]hosts: [{host: \[dq]h1.example.com\[dq], user: \[dq]test\[dq], name: \[dq]h1}, {host: \[dq]h2.example.com\[dq], \[dq]port\[dq]: 2222}]\f[R].
If no user is specified, the user defined in the top section of the
playbook file (or override) will be used.
If no port is specified, port 22 will be used.
.IP \[bu] 2
\f[CR]groups\f[R]: a list of groups from inventory to use.
Example: \f[CR]groups: [\[dq]dev\[dq], \[dq]staging\[dq]}\f[R].
A special group \f[CR]all\f[R] combines all the groups.
.IP \[bu] 2
\f[CR]tags\f[R]: a list of tags from inventory to use.
Example: \f[CR]tags: [\[dq]tag1\[dq], \[dq]tag2\[dq]}\f[R].
.IP \[bu] 2
\f[CR]names\f[R]: a list of host names from inventory to use.
Example: \f[CR]names: [\[dq]host1\[dq], \[dq]host2\[dq]}\f[R].
.PP
All the target types can be combined, i.e.
\f[CR]hosts\f[R], \f[CR]groups\f[R], \f[CR]tags\f[R], \f[CR]hosts\f[R]
and \f[CR]names\f[R] all can be used together in the same target.
To avoid possible duplicates, the final list of hosts is deduplicated by
the host+ip+user.
.PP
example of targets set in the playbook file:
.IP
.EX
targets:
  prod:
    hosts: [{host: \[dq]h1.example.com\[dq], user: \[dq]test\[dq]}, {\[dq]h2.example.com\[dq], \[dq]port\[dq]: 2222, name: \[dq]h2\[dq]}]
  staging:
    groups: [\[dq]staging\[dq]]
  dev:
    groups: [\[dq]dev\[dq], \[dq]staging\[dq]]
    names: [\[dq]host1\[dq], \[dq]host2\[dq]]
  all\-servers:
    groups: [\[dq]all\[dq]]

tasks:
  \- name: task1
    targets: [\[dq]dev\[dq], \[dq]host3.example.com:2222\[dq]]
    commands:
      \- name: command1
        script: echo \[dq]Hello World\[dq]
.EE
.PP
\f[I]Note: All the target types are available in the full playbook file
only.
The simplified playbook file only supports a single, anonymous target
type combining \f[CI]hosts\f[I] and \f[CI]names\f[I] together.\f[R]
.IP
.EX
targets: [\[dq]host1\[dq], \[dq]host2\[dq], \[dq]host3.example.com\[dq], \[dq]host4.example.com:2222\[dq]]
.EE
.PP
in this example, the playbook will be executed on hosts named
\f[CR]host1\f[R] and \f[CR]host2\f[R] from the inventory and on hosts
\f[CR]host3.example.com\f[R] with port \f[CR]22\f[R] and
\f[CR]host4.example.com\f[R] with port \f[CR]2222\f[R].
.SS Target overrides
There are several ways to override or alter the target defined in the
playbook file via command\-line arguments:
.IP \[bu] 2
\f[CR]\-\-inventory\f[R] set hosts from the provided inventory file or
url.
Example: \f[CR]\-\-inventory=inventory.yml\f[R] or
\f[CR]\-\-inventory=http://localhost:8080/inventory\f[R].
.IP \[bu] 2
\f[CR]\-\-target\f[R] set groups, names, tags from inventory or direct
hosts to run the playbook on.
Example: \f[CR]\-\-target=prod\f[R] (will run on all hosts in group
\f[CR]prod\f[R]) or \f[CR]\-\-target=example.com:2222\f[R] (will run on
host \f[CR]example.com\f[R] with port \f[CR]2222\f[R]).
User name can be provided as a part of the direct target address as
well, i.e.
\f[CR]\-\-target=user2\[at]example.com:2222\f[R]
.IP \[bu] 2
\f[CR]\-\-user\f[R] set the ssh user to run the playbook on remote
hosts.
Example: \f[CR]\-\-user=test\f[R].
.IP \[bu] 2
\f[CR]\-\-key\f[R] set the ssh key to run the playbook on remote hosts.
Example: \f[CR]\-\-key=/path/to/key\f[R].
.SS Target selection
The target selection is done in the following order:
.IP \[bu] 2
if \f[CR]\-\-target\f[R] is set, it will be used.
.RS 2
.IP \[bu] 2
first Spot will try to match on target name in the playbook file.
.IP \[bu] 2
if no match is found, Spot will try to match on the group name in the
inventory file.
.IP \[bu] 2
if no match is found, Spot will try to match on tags in the inventory
file.
.IP \[bu] 2
if no match is found, Spot will try to match on hostname in the
inventory file.
.IP \[bu] 2
if no match is found, Spot will try to match on host address in the
playbook file.
.IP \[bu] 2
if no match is found, Spot will use it as a host address.
.RE
.IP \[bu] 2
if \f[CR]\-\-target\f[R] is not set, Spot will try to check it
\f[CR]targets\f[R] list for the task.
If set, it will use it following the same logic as above.
.IP \[bu] 2
and finally, Spot will assume the \f[CR]default\f[R] target.
.SS Dynamic targets
Spot offers support for dynamic targets, allowing the list of targets to
be defined dynamically using variables.
This feature becomes particularly useful when users need to ascertain a
destination address within one task, and subsequently use it in another
task.
Here is an illustrative example:
.IP
.EX
tasks:
  \- name: get host
    targets: [\[dq]default\[dq]]
    script: |
      export thehost=$(curl \-s http://example.com/next\-host)
    options: {local: true}
    
  \- name: run on host
    targets: [\[dq]$thehost\[dq]]
    script: |
      echo \[dq]doing something on $thehost\[dq]
.EE
.PP
In this example, the host address is initially fetched from \c
.UR http://example.com/next-host
.UE \c
\&.
Following this, the task \[dq]run on host\[dq] is executed on the host
that was just identified.
This ability to use dynamic targets proves beneficial in a variety of
scenarios, especially when the list of hosts is not predetermined.
.PP
A practical use case for dynamic targets arises during the provisioning
of a new host, followed by the execution of commands on it.
Since the IP address of the new host isn\[aq]t known beforehand, dynamic
retrieval becomes essential.
.PP
\f[I]The reason the first task specifies
\f[CI]targets: [\[dq]default\[dq]]\f[I] is because Spot requires some
target to execute a task.
In this case, all commands in \[dq]get host\[dq] tasks are local and
won\[aq]t be invoked on a remote host.
The \f[CI]default\f[I] target is utilized by Spot if no alternative
target is specified via the command line.\f[R]
.SS Inventory
The inventory file is a simple yml (or toml) that can represent a list
of hosts or a list of groups with hosts.
In case if both groups and hosts are defined, the hosts will be merged
with groups and will add a new group named \f[CR]hosts\f[R].
.PP
By default, inventory is loaded from the file/url set in
\f[CR]SPOT_INVENTORY\f[R] environment variable.
This is the lowest priority and can be overridden by
\f[CR]inventory\f[R] from the playbook (next priority) and
\f[CR]\-\-inventory\f[R] flag (highest priority) .
This is an example of the inventory file with groups
.IP
.EX
groups:
  dev:
    \- {host: \[dq]h1.example.com\[dq], name: \[dq]h1\[dq], tags:[\[dq]us\-east1\[dq], \[dq]vpc\-1234567\[dq]]}
    \- {host: \[dq]h2.example.com\[dq], port: 2233, name: \[dq]h2\[dq]}
    \- {host: \[dq]h3.example.com\[dq], user: \[dq]user1\[dq]}
    \- {host: \[dq]h4.example.com\[dq], user: \[dq]user2\[dq], name: \[dq]h4\[dq]}
  staging:
    \- {host: \[dq]h5.example.com\[dq], port: 2233, name: \[dq]h5\[dq]}
    \- {host: \[dq]h6.example.com\[dq], user: \[dq]user3\[dq], name: \[dq]h6\[dq]}
.EE
.IP \[bu] 2
host: the hostname or IP address of the remote host.
.IP \[bu] 2
port: the ssh port of the remote host.
Optional, default is 22.
.IP \[bu] 2
user: the ssh user of the remote host.
Optional, default is the user defined in the playbook file or
\f[CR]\-\-user\f[R] flag.
.IP \[bu] 2
name: the name of the remote host.
Optional.
.IP \[bu] 2
tags: the list of tags of the remote host.
Optional.
.PP
In case if port is not defined, the default port 22 will be used.
If the user is not defined, the playbook\[aq]s user will be used.
.PP
This is an example of the inventory file with hosts only (no groups)
.IP
.EX
hosts:
  \- {host: \[dq]hh1.example.com\[dq], name: \[dq]hh1\[dq]}
  \- {host: \[dq]hh2.example.com\[dq], port: 2233, name: \[dq]hh2\[dq], user: \[dq]user1\[dq]}
  \- {host: \[dq]h2.example.com\[dq], port: 2233, name: \[dq]h2\[dq], tags:[\[dq]us\-east1\[dq], \[dq]vpc\-1234567\[dq]]}
  \- {host: \[dq]h3.example.com\[dq], user: \[dq]user1\[dq], name: \[dq]h3\[dq]}
  \- {host: \[dq]h4.example.com\[dq], user: \[dq]user2\[dq], name: \[dq]h4\[dq]}
.EE
.PP
This format is useful when you want to define a list of hosts without
groups.
.PP
In each case, inventory is automatically merged and a special group
\f[CR]all\f[R] will be created that contains all the hosts.
.PP
\f[I]Alternatively, the inventory can be represented using the TOML
format.\f[R]
.SS Export
Spot supports exporting all the destinations from selected/matched
targets to the file or stdout.
This is useful when users want to use the same
hosts/ports/server\-names/etc in other systems.
By default, with \f[CR]\-\-gen\f[R] option, Spot will export to stdout
in json format.
To export to the file, \f[CR]\-\-gen.output=/path/to/file\f[R] option
can be used.
.PP
This exported list of destinations can be consumed by another system,
but practically it will require some conversion from the spot\[aq]s json
to the format that is supported by the system.
This can be addressed by injecting \c
.UR https://stedolan.github.io/jq/
\f[CR]jq\f[R]
.UE \c
\ into the mix but spot also offers a better solution \- templating with
the standard go templates.
To turn this feature on, \f[CR]\-\-gen.template=/path/to/template\f[R]
option can be used.
.PP
Example of the template file, showing all the fields that can be used:
.IP
.EX
{{\- range .}}
\[dq]Name\[dq]: \[dq]{{.Name}}\[dq]
\[dq]Host:Port\[dq]: \[dq]{{.Host}}:{{.Port}}\[dq]
\[dq]User\[dq]: \[dq]{{.User}}\[dq]
\[dq]Tags\[dq]: [{{range .Tags}}\[dq]{{.}}\[dq]{{end}}]
{{\- end \-}}
.EE
.PP
\f[I]for more info see \c
.UR https://pkg.go.dev/text/template
go templates
.UE \c
\f[R]
.SS Runtime variables
Spot supports runtime variables that can be used in the playbook file.
The following variables are supported:
.IP \[bu] 2
\f[CR]{SPOT_REMOTE_HOST}\f[R]: The remote hostname or IP address.
.IP \[bu] 2
\f[CR]{SPOT_REMOTE_NAME}\f[R]: The remote custom name, set in inventory
or playbook as \f[CR]name\f[R].
.IP \[bu] 2
\f[CR]{SPOT_REMOTE_USER}\f[R]: The remote username.
.IP \[bu] 2
\f[CR]{SPOT_COMMAND}\f[R]: The command name.
.IP \[bu] 2
\f[CR]{SPOT_TASK}\f[R]: The task name.
.IP \[bu] 2
\f[CR]{SPOT_ERROR}\f[R]: The error message, if any.
.PP
Variables can be used in the following places: \f[CR]script\f[R],
\f[CR]copy\f[R], \f[CR]sync\f[R], \f[CR]delete\f[R], \f[CR]wait\f[R] and
\f[CR]env\f[R], for example:
.IP
.EX
tasks:
    name: deploy\-things
    commands:
      \- name: copy configuration
        copy: {\[dq]src\[dq]: \[dq]{SPOT_REMOTE_HOST}/conf.yml\[dq], \[dq]dst\[dq]: \[dq]/tmp/conf.yml\[dq], \[dq]mkdir\[dq]: true}
      \- name: sync things
        sync: {\[dq]src\[dq]: \[dq]testdata\[dq], \[dq]dst\[dq]: \[dq]/tmp/{SPOT_TASK}/things\[dq]}
      \- name: some command
        script: |
          ls \-laR /tmp/${SPOT_COMMAND}
        env: { FOO: bar, BAR: \[dq]{SPOT_COMMAND}\-blah\[dq] }
      \- name: delete things
        delete: {\[dq]loc\[dq]: \[dq]/tmp/things/{SPOT_REMOTE_USER}\[dq], \[dq]recur\[dq]: true}
.EE
.SS Ad\-hoc commands
Spot supports ad\-hoc commands that can be executed on the remote hosts.
This is useful when all is needed is to execute a command on the remote
hosts without creating a playbook file.
This command is optionally passed as a first argument, i.e.
\f[CR]spot \[dq]ls \-la /tmp\[dq]\f[R] and usually accompanied by the
\f[CR]\-\-target=<host>\f[R] (\f[CR]\-t <host>\f[R]) flags.
Example:
\f[CR]spot \[dq]ls \-la\[dq] \-t h1.example.com \-t h2.example.com\f[R].
.PP
All other overrides can be used with ad\-hoc commands as well, for
example \f[CR]\-\-user\f[R]and \f[CR]\-\-key\f[R] to specify the user
and sshkey to use when connecting to the remote hosts.
By default, Spot will use the current user and the default ssh key.
Inventory can be passed to such commands as well, for example
\f[CR]\-\-inventory=inventory.yml\f[R].
.PP
Adhoc commands always sets \f[CR]verbose\f[R] to \f[CR]true\f[R]
automatically, so the user can see the output of the command.
.SS Rolling Updates
Spot supports rolling updates, which means that the tasks will be
executed on the hosts one by one, waiting for the previous host to
finish before starting the next one.
This is useful when you need to update a service running on multiple
hosts but want to avoid downtime.
To enable rolling updates, use the \f[CR]\-\-concurrent=N\f[R] flag when
running the \f[CR]spot\f[R] command.
\f[CR]N\f[R] is the number of hosts to execute the tasks concurrently.
Example: \f[CR]spot \-\-concurrent=2\f[R].
In addition, the user can use a built\-in \f[CR]wait\f[R] command to
wait for a service to start before executing the next command.
See the Command Types section for more details.
Practically, the user will have a task with a series of commands, where
the last command will wait for the service to start by running a command
like \f[CR]curl \-s \-\-fail localhost:8080\f[R] and then the task will
be executed on the next host.
.SS Secrets
Spot supports secrets, which are encrypted string values that can be
used in the playbook file.
This feature is useful for storing sensitive information, such as
passwords or API keys.
Secrets are encrypted, and their values are decrypted at runtime.
Spot supports three types of secret providers: built\-in, Hashicorp
Vault, and AWS Secrets Manager.
Other providers can be added by implementing the
\f[CR]SecretsProvider\f[R] interface with a single \f[CR]GetSecrets\f[R]
method.
.PP
Using secrets is simple.
First, users need to define a secret provider in the command line
options or environment variables.
Then, users can add secrets to any command in the playbook file by
setting \f[CR]options.secrets\f[R], as shown in the following example:
.IP
.EX
tasks:
  \- name: access sensitive data
    commands:
      \- name: read api response
        script: |
          curl \-s \-u ${user}:${password} https://api.example.com  
          curl https://api.example.com \-H \[dq]Authorization: Bearer ${token}\[dq]
        options:
          secrets: [user, password, token]
.EE
.PP
In this case secrets for keys \f[CR]user\f[R], \f[CR]password\f[R] and
\f[CR]token\f[R] will be read from the secrets provider, decrypted at
runtime, and passed to the command in the environment.
Please note: if a user runs \f[CR]spot\f[R] with the
\f[CR]\-\-verbose\f[R] or \f[CR]\-\-dbg\f[R] flag, the secrets will be
replaced with \f[CR]****\f[R] in the output.
This is done to prevent secrets from being displayed or logged.
.SS Built\-in Secrets Provider
Spot includes a built\-in secrets provider that can be used to store
secrets in SQLite, MySQL, or Postgresql databases.
The provider can be configured using the following command line options
or environment variables:
.IP \[bu] 2
\f[CR]\-\-secrets.provider=spot\f[R]: selects the built\-in secret\[ga]s
provider.
.IP \[bu] 2
\f[CR]\-\-secrets.conn\f[R] or \f[CR]$SPOT_SECRETS_CONN\f[R]: the
connection string to the database
.RS 2
.IP \[bu] 2
sqlite: \f[CR]file:///path/to/database.db\f[R] or
\f[CR]/path/to/database.sqlite\f[R] or \f[CR]/path/to/database.db\f[R],
default: \f[CR]spot.db\f[R]
.IP \[bu] 2
mysql: \f[CR]user:password\[at]tcp(host:port)/dbname\f[R]
.IP \[bu] 2
postgresql:
\f[CR]postgres://user:password\[at]host:port/database?option1=value1&option2=value2\f[R]
.RE
.IP \[bu] 2
\f[CR]\-\-secrets.key\f[R] or \f[CR]$SPOT_SECRETS_KEY\f[R]: the
encryption key to use for decrypting secrets.
.PP
If \f[CR]spot\f[R] provider is selected, the table
\f[CR]spot_secrets\f[R] will be created in the database.
The table has the following columns: \f[CR]skey\f[R] and
\f[CR]sval\f[R].
The \f[CR]skey\f[R] column is the secret key, and the \f[CR]sval\f[R]
column is the encrypted secret value.
The \f[CR]skey\f[R] column is indexed for faster lookups.
It is recommended to use application\-specific prefixes for the secret
keys, for example, \f[CR]system\-name/service\-name/secret\-key\f[R].
This will allow to use the same database for multiple applications
without conflicts.
.PP
The built\-in secrets provider uses strong cryptography techniques to
ensure the safety of your secrets.
Below is a summary of the security methods employed:
.IP \[bu] 2
\f[B]Argon2 key derivation\f[R]: The Argon2 key derivation function
(argon2.IDKey) is used to derive a 32\-byte key from the provided user
key and a randomly generated salt.
This function is memory\-hard and designed to be resistant to GPU\-based
attacks, providing increased security for your secrets.
.IP \[bu] 2
\f[B]NaCl SecretBox encryption\f[R]: Secrets are encrypted and decrypted
using the \c
.UR https://pkg.go.dev/golang.org/x/crypto/nacl/secretbox
NaCl SecretBox
.UE \c
\ package, which provides authenticated encryption with additional data.
It uses XSalsa20 for encryption and Poly1305 for authentication,
ensuring the integrity and confidentiality of the stored secrets.
.IP \[bu] 2
\f[B]Random nonces and salts\f[R]: Spot generates random nonces for each
encryption operation and random salts for each key derivation operation.
These values are produced using the crypto/rand package, which generates
cryptographically secure random numbers.
.IP \[bu] 2
\f[B]Base64 encoding\f[R]: Encrypted secret values are stored in the
database as Base64 encoded strings, which provides a safe and compact
way to represent binary data in text form.
.PP
These methods work together to provide a robust and secure way to manage
secrets in Spot.
By using the built\-in secrets provider, users can be confident that
their sensitive data is securely stored and protected from unauthorized
access.
.SS Hashicorp Vault Secrets Provider
Spot supports Hashicorp Vault as a secrets provider.
To use it, a user needs to set the following command line options or
environment variables:
.IP \[bu] 2
\f[CR]\-\-secrets.provider=vault\f[R]: selects the Hashicorp Vault
secrets provider.
.IP \[bu] 2
\f[CR]\-\-secrets.vault.token\f[R] or
\f[CR]$SPOT_SECRETS_VAULT_TOKEN\f[R]: the Vault token to use for
authentication.
.IP \[bu] 2
\f[CR]\-\-secrets.vault.url\f[R] or \f[CR]$SPOT_SECRETS_VAULT_URL\f[R]:
the Vault server url.
.IP \[bu] 2
\f[CR]\-\-secrets.vault.path\f[R] or
\f[CR]$SPOT_SECRETS_VAULT_PATH\f[R]: the path to the secrets in Vault.
.SS AWS Secrets Manager Secrets Provider
Spot supports AWS Secrets Manager as a secrets provider.
To use it, a user needs to set the following command line options or
environment variables:
.IP \[bu] 2
\f[CR]\-\-secrets.provider=aws\f[R]: selects the AWS Secrets Manager
secrets provider.
.IP \[bu] 2
\f[CR]\-\-secrets.aws.region\f[R] or
\f[CR]$SPOT_SECRETS_AWS_REGION\f[R]: the AWS region to use for
authentication.
.IP \[bu] 2
\f[CR]\-\-secrets.aws.access\-key\f[R] or
\f[CR]$SPOT_SECRETS_AWS_ACCESS_KEY\f[R]: the AWS access key to use for
authentication.
.IP \[bu] 2
\f[CR]\-\-secrets.aws.secret\-key\f[R] or
\f[CR]$SPOT_SECRETS_AWS_SECRET_KEY\f[R]: the AWS secret key to use for
authentication.
.PP
note: by default, the AWS Secrets Manager secrets provider will use the
default AWS credential.
This means that the provider will use the credentials from the
environment variables \f[CR]AWS_ACCESS_KEY_ID\f[R] and
\f[CR]AWS_SECRET_ACCESS_KEY\f[R].
.SS Ansible Vault Secrets Provider
Spot gives the ability to use full encrypted \f[CR]YAML\f[R] files by \c
.UR https://docs.ansible.com/ansible/latest/cli/ansible-vault.html
ansbile\-vault
.UE \c
.PP
\f[CR]\-\-secrets.provider=ansible\-vault\f[R]: selects the Ansible
Vault secrets provider.
\f[CR]\-\-secrets.ansible.path\f[R] or
\f[CR]$SPOT_SECRETS_ANSIBLE_PATH\f[R] path to the ansible\-vault file
\f[CR]\-\-secrets.ansible.secret\f[R] or
\f[CR]$SPOT_SECRETS_ANSIBLE_SECRET\f[R] secret string for decrypting
ansible\-vault file
.PP
note: encrypted values in the vault should be in the following format
\f[CR]key[string]:value[string]\f[R] without nested \f[CR]lists\f[R] and
\f[CR]maps\f[R].
.SS Managing Secrets with \f[CR]spot\-secrets\f[R]
Spot provides a simple way to manage secrets for builtin providers using
the \f[CR]spot\-secrets\f[R] utility.
This command can be used to set, delete, get, and list secrets in the
database.
.IP \[bu] 2
\f[CR]spot\-secrets set <key> <value>\f[R]: sets the secret value for
the specified key.
.IP \[bu] 2
\f[CR]spot\-secrets get <key>\f[R]: gets the secret value for the
specified key.
.IP \[bu] 2
\f[CR]spot\-secrets delete <key>\f[R]: deletes the secret value for the
specified key.
.IP \[bu] 2
\f[CR]spot\-secrets list\f[R]: lists all the secret keys in the
database.
.IP
.EX
Usage:
  spot\-secrets [OPTIONS] <command>

Application Options:
  \-k, \-\-key=  key to use for encryption/decryption [$SPOT_SECRETS_KEY]
  \-c, \-\-conn= connection string to use for the secrets database (default: spot.db) [$SPOT_SECRETS_CONN]
      \-\-dbg   debug mode

Help Options:
  \-h, \-\-help  Show this help message

Available commands:
  del   delete a secret
  get   retrieve a secret
  list  list secrets keys
  set   add a new secret
.EE
.SS Why Spot?
Spot is simple.
It only has a few basic commands with a very limited set of options and
flags.
The playbook is just a list of commands to run, plus a list of remote
targets to apply those commands against.
Each command is made to be as intuitive and as direct as possible.
Despite its simplicity, Spot is surprisingly powerful and can help get
things done.
This tool was built out of frustration with the complexity of similar
tools.
All I wanted was something that is simple, easy to use, easy to
understand, and capable of handling most of the usual deployment tasks.
I didn\[aq]t want to have to check the documentation or resort to
googling every time I used it.
Spot is the result of that effort.
.PP
Spot is designed to provide a simple, efficient, and flexible solution
for deployment and configuration management.
It addresses the need for a tool that is easy to set up and use, while
still offering powerful features for managing infrastructure.
.PP
Below are some of the reasons why you should consider using Spot:
.IP " 1." 4
\f[B]Keeps it simple\f[R]: Spot concentrates on one task and one task
only \- deploying things with minimal headache.
It doesn\[aq]t try to solve all the problems in the universe; instead,
it offers a focused and sufficient set of features to address the
majority of use cases without unnecessary complexity.
.IP " 2." 4
\f[B]Conceptual simplicity and predictability\f[R]: Spot embraces
simplicity in its design and execution.
Rather than being declarative, tasks contain a direct list of
straightforward commands to achieve the desired outcome.
This approach ensures that Spot is highly predictable, as it strictly
follows the user\[aq]s instructions without attempting to interpret or
guess their intentions.
This makes it easier for users to understand and control the deployment
process.
.IP " 3." 4
\f[B]User\-friendly\f[R]: Spot prioritizes user\-friendliness by
providing a limited and intuitive set of command line options, making it
easy to get started with deploying projects.
Additionally, Spot uses well\-known YAML or TOML formats for its
playbook and inventory files.
The minimalistic structure of these files enhances readability and makes
them more approachable for users who want to focus on deploying their
projects without getting bogged down in complex syntax or unnecessary
details.
For simpler use cases, Spot also offers a simplified playbook format
that further streamlines the deployment process.
.IP " 4." 4
\f[B]Full control\f[R]: Spot gives users full control over their
deployments.
Users can select any set of tasks and hosts, and even limit which
commands are executed.
Spot provides a dry mode that allows users to preview the changes that
will be made before executing the playbook.
The verbose mode provides many details to help users understand
what\[aq]s going on during the deployment process, while the debug mode
gives maximum detailed logs for users who need to investigate deeper.
.IP " 5." 4
\f[B]Safe and secure\f[R]: Spot prioritizes security, offering seamless
integration with various secret vault solutions, as well as providing a
built\-in option.
This ensures that sensitive information is handled securely, giving
users peace of mind while managing their infrastructure.
.IP " 6." 4
\f[B]Flexible and extensible\f[R]: Spot is designed to adapt to various
deployment and configuration scenarios, managing different targets like
production, staging, and development environments.
It supports executing tasks on remote hosts directly or through
inventory files and URLs, integrating with existing inventory management
solutions.
Spot also allows for custom script execution on remote hosts and offers
built\-in commands for common operations, enabling the creation of
tailored workflows for deployment and configuration management.
.IP " 7." 4
\f[B]Concurrent Execution and Rolling Updates\f[R]: Spot supports
concurrent execution of tasks, speeding up deployment and configuration
processes by running on multiple hosts simultaneously.
This is especially helpful when managing large\-scale infrastructure or
when time is of the essence.
Spot also allows for rolling updates with user\-defined wait commands,
ensuring smooth and controlled deployment of changes across the
infrastructure.
.IP " 8." 4
\f[B]Customizable\f[R]: Spot offers various command\-line options and
environment variables that allow users to tailor its behavior to their
specific requirements.
Users can easily modify the playbook file, task, target, and other
parameters, as well as control the execution flow by skipping or running
specific commands.
.IP " 9." 4
\f[B]Lightweight\f[R]: Spot is a lightweight tool, written in Go, that
does not require heavy dependencies or a complex setup process.
It can be easily installed and run on various platforms, making it an
ideal choice for teams looking for a low\-overhead solution for
deployment and configuration management.
.IP "10." 4
\f[B]Ready\-to\-use binaries and packages\f[R]: Spot is available as
ready\-to\-use binaries and packages for various platforms, including
Linux, macOS, and Windows.
Users can download and install the appropriate package for their
platform, making it easy to get started with Spot without having to
build from source.
Spot provides binaries for both x86, arm, and arm64 architectures, as
well as rpm, deb and apk packages for Linux users.
.PP
In conclusion, Spot is a powerful and easy\-to\-use tool that simplifies
the process of deployment and configuration management while offering
the flexibility and extensibility needed to cater to various use cases.
.SS Is it replacing Ansible?
Spot is not designed as a direct replacement for Ansible; however, in
certain use cases, it can address the same challenges effectively.
While both tools can be used for deployment and configuration
management, there are some key differences between them:
.IP \[bu] 2
\f[B]Complexity\f[R]: Ansible is a more feature\-rich and mature tool,
offering a wide range of modules and plugins that can automate many
different aspects of infrastructure management.
Spot, on the other hand, is designed to be simple and lightweight,
focusing on a few core features to streamline the deployment and
configuration process.
.IP \[bu] 2
\f[B]Learning Curve\f[R]: Due to its simplicity, Spot has a lower
learning curve compared to Ansible.
It\[aq]s easier to start with Spot, making it more suitable for smaller
projects or teams with limited experience in infrastructure automation.
Ansible, while more powerful, can be more complex to learn and
configure, especially for newcomers.
.IP \[bu] 2
\f[B]Customization\f[R]: While both tools offer customization options,
Ansible has a more extensive set of built\-in modules and plugins that
can handle a wide range of tasks out of the box.
Spot, in contrast, relies on custom scripts and a limited set of
built\-in commands for its functionality, which might require more
manual configuration and scripting for certain use cases.
.IP \[bu] 2
\f[B]Community and Ecosystem\f[R]: Ansible has a large and active
community, as well as a vast ecosystem of roles, modules, and
integrations.
This can be beneficial when dealing with common tasks or integrating
with third\-party systems.
Spot, being a smaller and simpler tool, doesn\[aq]t have the same level
of community support or ecosystem.
.IP \[bu] 2
\f[B]Ease of installation and external dependencies\f[R]: One of the
most significant benefits of Spot is that it has no dependencies.
Being written in Go, it is compiled into a single binary that can be
easily distributed and executed on various platforms.
This eliminates needing to install or manage any additional software,
libraries, or dependencies to use Spot.
Ansible, on the other hand, is written in Python and requires Python to
be installed on both the control host (where Ansible is run) and the
managed nodes (remote hosts being managed).
Additionally, Ansible depends on several Python libraries, which must be
installed and maintained on the control host.
Some Ansible modules may also require specific libraries or packages to
be installed on the managed nodes, adding to the complexity of managing
dependencies.
.PP
Spot is an appealing choice for those seeking a lightweight, simple, and
easy\-to\-use tool for deployment and configuration management,
especially for smaller projects or when extensive features aren\[aq]t
necessary.
Its single binary distribution, easy\-to\-comprehend structure, and
minimal dependencies offer a low\-maintenance solution.
However, if a more comprehensive tool with a wide range of built\-in
modules, plugins, and integrations is needed, Ansible may be a better
fit.
While Ansible has advanced features and a robust ecosystem, its reliance
on Python and additional libraries can sometimes be less convenient in
certain environments or situations with specific constraints.
.SS Getting the latest development version
If you want to try the latest development version, you can install it
directly from the master branch.
There are two ways to do this:
.IP \[bu] 2
\f[B]Using go get\f[R]:
\f[CR]go install github.com/umputun/spot/cmd/spot\[at]master\f[R] and
\f[CR]go install github.com/umputun/spot/cmd/secrets\[at]master\f[R].
Note that this will install the latest development version of spot and
secrets, which may not be stable or fully tested.
.IP \[bu] 2
\f[B]Using git\f[R]: \f[CR]git clone github.com/umputun/spot\f[R] then
\f[CR]cd spot\f[R] and \f[CR]make build\f[R].
This will install the latest development version of spot and secrets to
\f[CR]spot/.bin/spot\f[R] and \f[CR]spot/.bin/sport\-secrets\f[R],
respectively.
.PP
\f[B]pls note that you need to have go 1.16+ installed on your
machine.\f[R]
.SS Status
The project is currently in active development, and breaking changes may
occur until the release of version 1.0.
However, we strive to minimize disruptions and will only introduce
breaking changes when there is a compelling reason to do so.
.PP
\f[I]Update: Version 1 has been released and is now considered stable.
We do not anticipate any breaking changes for this version.\f[R]
.SS Contributing
Please feel free to open a discussion, submit issues, fork the
repository, and send pull requests.
See \c
.UR https://github.com/umputun/spot/blob/master/CONTRIBUTING.md
CONTRIBUTING.md
.UE \c
\ for more information.
.SS License
This project is licensed under the MIT License.
See the LICENSE file for more information.