(maint) Update development documentation

This commit adds documentation for how to setup a development environment for VM Pooler. This commit also moves the API under the new docs directory and modifies the README appropriately.
puppetlabs · Jun 16, 2017 · db27049 · db27049
1 parent ee3f5e2
commit db27049
Show file tree

Hide file tree

Showing 4 changed files with 237 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -74,7 +74,7 @@ vmpooler provides an API and web front-end (dashboard) on port `:4567`.  See the
 
 ### API
 
-vmpooler provides a REST API for VM management.  See the [API documentation](API.md) for more information.
+vmpooler provides a REST API for VM management.  See the [API documentation](docs/API.md) for more information.
 
 ### Dashboard
 
@@ -93,6 +93,10 @@ A dashboard is provided to offer real-time statistics and historical graphs.  It
 
 - [vagrant-vmpooler](https://github.com/briancain/vagrant-vmpooler) Use Vagrant to create and manage your vmpooler instances.
 
+## Development and further documentation
+
+For more information about setting up a development instance of VM Pooler or other subjects, see the [docs/](docs) directory.
+
 ## Build status
 
 [![Build Status](https://travis-ci.org/puppetlabs/vmpooler.png?branch=master)](https://travis-ci.org/puppetlabs/vmpooler)

diff --git a/API.md → docs/API.md b/API.md → docs/API.md
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,5 @@
+# Documentation for VM Pooler
+
+
+* [Setting up a Development Environment](dev-setup.md)
+* [API Documentation](API.md)
diff --git a/docs/dev-setup.md b/docs/dev-setup.md
@@ -0,0 +1,227 @@
+# Setting up a VM Pooler development environment
+
+## Requirements
+
+* Supported on OSX, Windows and Linux
+
+* Ruby or JRuby
+
+  Note - Ruby 1.x support will be removed so it is best to use more modern ruby versions
+
+  Note - It is recommended to user Bundler instead of installing gems into the system repository
+
+* A local Redis server
+
+  Either a containerized instance of Redis or a local version is fine.
+
+## Setup source and ruby
+
+* Clone repository, either from your own fork or the original source
+
+* Perform a bundle install
+
+```
+~/ > git clone https://github.com/puppetlabs/vmpooler.git
+Cloning into 'vmpooler'...
+remote: Counting objects: 3411, done.
+...
+
+~/ > cd vmpooler
+
+~/vmpooler/ > bundle install
+Fetching gem metadata from https://rubygems.org/.........
+Fetching version metadata from https://rubygems.org/..
+Resolving dependencies...
+Installing rake 12.0.0
+...
+Bundle complete! 16 Gemfile dependencies, 37 gems now installed.
+```
+
+## Setup environment variables
+
+### `VMPOOLER_DEBUG`
+
+Setting the `VMPOOLER_DEBUG` environment variable will instruct VM Pooler to:
+
+* Output log messages to STDOUT
+
+* Allow the use of the dummy authentication method
+
+* Add interrupt traps so you can stop VM Pooler when run interactively
+
+Linux, OSX
+```bash
+~/vmpooler/ > export VMPOOLER_DEBUG=true
+```
+
+Windows (PowerShell)
+```powershell
+C:\vmpooler > $ENV:VMPOOLER_DEBUG = 'true'
+```
+
+
+### `VMPOOLER_CONFIG`
+
+When `VMPOOLER_CONFIG` is set, VM Pooler will read its configuration from the content of the environment variable instead of from the `vmpooler.yaml` configuration file.
+
+Note that this variable does not point a different configuration file, but stores the contents of a configuration file.
+
+
+## Setup VM Pooler Configuration
+
+You can either create a `vmpooler.yaml` file or set the `VMPOOLER_CONFIG` environment variable with the equivalent content.
+
+Example minimal configuration file:
+```yaml
+---
+:providers:
+  :dummy:
+
+:redis:
+  server: 'localhost'
+
+:auth:
+   provider: dummy
+
+:tagfilter:
+  url: '(.*)\/'
+
+:config:
+  site_name: 'vmpooler'
+  # Need to change this on Windows
+  logfile: '/var/log/vmpooler.log'
+  task_limit: 10
+  timeout: 15
+  vm_checktime: 15
+  vm_lifetime: 12
+  vm_lifetime_auth: 24
+  allowed_tags:
+    - 'created_by'
+    - 'project'
+  domain: 'company.com'
+  prefix: 'poolvm-'
+
+# Uncomment the lines below to suppress metrics to STDOUT
+# :statsd:
+#   server: 'localhost'
+#   prefix: 'vmpooler'
+#   port: 8125
+
+:pools:
+  - name: 'pool01'
+    size: 5
+    provider: dummy
+  - name: 'pool02'
+    size: 5
+    provider: dummy
+```
+
+## Running VM Pooler locally
+
+* Run `bundle exec ruby vmpooler`
+
+  If using JRuby, you may need to use `bundle exec jruby vmpooler`
+
+You should see output similar to:
+```
+~/vmpooler/ > bundle exec ruby vmpooler
+[2017-06-16 14:50:31] starting vmpooler
+[2017-06-16 14:50:31] [!] Creating provider 'dummy'
+[2017-06-16 14:50:31] [dummy] ConnPool - Creating a connection pool of size 1 with timeout 10
+[2017-06-16 14:50:31] [*] [disk_manager] starting worker thread
+[2017-06-16 14:50:31] [*] [snapshot_manager] starting worker thread
+[2017-06-16 14:50:31] [*] [pool01] starting worker thread
+[2017-06-16 14:50:31] [*] [pool02] starting worker thread
+[2017-06-16 14:50:31] [dummy] ConnPool - Creating a connection object ID 1784
+== Sinatra (v1.4.8) has taken the stage on 4567 for production with backup from Puma
+*** SIGUSR2 not implemented, signal based restart unavailable!
+*** SIGUSR1 not implemented, signal based restart unavailable!
+*** SIGHUP not implemented, signal based logs reopening unavailable!
+Puma starting in single mode...
+* Version 3.9.1 (ruby 2.3.1-p112), codename: Private Caller
+* Min threads: 0, max threads: 16
+* Environment: development
+* Listening on tcp://0.0.0.0:4567
+Use Ctrl-C to stop
+[2017-06-16 14:50:31] [!] [pool02] is empty
+[2017-06-16 14:50:31] [!] [pool01] is empty
+[2017-06-16 14:50:31] [ ] [pool02] Starting to clone 'poolvm-nexs1w50m4djap5'
+[2017-06-16 14:50:31] [ ] [pool01] Starting to clone 'poolvm-r543eibo4b6tjer'
+[2017-06-16 14:50:31] [ ] [pool01] Starting to clone 'poolvm-neqmu7wj7aukyjy'
+[2017-06-16 14:50:31] [ ] [pool02] Starting to clone 'poolvm-nsdnrhhy22lnemo'
+[2017-06-16 14:50:31] [ ] [pool01] 'poolvm-r543eibo4b6tjer' is being cloned from ''
+[2017-06-16 14:50:31] [ ] [pool01] 'poolvm-neqmu7wj7aukyjy' is being cloned from ''
+[2017-06-16 14:50:31] [ ] [pool02] 'poolvm-nexs1w50m4djap5' is being cloned from ''
+[2017-06-16 14:50:31] [ ] [pool01] Starting to clone 'poolvm-edzlp954lyiozli'
+[2017-06-16 14:50:31] [ ] [pool01] Starting to clone 'poolvm-nb0uci0yrwbxr6x'
+[2017-06-16 14:50:31] [ ] [pool02] Starting to clone 'poolvm-y2yxgnovaneymvy'
+[2017-06-16 14:50:31] [ ] [pool01] Starting to clone 'poolvm-nur59d25s1y8jko'
+...
+```
+
+### Common Errors
+
+* Forget to set VMPOOLER_DEBUG environment variable
+
+VM Pooler will fail to start with an error similar to below
+```
+~/vmpooler/ > bundle exec ruby vmpooler
+
+~/vmpooler/lib/vmpooler.rb:44:in `config': Dummy authentication should not be used outside of debug mode; please set environment variable VMPOOLER_DEBUG to 'true' if you want to use dummy authentication (RuntimeError)
+        from vmpooler:8:in `<main>'
+...
+```
+
+* Error in VM Pooler configuration
+
+If there is an error in the VM Pooler configuration file, or any other fatal error in the Pool Manager, VM Pooler will appear to be running but no log information is displayed.  This is due to the error not being displayed until you press `Ctrl-C` and then suddenly you can see the cause of the issue.
+
+For example, when running VM Pooler on Windows, but with a unix style filename for the VM Pooler log
+
+```powershell
+C:\vmpooler > bundle exec ruby vmpooler
+[2017-06-16 14:49:57] starting vmpooler
+== Sinatra (v1.4.8) has taken the stage on 4567 for production with backup from Puma
+*** SIGUSR2 not implemented, signal based restart unavailable!
+*** SIGUSR1 not implemented, signal based restart unavailable!
+*** SIGHUP not implemented, signal based logs reopening unavailable!
+Puma starting in single mode...
+* Version 3.9.1 (ruby 2.3.1-p112), codename: Private Caller
+* Min threads: 0, max threads: 16
+* Environment: development
+* Listening on tcp://0.0.0.0:4567
+Use Ctrl-C to stop
+
+# [ NOTHING ELSE IS LOGGED ]
+```
+
+Once `Ctrl-C` is pressed the error is shown
+
+```powershell
+...
+== Sinatra has ended his set (crowd applauds)
+Shutting down.
+C:/tools/ruby2.3.1x64/lib/ruby/2.3.0/open-uri.rb:37:in `initialize': No such file or directory @ rb_sysopen - /var/log/vmpooler.log (Errno::ENOENT)
+        from C:/tools/ruby2.3.1x64/lib/ruby/2.3.0/open-uri.rb:37:in `open'
+        from C:/tools/ruby2.3.1x64/lib/ruby/2.3.0/open-uri.rb:37:in `open'
+        from C:/vmpooler/lib/vmpooler/logger.rb:17:in `log'
+        from C:/vmpooler/lib/vmpooler/pool_manager.rb:709:in `execute!'
+        from vmpooler:26:in `block in <main>'
+```
+
+## Default VM Pooler URLs
+
+| Endpoint  | URL                                                                  |
+|-----------|----------------------------------------------------------------------|
+| Dashboard | [http://localhost:4567/dashboard/](http://localhost:4567/dashboard/) |
+| API       | [http://localhost:4567/api/v1]([http://localhost:4567/api/v1)        |
+
+## Use the VM Pooler API locally
+
+Once a local VM Pooler instance is running you can use any tool you need to interact with the API.  The dummy authentication provider will allow a user to connect if the username and password are not the same:
+
+* Authentication is successful for username `Alice` with password `foo`
+
+* Authentication will fail for username `Alice` with password `Alice`
+
+Like normal VM Pooler, tokens will be created for the user and can be used for regular VM Pooler operations.