Skip to content
This repository has been archived by the owner on Mar 23, 2023. It is now read-only.

Adds support for docker-machine #77

Merged
merged 9 commits into from
Aug 10, 2015
Merged

Adds support for docker-machine #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
dc0c179
Adds support for docker-machine
labianchin Jul 26, 2015
f8f144c
Add docker-machine support to readme.
labianchin Jul 28, 2015
d0452bd
Fixes from the feedback of brikis98
labianchin Jul 31, 2015
d5ccbed
Adds tests to check proper DOCKER_HOST variables configuration
labianchin Aug 5, 2015
bdaac15
Remove port configuration on rsync, try to mimic docker-machine scp
labianchin Aug 5, 2015
695af75
Remove python dependecy python to get json values from docker-machine…
labianchin Aug 6, 2015
e63d07b
Fixes in docs and typos, add more tests to get_json_value
labianchin Aug 7, 2015
882600f
Adds more tests for get_json_value and clarify docs
labianchin Aug 7, 2015
0b072f9
Adds back install_rsync_on_docker_host to watch_and_sync
labianchin Aug 8, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,28 @@ http://dockerhost:5000
When you install docker-osx-dev, it adds an entry to your `/etc/hosts` file so
that `http://dockerhost` works as a URL for testing your Docker containers.

# docker-machine support

`docker-machine` support is experimental. You can use it as the way it is used for
`boot2docker`, but run `docker-machine env` before. So as an example, run as:

```
> docker-machine create --driver virtualbox <machine-name>
> eval "$(docker-machine env <machine-name>)"
> docker-osx-dev install
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, just realized I have one more question on this: do you have to run the install command every time you run docker-machine create? Or only the eval part?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I am little confused on what should be the role of the install for docker-machine.

   install_dependencies # Should actually run before docker-machine create. Once for osx user.
   init_docker_host # Per machine, should run after docker-machine create. This seems only important for the next command
   install_rsync_on_docker_host # Is this really important? the sync will also install rsync.
   add_docker_host # Per machine, should run after docker-machine create. Also, is it important?
   add_environment_variables # Not used for docker-machine

I would avoid making the install create the docker-machine, as in the creation the user needs to supply the driver he wants and all the driver's configuration. It is also seems misleading that a install command would create a machine.

I'd suggest the following flow:

docker-osx-dev install_dependencies # once per osx user
docker-machine create --driver virtualbox <machine-name>
eval "$(docker-machine env <machine-name>)"
cd /foo/bar
docker-osx-dev

In this way there is no need to run install after docker-machine create. The install would only make sense for boot2docker.

> cd /foo/bar
> docker-osx-dev
[INFO] Performing initial sync of paths: /foo/bar
[INFO] Watching: /foo/bar
```

In this case, `docker-osx-dev` will use the machine defined in the `DOCKER_MACHINE_NAME` env var,
defined by `docker-machine env`. Alternatively, use the `--machine-name <machine-name>` argument.

Note: when running `docker-osx-dev` for `boot2docker`, please make sure the env var `DOCKER_MACHINE_NAME`
is not defined.


# How it works

The `install command` installs all the software you need:
Expand Down
142 changes: 109 additions & 33 deletions src/docker-osx-dev
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,9 @@ readonly INSTALL_COMMAND="install"
readonly TEST_COMMAND="test_mode"
readonly DEFAULT_COMMAND="$SYNC_COMMAND"

# Boot2Docker constants
test -n "$DOCKER_HOST_NAME" || readonly DOCKER_HOST_NAME="dockerhost"
# docker host constants

readonly BOOT2DOCKER_USER="docker"
readonly BOOT2DOCKER_SSH_URL="$BOOT2DOCKER_USER@$DOCKER_HOST_NAME"

# docker-compose constants
readonly DEFAULT_COMPOSE_FILE="docker-compose.yml"
Expand All @@ -55,7 +54,10 @@ PATHS_TO_SYNC=""
EXCLUDES=""
INCLUDES=""
CURRENT_LOG_LEVEL="$DEFAULT_LOG_LEVEL"
BOOT2DOCKER_SSH_KEY=""
DOCKER_HOST_USER=""
DOCKER_HOST_SSH_URL=""
DOCKER_HOST_SSH_KEY=""
DOCKER_HOST_SSH_COMMAND=""


################################################################################
Expand Down Expand Up @@ -279,8 +281,12 @@ function configure_log_level {
#
# Configures the Boot2Docker SSH key by looking into the Boot2Docker config
#
function configure_boot2docker_ssh_key {
BOOT2DOCKER_SSH_KEY=$(boot2docker cfg | grep "^SSHKey = " | sed -e 's/^SSHKey = "\(.*\)"/\1/')
function configure_boot2docker {
test -n "$DOCKER_HOST_NAME" || DOCKER_HOST_NAME="dockerhost"
DOCKER_HOST_SSH_KEY=$(boot2docker cfg | grep "^SSHKey = " | sed -e 's/^SSHKey = "\(.*\)"/\1/')
DOCKER_HOST_USER="$BOOT2DOCKER_USER"
DOCKER_HOST_SSH_URL="$BOOT2DOCKER_USER@$DOCKER_HOST_NAME"
DOCKER_HOST_SSH_COMMAND="boot2docker ssh"
}

#
Expand All @@ -297,7 +303,7 @@ function find_boot2docker_vm_name {
# Returns mounted volumes with vboxsf-type in boot2docker instance.
#
function find_vboxsf_mounted_folders {
boot2docker ssh mount | grep 'type vboxsf' | awk '{print $3}'
$DOCKER_HOST_SSH_COMMAND mount | grep 'type vboxsf' | awk '{print $3}'
}

#
Expand All @@ -312,7 +318,7 @@ function umount_vboxsf_mounted_folder {
local vbox_shared_folder=''
while read -r vbox_shared_folder; do
log_info "Removing shared folder: $vbox_shared_folder"
boot2docker ssh sudo umount "$vbox_shared_folder"
$DOCKER_HOST_SSH_COMMAND sudo umount "$vbox_shared_folder"
done <<< "$vbox_shared_folders"
}

Expand Down Expand Up @@ -375,17 +381,79 @@ function init_boot2docker {
log_info "Starting Boot2Docker VM"
boot2docker start --vbox-share=disable
fi

configure_boot2docker
}

################################################################################
# docker-machine manipulation
################################################################################

#
# Read a json string from stdin and a key as argument, does a single string match and
# returns the first string value that matches the given key
# Examples:
# $ echo -e '{ "foo": "bar" }' | get_json_value "foo"
# bar
# $ echo -e '{ "foo": { "baz": "bar" } }' | get_json_value "baz"
# bar
#
function get_json_value () {
sed -n 's/.*"'"$*"'": *"\([^"]*\)".*/\1/p' | head -n 1
}

#
# Configures variables based on the output of `docker-machine inspect $DOCKER_MACHINE_NAME`
#
function configure_docker_machine {
DOCKER_HOST_NAME="$DOCKER_MACHINE_NAME"
DOCKER_MACHINE_INSPECT=$(docker-machine inspect "$DOCKER_MACHINE_NAME")
DOCKER_HOST_USER=$(echo -e "$DOCKER_MACHINE_INSPECT" | get_json_value "SSHUser")
DOCKER_HOST_IP=$(echo -e "$DOCKER_MACHINE_INSPECT" | get_json_value "IPAddress")
DOCKER_MACHINE_STORE_PATH=$(echo -e "$DOCKER_MACHINE_INSPECT" | get_json_value "StorePath")

DOCKER_HOST_SSH_URL="$DOCKER_HOST_USER@$DOCKER_HOST_IP"
DOCKER_HOST_SSH_KEY="$DOCKER_MACHINE_STORE_PATH/id_rsa"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way to avoid hard-coding id_rsa?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find any way to do that. docker-machine inspect doesn't expose this.

DOCKER_HOST_SSH_COMMAND="docker-machine ssh $DOCKER_MACHINE_NAME"
log_debug "Running for the docker-machine name: $DOCKER_MACHINE_NAME"
}

#
# Initializes and starts up the docker-machine VM.
#
function init_docker_machine {
log_info "Initializing docker machine $DOCKER_MACHINE_NAME"
docker-machine start "$DOCKER_MACHINE_NAME"
eval "$(docker-machine env $DOCKER_MACHINE_NAME)"
configure_docker_machine
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, here, this eval will run something like this:

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.103:2376"
export DOCKER_CERT_PATH="/Users/someone/.docker/machine/machines/local"
export DOCKER_MACHINE_NAME="local"

We can see that DOCKER_MACHINE_NAME will be redefined.

Also, as in the docker-machine documentation, if the user runs eval "$(docker-machine env <machine-name>)", it will define the DOCKER_MACHINE_NAME to the env. Then if he runs docker-osx-dev, he would not need to pass the --machine-name argument.

I guess this is fine, I can add to the Readme.md about this behavior.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


################################################################################
# Generic docker host manipulation
################################################################################

#
# Initializes a docker host
# Initializes boot2docker, unless DOCKER_MACHINE_NAME is defined
#
function init_docker_host {
if [ -n "$DOCKER_MACHINE_NAME" ]; then
init_docker_machine
else
init_boot2docker
fi
}

#
# Installs rsync on the Boot2Docker VM, unless it's already installed.
#
function install_rsync_on_boot2docker {
log_info "Installing rsync in the Boot2Docker image"
function install_rsync_on_docker_host {
log_info "Installing rsync in the Docker Host image"

boot2docker ssh "if ! type rsync > /dev/null 2>&1; then tce-load -wi rsync; fi"
$DOCKER_HOST_SSH_COMMAND "if ! type rsync > /dev/null 2>&1; then tce-load -wi rsync; fi"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick sanity check, in case you haven't done it already: search the code for boot2docker ssh calls and make sure all of them are updated to $DOCKER_HOST_SSH_COMMAND.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Yes, there are two left that I forgot. Thanks for calling that out.

}


################################################################################
# Environment setup
################################################################################
Expand Down Expand Up @@ -428,16 +496,18 @@ function get_env_file {
# Adds environment variables necessary for running Boot2Docker
#
function add_environment_variables {
local readonly env_file=$(get_env_file)
local readonly boot2docker_exports=$(boot2docker shellinit 2>/dev/null)
local readonly exports_to_add_to_env_file=$(determine_boot2docker_exports_for_env_file "$boot2docker_exports")

if [[ ! -z "$exports_to_add_to_env_file" ]]; then
log_info "Adding new environment variables to $env_file: $exports_to_add_to_env_file"
echo -e "$exports_to_add_to_env_file" >> "$env_file"
log_instructions "To pick up important new environment variables in the current shell, run:\n\tsource $env_file"
else
log_warn "All Boot2Docker environment variables already defined, will not overwrite"
if [ -z "$DOCKER_MACHINE_NAME" ]; then
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't used docker-machine much, so just to confirm: it doesn't rely on any environment variables?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Well, it does. Please see here: https://github.com/brikis98/docker-osx-dev/pull/77/files#r36412414

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So should the call that adds those env vars be moved into this function instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could. But add_environment_variables is writing to bashrc or zshrc. docker-machine env vars should be avoided to be written there. If the user has multiple machines the env vars would get duplicated.
Also the purpose of the init_docker_machine is to run the commands from the machine documentation: start and env.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotcha.

local readonly env_file=$(get_env_file)
local readonly boot2docker_exports=$(boot2docker shellinit 2>/dev/null)
local readonly exports_to_add_to_env_file=$(determine_boot2docker_exports_for_env_file "$boot2docker_exports")

if [[ ! -z "$exports_to_add_to_env_file" ]]; then
log_info "Adding new environment variables to $env_file: $exports_to_add_to_env_file"
echo -e "$exports_to_add_to_env_file" >> "$env_file"
log_instructions "To pick up important new environment variables in the current shell, run:\n\tsource $env_file"
else
log_warn "All Boot2Docker environment variables already defined, will not overwrite"
fi
fi
}

Expand Down Expand Up @@ -482,8 +552,8 @@ function add_docker_host {
if grep -q "^[^#]*$DOCKER_HOST_NAME" "$HOSTS_FILE" ; then
log_warn "$HOSTS_FILE already contains $DOCKER_HOST_NAME, will not overwrite"
else
local readonly boot2docker_ip=$(boot2docker ip)
local readonly host_entry="\n$boot2docker_ip $DOCKER_HOST_NAME"
DOCKER_HOST_IP=${DOCKER_HOST_IP-$(boot2docker ip)}
local readonly host_entry="\n$DOCKER_HOST_IP $DOCKER_HOST_NAME"

log_info "Adding $DOCKER_HOST_NAME entry to $HOSTS_FILE so you can use http://$DOCKER_HOST_NAME URLs for testing"
log_instructions "Modifying $HOSTS_FILE requires sudo privileges, please enter your password."
Expand Down Expand Up @@ -558,6 +628,7 @@ function install_dependencies {
brew_install "caskroom/cask/brew-cask" "Cask" "" false
brew_install "boot2docker" "Boot2Docker" "boot2docker" false
brew_install "docker-compose" "Docker Compose" "docker-compose" false
brew_install "docker-machine" "Docker Machine" "docker-machine" false
brew_install "fswatch" "fswatch" "fswatch" false
brew_install "coreutils" "GNU core utilities" "greadlink" false
}
Expand Down Expand Up @@ -628,9 +699,9 @@ function do_rsync {
read -a includes <<< "$INCLUDES"
local readonly include_flags="${includes[@]/#/--include }"

local readonly rsh_flag="--rsh=\"ssh -i $BOOT2DOCKER_SSH_KEY -o StrictHostKeyChecking=no\""
local readonly rsh_flag="--rsh=\"ssh -i $DOCKER_HOST_SSH_KEY -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null\""

local readonly rsync_cmd="rsync $RSYNC_FLAGS $include_flags $exclude_flags $rsh_flag $path_to_sync $BOOT2DOCKER_SSH_URL:$parent_folder"
local readonly rsync_cmd="rsync $RSYNC_FLAGS $include_flags $exclude_flags $rsh_flag $path_to_sync $DOCKER_HOST_SSH_URL:$parent_folder"
log_debug "$rsync_cmd"

eval "$rsync_cmd" 2>&1 | log_info
Expand Down Expand Up @@ -679,11 +750,11 @@ function initial_sync {

local readonly dir_string=$(join " " "${dirs_to_create[@]}")
local readonly mkdir_string="sudo mkdir -p $dir_string"
local readonly chown_string="sudo chown -R $BOOT2DOCKER_USER $dir_string"
local readonly chown_string="sudo chown -R $DOCKER_HOST_USER $dir_string"
local readonly ssh_cmd="$mkdir_string && $chown_string"

log_debug "Creating parent directories in Docker VM: $ssh_cmd"
boot2docker ssh "$ssh_cmd"
$DOCKER_HOST_SSH_COMMAND "$ssh_cmd"

sync "${paths_to_sync[@]}"
log_info "Initial sync done"
Expand Down Expand Up @@ -726,6 +797,7 @@ function instructions {
echo -e " $INSTALL_COMMAND\tInstall docker-osx-dev and all of its dependencies."
echo -e
echo -e "Options:"
echo -e " -m, --machine-name name\t\tWhen suplied syncs with the given docker machine host"
echo -e " -s, --sync-path PATH\t\t\tSync PATH to the Boot2Docker VM. No wildcards allowed. May be specified multiple times. Default: $DEFAULT_PATHS_TO_SYNC"
echo -e " -e, --exclude-path PATH\t\tExclude PATH while syncing. Behaves identically to rsync's --exclude parameter. May be specified multiple times. Default: $DEFAULT_EXCLUDES"
echo -e " -c, --compose-file COMPOSE_FILE\tRead in this docker-compose file and sync any volumes specified in it. Default: $DEFAULT_COMPOSE_FILE"
Expand All @@ -735,7 +807,7 @@ function instructions {
echo -e
echo -e "Overview:"
echo -e
echo -e "docker-osx-dev is a script you can use to sync folders to the Boot2Docker VM using rsync."
echo -e "docker-osx-dev is a script you can use to sync folders to the Boot2Docker (or docker-machine) VM using rsync."
echo -e "It's an alternative to using VirtualBox shared folders, which are agonizingly slow and break file watchers."
echo -e "For more info, see: https://github.com/brikis98/docker-osx-dev"
echo -e
Expand Down Expand Up @@ -942,9 +1014,8 @@ function configure_includes {
#
function watch_and_sync {
log_info "Starting docker-osx-dev file syncing"
configure_boot2docker_ssh_key
init_boot2docker
install_rsync_on_boot2docker
init_docker_host
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you not call install_rsync_on_docker_host here as before?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can move back here. But I believe this slows down the startup of docker-osx-dev after the install.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't cause much slow down, as the command checks if rsync is installed before trying to install it. The benefit is that if you create a new docker VM for whatever reason, rsync will get installed on it any time you run docker-osx-dev.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I can add that back.

install_rsync_on_docker_host
initial_sync
watch
}
Expand All @@ -956,8 +1027,8 @@ function watch_and_sync {
function install {
log_info "Starting install of docker-osx-dev"
install_dependencies
init_boot2docker
install_rsync_on_boot2docker
init_docker_host
install_rsync_on_docker_host
add_docker_host
add_environment_variables
print_next_steps
Expand Down Expand Up @@ -1058,6 +1129,11 @@ function handle_command {
log_level="$2"
shift
;;
-m|--machine-name)
assert_valid_arg "$2" "$key"
DOCKER_MACHINE_NAME="$2"
shift
;;
-h|--help)
instructions
exit 0
Expand Down
86 changes: 86 additions & 0 deletions test/docker-osx-dev.bats
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
source src/docker-osx-dev "test_mode"
load test_helper


@test "index_of doesn't find match in empty array" {
array=()
run index_of "foo" "${array[@]}"
Expand Down Expand Up @@ -415,3 +416,88 @@ export NEW_ENV_VARIABLE_2=VALUE2"
run find_path_to_sync_parent "/some/path/.git/foo"
assert_output '/some/path'
}

@test "get_json should get value for a single key json" {
output=$(echo -e '{ "foo": "bar" }' | get_json_value "foo")

assert_output "bar"
}

@test "get_json should get value for a nested json" {
output=$(echo -e '{ "foo": { "bar": "baz" } }' | get_json_value "bar")

assert_output "baz"
}

@test "get_json should get value for a json with a key that appears twice" {
output=$(echo -e '{ "foo": { "k": "first" },\n "k": "second" }' | get_json_value "k")

assert_output "first"
}

@test "get_json should return empty for a json with a non existent key" {
output=$(echo -e '{ "foo": "val1", "bar": "val2" }' | get_json_value "baz")

assert_output ""
}

@test "get_json should return empty for a json with the value as integer" {
output=$(echo -e '{ "foo": 22 }' | get_json_value "foo")

assert_output ""
}

@test "get_json should return empty for a json with the value as object" {
output=$(echo -e '{ "foo": {"bar" : "baz"} }' | get_json_value "foo")

assert_output ""
}

@test "get_json should return empty for a json with the value as array" {
output=$(echo -e '{ "foo": ["bar"] }' | get_json_value "foo")

assert_output ""
}


@test "init_docker_host should call configure_boot2docker set DOCKER_HOST vars" {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add a bunch of unit tests for the get_json_value function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

unset DOCKER_HOST_NAME
stub boot2docker 'SSHKey = "/Users/someone/.ssh/id_boot2docker"'
export PATH="$BATS_TEST_DIRNAME/stub:$PATH"

init_docker_host

assert_equal "docker" "$DOCKER_HOST_USER"
assert_equal "docker@dockerhost" "$DOCKER_HOST_SSH_URL"
assert_equal "boot2docker ssh" "$DOCKER_HOST_SSH_COMMAND"
assert_equal "/Users/someone/.ssh/id_boot2docker" "$DOCKER_HOST_SSH_KEY"
rm_stubs
}

@test "configure_docker_machine should set DOCKER_HOST vars" {
export DOCKER_MACHINE_NAME="some-machine"
docker_inspect_output=$(cat <<EOF
{
"Driver": {
"SSHUser": "user",
"IPAddress": "10.254.1.14"
},
"StorePath": "/Users/someone/.docker/machine/machines/some-machine"
}
EOF
)
stub docker-machine "$docker_inspect_output"
export PATH="$BATS_TEST_DIRNAME/stub:$PATH"

configure_docker_machine

assert_equal "some-machine" "$DOCKER_HOST_NAME"
assert_equal "user" "$DOCKER_HOST_USER"
assert_equal "10.254.1.14" "$DOCKER_HOST_IP"

assert_equal "/Users/someone/.docker/machine/machines/some-machine/id_rsa" "$DOCKER_HOST_SSH_KEY"
assert_equal "user@10.254.1.14" "$DOCKER_HOST_SSH_URL"
assert_equal "docker-machine ssh some-machine" "$DOCKER_HOST_SSH_COMMAND"
rm_stubs
}

Loading