Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(kitchen+travis): use bootstrapped amazonlinux-1 images #50

Conversation

myii
Copy link
Member

@myii myii commented Nov 4, 2019


@dafyddj Needed amazonlinux-1 for the epel-formula so took this opportunity to standardise the way bootstrapped images are provided by the ssf-formula. Up to now, it's been using an inline if block solely for the vault-formula; now it can provide amazonlinux-1 to formulas in the same way as pre-salted images.

I've tested all of these across all three suites (and the combined one) here:

@myii myii requested a review from dafyddj November 4, 2019 10:26
Copy link
Contributor

@dafyddj dafyddj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad to see the AmazonLinux 1 test platforms filled out with the different Salt releases.
However I do have one comment.

There is a difference between using salt-bootstrap in the provisioner section and adding a provision_command in the driver section.
The latter adds a RUN instruction to the Dockerfile used to create the image, which takes advantage of the Docker layer cache, making subsequent runs very fast.
The former method means that Salt is installed from scratch the first time every instance is created/converged.

The time difference is significant:

# First run `provision_command`
dafydd@meshuga salt-formulas/vault-formula (dev)$ kitchen test amazonlinux$
...
-----> Kitchen is finished. (4m4.93s)

# Subsequent run `provision_command`
dafydd@meshuga salt-formulas/vault-formula (dev)$ kitchen test amazonlinux$
...
-----> Kitchen is finished. (2m40.61s)

# Docker images pruned and removed before following test
# First run `salt_install: bootstrap`
dafydd@meshuga salt-formulas/vault-formula (ci/use-bootstrapped-amazonlinux-1-images)$ kitchen test amazonlinux-1-2019-2-py2
...
-----> Kitchen is finished. (6m46.85s)

# Subsequent run `salt_install: bootstrap`
dafydd@meshuga salt-formulas/vault-formula (ci/use-bootstrapped-amazonlinux-1-images)$ kitchen test amazonlinux-1-2019-2-py2
...
-----> Kitchen is finished. (5m4.35s)

That being said, I don't have a huge preference either way, as I don't see AmazonLinux 1 being heavily used in local development any more (where the time savings would make the most difference), but just wanted to make the point, and leave something to refer to in the future.

So I'm happy to merge it if you still want to.

@myii
Copy link
Member Author

myii commented Nov 5, 2019

@dafyddj Nope, let's not merge this just yet, after your excellent feedback. That's got me wondering, though: when is the right time to use the provisioner? Surely there must be some reason for having that feature; are you aware of what that might be? As far as I can recall, most/all of the formulas were using this rather than the provision_command, before we started work on all of this standardisation.

@myii
Copy link
Member Author

myii commented Nov 5, 2019

My initial test related to this PR appear to confirm this.

@dafyddj
Copy link
Contributor

dafyddj commented Nov 5, 2019

Surely there must be some reason for having that feature; are you aware of what that might be?

The speed difference is very specific to Docker and the way it builds up an image from cached layers. Another driver such as kitchen-vagrant wouldn't see the speed increase, and so either method could be used, so I think this is just a case of "there's more than one way to do it" - some ways turn out better than others.

Also, the use of pre-Salted images made this issue largely irrelevant.

@myii
Copy link
Member Author

myii commented Nov 5, 2019

@dafyddj Of course, kitchen-vagrant and other provisioners, thanks for reminding me of that.

I'll make the adjustments in the ssf-formula and then finalise this PR, probably within the next day.

@myii myii force-pushed the ci/use-bootstrapped-amazonlinux-1-images branch from 4e72c3b to a9576e8 Compare November 6, 2019 10:22
myii added a commit to myii/epel-formula that referenced this pull request Nov 6, 2019
myii added a commit to myii/ssf-formula that referenced this pull request Nov 6, 2019
@myii
Copy link
Member Author

myii commented Nov 6, 2019

@dafyddj I've updated this PR. I've also applied these changes to saltstack-formulas/epel-formula#55. Would you mind merging these?

myii added a commit to myii/ssf-formula that referenced this pull request Nov 6, 2019
@myii myii merged commit 3f8d3a9 into saltstack-formulas:master Nov 7, 2019
@myii myii deleted the ci/use-bootstrapped-amazonlinux-1-images branch November 7, 2019 00:37
@myii
Copy link
Member Author

myii commented Nov 7, 2019

Merging since already confirmed essentially.

myii pushed a commit to myii/ssf-formula that referenced this pull request Nov 7, 2019
# [1.48.0](v1.47.0...v1.48.0) (2019-11-07)

### Features

* **kitchen:** use `provision_command` for `amazonlinux-1` images ([b7835f2](b7835f2)), closes [/github.com/saltstack-formulas/vault-formula/pull/50#pullrequestreview-312037893](https://github.com//github.com/saltstack-formulas/vault-formula/pull/50/issues/pullrequestreview-312037893)
* **vault:** use bootstrapped `amazonlinux-1` images ([df1b8d6](df1b8d6)), closes [#92](#92)
@saltstack-formulas-travis

🎉 This PR is included in version 1.2.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

kitchen.yml Show resolved Hide resolved
myii pushed a commit to myii/ssf-formula that referenced this pull request Nov 9, 2019
saltstack-formulas-travis pushed a commit to saltstack-formulas/epel-formula that referenced this pull request Feb 13, 2020
## [1.15.1](v1.15.0...v1.15.1) (2020-02-13)

### Bug Fixes

* **map.jinja:** update link to `rpm` for `EPEL-8` ([3e90e0d](3e90e0d))

### Continuous Integration

* **gemfile:** restrict `train` gem version until upstream fix [skip ci] ([bddcd80](bddcd80))
* **kitchen:** avoid using bootstrap for `master` instances [skip ci] ([30ade3d](30ade3d))
* **kitchen:** use `develop` image until `master` is ready (`amazonlinux`) [skip ci] ([5db2344](5db2344))
* **kitchen:** use `provision_command` for `amazonlinux-1` images ([2cfcfde](2cfcfde)), closes [/github.com/saltstack-formulas/vault-formula/pull/50#pullrequestreview-312037893](https://github.com//github.com/saltstack-formulas/vault-formula/pull/50/issues/pullrequestreview-312037893)
* **kitchen:** use `stable` for `amazonlinux-1` bootstrap [skip ci] ([472434b](472434b))
* **kitchen+travis:** use bootstrapped `amazonlinux-1` images [skip ci] ([37d2fc6](37d2fc6))
* **travis:** apply changes from build config validation [skip ci] ([904b4fc](904b4fc))
* **travis:** opt-in to `dpl v2` to complete build config validation [skip ci] ([5fa5251](5fa5251))
* **travis:** quote pathspecs used with `git ls-files` [skip ci] ([6d28624](6d28624))
* **travis:** run `shellcheck` during lint job [skip ci] ([8491178](8491178))
* **travis:** use `major.minor` for `semantic-release` version [skip ci] ([f7f2342](f7f2342))
* **travis:** use build config validation (beta) [skip ci] ([ecab107](ecab107))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants