Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build docker images when building packages #8898

Merged
merged 30 commits into from
Nov 13, 2018

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Nov 1, 2018

Include generation of docker images in the process of building the rest of released packages by extending mage package. Ideally it could be extended to build images for windows too.

Dockerfile, entry point templates and default beats configurations based on current ones.

Pending:

  • Fix permission issues
  • Export built image as artifact with docker save or similar
  • Complete parametrization of docker options
  • Make it work for all beats
    • Support capabilities for packetbeat and heartbeat
    • Run as root for auditbeat
    • Heartbeat user is heartbeat-elastic 🤔 it should be heartbeat as other beats
    • Install libsystemd in journalbeat (systemd-libs already installed in centos:7 base image)
    • Support community beats
  • Check that beats can be configured using environment variables
  • Fix license label for licensed builds
  • Add packaging tests for docker

@jsoriano jsoriano added :Packaging in progress Pull request is currently in progress. v7.0.0 labels Nov 1, 2018
@jsoriano jsoriano changed the title Build docker images when building rest of packages Build docker images when building packages Nov 1, 2018
@jsoriano jsoriano added review and removed in progress Pull request is currently in progress. labels Nov 6, 2018
@jsoriano
Copy link
Member Author

jsoriano commented Nov 6, 2018

@andrewkroh @graphaelli some sanity tests are still pending, but I think this is ready for a first review, it'd be nice if you could give it another look. Thanks!

dev-tools/packaging/packages.yml Outdated Show resolved Hide resolved
dev-tools/mage/dockerbuilder.go Show resolved Hide resolved
dev-tools/packaging/packages.yml Show resolved Hide resolved
@@ -457,3 +502,122 @@ func readZip(zipFile string) (*packageFile, error) {

return p, nil
}

func readDocker(dockerFile string) (*packageFile, *dockerInfo, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting 👍

Copy link
Member

@graphaelli graphaelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the repo and ports updates, they worked as expected. No other changes from APM server are needed.

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work.

Are there any plans to add some additional tests that use the built containers? Like system tests that use the containers. I haven't thought through what would be useful to test so I'm just wondering.

.gitignore Outdated Show resolved Hide resolved
dev-tools/packaging/package_test.go Outdated Show resolved Hide resolved
@jsoriano
Copy link
Member Author

@andrewkroh thanks for the review, I have addressed your comments.

Regarding additional tests, it could be possible, we could maybe modify system tests to use docker, so the docker image is also tested with them, but I don't have any plan for it at the moment.

@jsoriano jsoriano merged commit 9294384 into elastic:master Nov 13, 2018
@jsoriano jsoriano deleted the mage-package-docker branch November 13, 2018 19:27
@jsoriano jsoriano added the v6.6.0 label Dec 7, 2018
jsoriano added a commit to jsoriano/beats that referenced this pull request Dec 7, 2018
Include generation of docker images in the process of building the
rest of released packages by extending `mage package`.

(cherry picked from commit 9294384)
jsoriano added a commit that referenced this pull request Dec 8, 2018
…uilding packages (#9433)

Include generation of docker images in the process of building the
rest of released packages by extending `mage package`. (#8898)

(cherry picked from commit 9294384)

Tag container images with snapshot information (#9063)

(cherry picked from commit 1dd4fbf)

* Compress docker artifacts with gzip (#9223)

(cherry picked from commit 675f2b0)

Co-authored-by: Gil Raphaelli <g@raphaelli.com>
jsoriano added a commit to jsoriano/beats that referenced this pull request Feb 27, 2019
… docker images when building packages (elastic#9433)

Include generation of docker images in the process of building the
rest of released packages by extending `mage package`. (elastic#8898)

(cherry picked from commit 9294384)

Tag container images with snapshot information (elastic#9063)

(cherry picked from commit 1dd4fbf)

* Compress docker artifacts with gzip (elastic#9223)

(cherry picked from commit 675f2b0)

Co-authored-by: Gil Raphaelli <g@raphaelli.com>
DStape pushed a commit to DStape/beats that referenced this pull request Aug 20, 2019
Include generation of docker images in the process of building the
rest of released packages by extending `mage package`.
DStape pushed a commit to DStape/beats that referenced this pull request Aug 20, 2019
… docker images when building packages (elastic#9433)

Include generation of docker images in the process of building the
rest of released packages by extending `mage package`. (elastic#8898)

(cherry picked from commit 9294384)

Tag container images with snapshot information (elastic#9063)

(cherry picked from commit 1dd4fbf)

* Compress docker artifacts with gzip (elastic#9223)

(cherry picked from commit 675f2b0)

Co-authored-by: Gil Raphaelli <g@raphaelli.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants