Skip to content
This repository has been archived by the owner on Mar 30, 2023. It is now read-only.

Add Logstash to the Marketplace UI and ARM template #223

Merged
merged 32 commits into from
Oct 4, 2018
Merged

Conversation

russcam
Copy link
Contributor

@russcam russcam commented Sep 28, 2018

This PR adds Logstash to the Marketplace UI and ARM template. This is an initial implementation, with possibility to expand in future based on feedback and usage.

  • Provide the option to deploy a single instance of Logstash, with configurable VM SKU. The default will remain the same and not deploy Logstash.

  • Allow configurable JVM heap size.

  • Allow a Base-64 encoded string form of a Logstash conf file be provided, to start Logstash with. Expose variables that can be referenced within the conf file

    • ${ELASTICSEARCH_URL}: the Elasticsearch endpoint
    • ${LOGSTASH_SYSTEM_PASSWORD}: password of the built-in logstash_system user
    • ${ELASTICSEARCH_CACERT}: the path to the CA cert used to secure the Elasticsearch

    The value is Base-64 encoded to match how the value is encoded when received from an uploaded file within the UI.

  • Allow Logstash keystore password to be specified, for use in Logstash versions that have keystore functionality.

  • Allow additional Logstash plugins to be installed.

  • Allow additional YAML configuration to append to logstash.yml configured by the template.

  • For Logstash 6.3.0+ and for Logstash < 6.3.0 where X-Pack plugin is installed, configure X-Pack monitoring. Document whether monitoring is enabled based on if and how TLS is configured for the Elasticsearch HTTP layer.

Closes #157

russcam and others added 30 commits September 6, 2018 14:35
This commit separates out the installation of Java into a separate script, to be reused by both Elasticsearch and Logstash installations
@russcam
Copy link
Contributor Author

russcam commented Sep 28, 2018

Before this PR is merged, a 6.3 branch must be created and the elastic.co docs set to pick up documentation from that branch. This then makes the way to create a 6.4 branch which will include documentation about Logstash (in addition to subsequent PRs that will add support for 6.4.0 and 6.4.1).

@russcam
Copy link
Contributor Author

russcam commented Sep 28, 2018

@jsvd I've requested a review 😃

Copy link
Member

@Mpdreamz Mpdreamz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, 👍 on the extensive integration tests.

<strong>Check that the size you select is <a href="https://azure.microsoft.com/en-au/regions/services/">available in the region you choose</a></strong>.
</td><td><code>Standard_D1</code></td></tr>

<tr><td>logstashHeapSize</td><td>integer</td>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to expose this? Since logstash always deploys on a dedicated VM can we just do the right thing™?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the heap size? If so, I wasn't sure what might be a sensible default, beyond the default in JVM.options (which I think is 1GB?). Thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@russcam mind opening an issue for whether we can rid of the parameter?

@russcam russcam merged commit db37647 into master Oct 4, 2018
@russcam russcam deleted the feature/logstash branch October 16, 2018 04:31
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants