Restarting Elasticsearch after the first start is not ideal #141
Assignees
Labels
feature
New feature or request
Comments
|
I'd love to minimize restarts. The current version is just the outcome of sometimes needing Elasticsearch running and sometimes not. If you find a better way, I'm all for it. Maybe the current way isn't needed any more due to all the changes we had. |
github-merge-queue bot
pushed a commit
that referenced
this issue
Oct 16, 2023
Restarting Elasticsearch takes quite a while and may lead to connection issues as well as sync issues. So keeping restarts to a minimum is important. These changes will make sure that, even when the `Restart Elasticsearch` handler is notified, it will only restart if Elasticsearch was running before. If there's a fresh start (after reconfiguration) we don't need to restart again. Same goes for Logstash and Kibana. Some restarts of these tools happen fairly fast. But others (like after fresh installs or updates) will trigger internal jobs that should not be intercepted by another restart. Beats restart very fast and as far as I know there's not a big downside to restarting them right after the first start so I didn't include them in the change. Additionally, this PR will make sure some tasks in `verify.yml` of the full stack are only run when the service to be checked is actually running on this node. This helps with spreading services over nodes to save ressources. Since GitHub hosted runners are quite low on ressources we can't run every service on every node in a cluster setup anymore. So this PR will make sure that only Elasticsearch runs everywhere and the others are spread out. Caches get cleared after every role in during a Molecule test. This helps with saving ressources, too. Elasticsearch still won't sync all shards due to full volumes, the watermarks for Elasticseach are set to extremely high volumes so that the cluster can at least get into sync. fixes #278 fixes #141 fixes #194
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We flush handlers in security file yaml after the first start of the new initialized cluster. That lead to directly restart of Elasticsearch after the first start. This is maybe not ideal. We can put a when condition to adjust this behavior.
Also, we put two tasks to start Elasticsearch. The first one in security file yaml and the next one in main.yml. This led to two changes in Ansible output. What about putting when condition to not trigger the second one if security is enabled?
What do you think @widhalmt?
The text was updated successfully, but these errors were encountered: