Notes on diagnosing the cause of Zeppelin/Spark/Yarn failures #566
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The contents of that file are all Greek to me ... over to Stelios :) |
stvoutsin
approved these changes
Aug 26, 2021
|
|
||
| hs_err_pid27205.log | ||
| "There is insufficient memory for the Java Runtime Environment to continue." | ||
|
|
There was a problem hiding this comment.
So these are two separate issues? Or are they related?
Issue 1: Zeppelin ran out of disk space due to log files
Issue 2: Worker nodes dies with out of memory exceptions
Any hints in Grafana/Prometheus? We should be able to go back and check resources usage at the time if the monitor node wasn't killed.
There was a problem hiding this comment.
Separate issues that happened at separate times.
There was a problem hiding this comment.
Need to do some more work on the Grafana/Prometheus dashboards before we can get really useful information out of them.
Getting there, but not the primary information source yet.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.