fix(clouddriver): Return a call stack for better error handling #5584
Conversation
wattdave
requested review from
ajordens,
aravindmd and
jeyrschabu
as code owners
|
The following commits need their title changed:
Please format your commit title into the form:
This allows us to easily generate changelogs & determine semantic version numbers when cutting releases. You can read more about commit conventions here. |
wattdave
changed the title
|
The build errors here puzzle me. Can anyone provide some information on how my changes might have caused them? |
|
@Mergifyio update |
✅ Branch has been successfully updated |
Pretty sure #5585 fixes that error, which is why I updated your branch to pick it up. |
| @@ -550,7 +550,9 @@ class ClusterCachingAgent implements CachingAgent, OnDemandAgent, AccountAware, | |||
| cacheTargetGroups(data, targetGroups) | |||
| cacheLaunchTemplate(data, launchTemplates) | |||
| } catch (Exception ex) { | |||
| log.warn("Failed to cache ${asg.autoScalingGroupName} in ${account.name}/${region}", ex) | |||
There was a problem hiding this comment.
I'm surprised this log statement doesn't print a stack trace already.
There was a problem hiding this comment.
Was going to comment the same but figured I must be missing something obvious.
Wonder if it's something like this? https://stackoverflow.com/a/9387598
There was a problem hiding this comment.
Here's the log it provides, unfortunately: message: Failed to cache fuzzed-metric-name-Metrics fuzzed_asg in fuzzed-location/us-west-2 stack_trace: java.lang.NullPointerException: null (copied from our Splunk.)
There was a problem hiding this comment.
For what it's worth, we find lots of other useful stack traces in other locations in the logs. Not sure why this one is special, but it very much is.
There was a problem hiding this comment.
Searching a bit through the code, I see that log.warn('stuff', exception) is uncommon in the code. https://github.com/spinnaker/clouddriver/blob/master/clouddriver-artifacts/src/main/java/com/netflix/spinnaker/clouddriver/artifacts/http/HttpArtifactConfiguration.java#L53 https://github.com/spinnaker/clouddriver/blob/master/cats/cats-sql/src/main/kotlin/com/netflix/spinnaker/cats/sql/cache/SqlCache.kt#L917 https://github.com/spinnaker/clouddriver/blob/master/clouddriver-azure/src/main/groovy/com/netflix/spinnaker/clouddriver/azure/health/AzureHealthIndicator.groovy#L79 https://github.com/spinnaker/clouddriver/blob/master/clouddriver-web/src/main/java/com/netflix/spinnaker/clouddriver/controllers/ServerGroupManagerController.java#L64...
There was a problem hiding this comment.
Wonder if the fact that this is groovy is causing grief here?
There was a problem hiding this comment.
Erm, well - I would have to deploy it onto our local Spinnaker servers to be able to answer that, and I don't have access to do that.
There was a problem hiding this comment.
I presume we're on a later slf4j than 1.6.0? This link smells like what we're seeing: https://stackoverflow.com/questions/5951209/how-to-log-exception-and-message-with-placeholders-with-slf4j
There was a problem hiding this comment.
Let me put together a quick sample app to tinker with this...
There was a problem hiding this comment.
FYI:
spinnaker/spinnaker#5473
i think is also relevant ;) I'm watching this & other thread ;) Was rather confused to why stack traces aren't working myself.
We are seeing large numbers of "NullPointerException" from this particular exception in our CloudDriver logs, and we would like to isolate where underneath the problem lies. The code as written prints only the exception name in the logs, with no stack trace.