-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sanitize NAN values from timeseries data #353
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please explain an example of how to reproduce the problem in the "Motivation and Context" section and explain in more detail how you tested it on xdmod-dev in the "Tests performed" section?
Without being sure how to reproduce the error, I'm not sure yet if changing NaN to zero is the right solution.
…dded docblock comment. Fixed static call of sanitize in gettimeseries()
…allback using pass by reference to achieve same goal
Having thought about this - I think that the NaN values should be removed from the data series rather than set to 0 - since 0 could be a valid value from a valid measurement and NaN typically indicates that there was not a valid measurement. |
Looks like the PR description should be updated to say it replaces them with null instead of 0. |
Updated the description that the value is replaced with |
Description
Add check to sanitize job timeseries data of NAN values due to Symfony throwing double NAN error when json encoding. Added helper function which replaces any NAN values in the series data with
null
.Motivation and Context
Charts with NAN data would return a null record.
Tests performed
Observed changes on xdmod-dev. Job that has a NaN value(s) can be seen here .
After checking mongodb-access, it was confirmed the value is NaN inside Mongodb for the job linked above. There was an error for the cpuuser value that was NaN stating that it couldn't collect enough data for the specified time period.
Types of changes
Checklist: