-
-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CRITICAL BUG] event serialization error in store.py #330
Comments
@bhavikapanara looks like the numpy.float32 doesn't have a default string representation. Here is more on this. Should be a simple fix. |
Please test thoroughly end to end with UI rendering falls and object detections in the timeline. |
@bhavikapanara there is a new critical serialization issue after the #332 merge. Worse yet, the new merge was pushed to production and effectively stopped all edge devices from functioning. Now we not only have to promptly resolve the root cause, but also include a recovery script in the next merge that cleans up corrupted timeline event files and allows edge devices to resume function without user intervention. Not an ideal situation, but this is the price we pay for incomplete end to end testing. Looking at the exception in the edge log, the format store.py serializes post #332 merge is some sort of weird proprietary stuff that cannot be deserialized by the regular json library used in the flasker web API. Please review carefully and make sure to test end to end, including UI before submitting the next PR. Also add more tests for to REST API to be sure it can function with the event serialization format. Here is the new exception in the web API when it tries to respond to a UI request:
And here is what I see in the events log. See how It moved from a previously simple and clean serialization format to a new post merge format that looks weird cannot be deserialized easily:
|
@bhavikapanara here is a possible solution to the graceful recovery from corrupted event timeline files. Starting here:
add handler for the new exception we are seeing : as follows
This will unfortunately wipe out all events in the file that has corrupted data, including good ones, which is not ideal, but hopefully we will not get into this same type of issues anytime soon. |
@ivelin Instead of removing the timeline, can we ignore the only corrupt event data to render on UI? |
That would be better. How do you suggest we do that? |
Users are starting to complain that their edge is not responsive. We need to push a fix sooner than later. |
Yes...will do that as soon as possible |
🎉 This issue has been resolved in version 1.14.5 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
## [1.11.1](v1.11.0...v1.11.1) (2021-05-17) ### Bug Fixes * build and publish python API docs ([2091c54](2091c54)), closes [ambianic#342](https://github.com/ivelin/ambianic-edge/issues/342) * improve fall detection debug info ([7762752](7762752)), closes [ambianic#309](https://github.com/ivelin/ambianic-edge/issues/309) * json serialization redundancy issue in store.py; closes [ambianic#337](https://github.com/ivelin/ambianic-edge/issues/337); closes [ambianic#300](https://github.com/ivelin/ambianic-edge/issues/300) ([bb8d20a](bb8d20a)), closes [ambianic#339](https://github.com/ivelin/ambianic-edge/issues/339) * revert [ambianic#314](https://github.com/ivelin/ambianic-edge/issues/314); see [ambianic#330](https://github.com/ivelin/ambianic-edge/issues/330) for details; separate inference output formatting for each detection model; closes [ambianic#254](https://github.com/ivelin/ambianic-edge/issues/254)" ([ee79c40](ee79c40)), closes [ambianic#329](https://github.com/ivelin/ambianic-edge/issues/329) * separate inference output formatting for each detection model; [issues [ambianic#330](https://github.com/ivelin/ambianic-edge/issues/330), [ambianic#254](https://github.com/ivelin/ambianic-edge/issues/254)] ([b83e4f8](b83e4f8)), closes [ambianic#332](https://github.com/ivelin/ambianic-edge/issues/332) * separate inference output formatting for each detection model; closes [ambianic#254](https://github.com/ivelin/ambianic-edge/issues/254) ([8768c17](8768c17)), closes [ambianic#314](https://github.com/ivelin/ambianic-edge/issues/314) * temporary solution to manage yaml.dump() encoding issue; closes [ambianic#330](https://github.com/ivelin/ambianic-edge/issues/330) ([db4b00a](db4b00a)), closes [ambianic#333](https://github.com/ivelin/ambianic-edge/issues/333)
Describe the bug
After merging PR #314 , there is an exception in store.py when it tries to save a detection event.
To Reproduce
Run edge with fall detection and observe log.
The text was updated successfully, but these errors were encountered: