-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added JSONDecodeError for rapidjson >= 0.7.1 with fallback to ValueError. #199
Conversation
# Defined for rapidjson >= 0.7.1 | ||
from rapidjson import JSONDecodeError | ||
except ImportError: | ||
JSONDecodeError = ValueError |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before 0.7.1, the error thrown was a ValueError. This should ensure that the error in contrib.filterparse
still gets caught.
I'd like to add a step to CI that runs tests with rapidjson if we're committed to supporting it as a primary JSON backend, going forward. Edit: oh wait, we are testing this already. I will try to add a test that fails without this PR. |
Codecov Report
@@ Coverage Diff @@
## master #199 +/- ##
==========================================
- Coverage 64.9% 64.04% -0.87%
==========================================
Files 37 37
Lines 5557 5562 +5
==========================================
- Hits 3607 3562 -45
- Misses 1950 2000 +50
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #199 +/- ##
==========================================
+ Coverage 64.9% 64.99% +0.08%
==========================================
Files 37 37
Lines 5557 5565 +8
==========================================
+ Hits 3607 3617 +10
+ Misses 1950 1948 -2
Continue to review full report at Codecov.
|
fwiw I think when we last discussed this we came down on not knowing whether rapidjson was really the right JSON backend to choose, but we needed to do a more comprehensive survey of libraries to know what to support and we hadn't assigned that to anyone. |
I'm not sure I follow exactly what additional test you want to add, but this PR looks good to me right now. One minor suggestion, for redirecting stderr you might want to just copy over the functions we use in testing signac-flow. |
@vyasr I did some reading on that subject while working on this PR. Seeing this comment on rapidjson's issue tracker suggests to me that we would be better served by |
@vyasr I added the test already, so that's done. I also made the change you suggested to use signac-flow's redirection. I think this PR is ready to merge. |
I agree, should be ready for merge. And regarding choice of library, to my knowledge ujson provides the best balance of compatibility with the Python library's json module and performance, but I haven't done exhaustive research into that. We might just need to make a quick test where replace the code in signac's json module with the relevant import and see how performance improves/degrades. Actually changing which modules we support should not be too hard once we know the correct solution that provides the closest mirror to the existing functionality we expect. |
I just went to look at issues and found #171 which I made :D good to know we're rehashing and coming to the same conclusions, it looks like your benchmarks agree with ujson as the best alternative. @csadorf if you want to assign that issue to a particular benchmark we can assign it to someone to work on. |
Motivation and Context
Resolves #172.
Types of Changes
Checklist:
If necessary: