Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to select path segments using Python re regexprs #146

Closed
wants to merge 14 commits into from

Conversation

AlainLich
Copy link

Hi,
In case there is an interest, this allows to use python re regular expressions, like in following examples:

import dpath
dpath.options.DPATH_ACCEPT_RE_REGEXP = True

js = loadJson()
selPath = 'Config/{(Env|Cmd)}'
x = dpath.util.search(js.lod, selPath)
print(x)

selPath = [ re.compile('(Config|Graph)') , re.compile('(Env|Cmd|Data)') ]
x = dpath.util.search(js.lod, selPath)
print(x)

selPath = '{(Config|Graph)}/{(Env|Cmd|Data)}'
x = dpath.util.search(js.lod, selPath)
print(x)

I found this useful navigating JSON.

Remarks:

. This runs under Python3 as shown on https://github.com/AlainLich/dpath-python/actions/runs/896380770 ;
. the trickwith dpath.options.DPATH_ACCEPT_RE_REGEXP, means the option has to be enabled make this compatible
with earlier versions and will not break if users have path segments in curly brackets.

WARNING: this will not work under Python2.7, since it requires Python3 features like 'f' strings; however I might
do a compatible version if there is interest.

Changes (most in random tests):

  1. quite straightforward changes in dpath/segments.py
  2. added DPATH_ACCEPT_RE_REGEXP in options as explained above
  3. in dpath/util.py re.compile extended regexprs once when they are found
  4. added `test/test_path_ext.py ̀ to test this extension
  5. made all tests using hypothesis run under unittest, in particular this permits to select and reproduce a single test function
  6. debugged a rare situation in the random test generation, reproduced in file issues/err_walk.py where a path segment
    gets generated which is interpreted by the bash globbing function resulting in a test failure (e.g. ̀seg[0]) which is globbed into something accepting seg0` but not its literal version with the brackets

…tests in conformance with options.ALLOW_EMPTY_STRING_KEYS

Still required to add tests for regext path segments.
Ran tests with Python-3 and Python-2.7 (nose)
Still needed: test and documentation of added functionality
Random test has shown 1 issue not related to new feature, non repeatable suspect binary string
with single null byte;documented in  issues/nonrepeatableErr.txt.
…ments

Uncontrolled test complexity apparently caused by translation of "***.."" into
costly regexp in test framework; now simplified.
…y, test generator

coming up with keys interpreted literally when walking tree, but interpreted as a bash glob
when matching. See in file issues/err_walk.py. (e.g '[0]' is also a glob meaning '0' ...)
@moomoohk
Copy link
Collaborator

moomoohk commented Dec 4, 2022

@AlainLich Hey, are you still interested in working on this PR?

@AlainLich
Copy link
Author

@moomoohk Hello, I have not touched this for a year and a half, could be interested to work on this PR, likely at a slow pace, since this would facilitate my own use (loading via pip ).

However, only on Python3... not interested in Python2 compatibility and such.

@moomoohk
Copy link
Collaborator

moomoohk commented Dec 4, 2022

That's great to hear :)
Slow pace is alright by me, as well as Python 3+. On that note, I've decided to drop Python 2 support entirely a while ago!
Minimum supported version is 3.7 as of now.
I've refactored the codebase quite a bit in the last year and a half. Mainly modernizing, upgrading to Python 3 and blowing some dust off...
If you have any questions feel free to reach out here or on Gitter.

@AlainLich
Copy link
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch master or any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@moomoohk
Copy link
Collaborator

moomoohk commented Dec 4, 2022

Yes indeed. Please start by merging master into your branch and take it from there.

@AlainLich
Copy link
Author

How do you suggest to proceed. I suppose the simpler is for me to pull the latest from branch master or any other branch you find more relevant and look at the changes. Then I will try to assess what needs to be changed to integrate the PR 'as is'.

Please advise.

@AlainLich
Copy link
Author

Hi, I have entered a new PR #177 with revised code integrated in the current commit for branch master.

This PR should be closed.

@moomoohk moomoohk closed this Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants