Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recursive as dictionary #1643

Merged
merged 7 commits into from
Jul 18, 2019
Merged

Recursive as dictionary #1643

merged 7 commits into from
Jul 18, 2019

Conversation

amarkpayne
Copy link
Member

Motivation or Problem

The methods as_dict and make_object allow for any object that inherits from RMGObject to be easily saved to a yaml file and reconstructed from a yaml file, respectively. However, as currently written is RMG there is a limited recursion depth to how many objects, dictionaries, and lists can be nested inside of each other.

For example, consider an object that has an attribute that is a dictionary which itself contains another object. The desired affect of as_dict would be for the object nested inside of this dictionary to be expanded to a dictionary (i.e. for as_dict to be called on this object as well). However, the current implementation will return for this attribute a dictionary that contains the object itself rather than the expanded object.

Furthermore, many objects have to rewrite their as_dict and make_objects to function properly, even if they inherit from RMGObject. Ideally, newly created objects that inherit from RMGObject should have working as_dict and make_object methods without needing to overwrite these methods (at least more times than not).

Description of Changes

Inside the file where we define the RMGObject I have created method for recursively calling as_dict for all objects inside of a dictionary or list. I have also created a method for recursively calling make_object for all objects inside of a dictionary. Becasue these functions are recursive by calling themselves, the recursion limit for nested objects and iterables should be whatever the recursion stack limit of python is (very large).

While these functions can be called as standalone functions (potentially useful for scripting), I have used these functions to help re-write the as_dict and make_object methods of the RMGObject class.

With this, I have also cleaned-up the affected code, and made it so that the as_dict and make_object methods are only re-written when absolutely necessary (for example, see statmech rotations and torsions), which thanks to the new functions require very little code even when necessary. Note that this is usually necessary whenever passing all attributes to the init method causes issues.

Testing

I have ran the unit tests, and even successfully recreated an ArkaneSpecies object from a YAML file, but it is likely that there are affected classes that weren't properly unit tested. I am specifically concerned about the Wilhoit class, and the NASA class can probably be cleaned up further.

Reviewer Note

Because we have a few pull requests open or about to be opened that define new objects that will be saved to a YAML file, these PRs will require these changes, and thus this PR needs to be merged in first. For example, this affects #1577, #1641, and #1561

Copy link
Member

@alongd alongd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this additions! Below are some comments and questions

rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Show resolved Hide resolved
rmgpy/rmgobject.pyx Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
@rwest
Copy link
Member

rwest commented Jun 26, 2019

Not had chance to review the code, but from the pull request description this sounds like a great improvement. 👏🏻

@amarkpayne
Copy link
Member Author

Thanks @alongd for the timely review and great comments! I have addressed your comments and added fixup commits to add changes where I could. Let me know what you think. In the meantime I think it might be worthwhile if I add some additional unit testing to catch some of the points that you addressed.

@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 4aabaf8 to 9d6229e Compare June 28, 2019 21:24
@codecov
Copy link

codecov bot commented Jul 1, 2019

Codecov Report

Merging #1643 into master will increase coverage by 0.02%.
The diff coverage is 0%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1643      +/-   ##
==========================================
+ Coverage   41.68%   41.71%   +0.02%     
==========================================
  Files         176      176              
  Lines       29374    29359      -15     
  Branches     6059     6053       -6     
==========================================
  Hits        12246    12246              
+ Misses      16259    16244      -15     
  Partials      869      869
Impacted Files Coverage Δ
rmgpy/quantity.py 0% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c1f46c...13b1cb6. Read the comment docs.

@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 579ba53 to 228b502 Compare July 11, 2019 15:04
@amarkpayne
Copy link
Member Author

@alongd I have applied a few of the changes that we discussed offline and also added some unit tests. I also further refactored quantity, NASA/NASAPolynomial, and Wilhoit. With this I have added everything that I wanted to add for this PR, so this should be ready for one last round of review (and final changes if necessary).

@amarkpayne
Copy link
Member Author

As we discussed offline, the fixup commits that I squashed are on the branch rad_w_fixup on official

Copy link
Member

@alongd alongd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these recent additions!
Please see some comments below

rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
rmgpy/quantity.py Show resolved Hide resolved
rmgpy/statmech/rotation.pyx Show resolved Hide resolved
Copy link
Member Author

@amarkpayne amarkpayne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @alongd and @mliu49 for the comments, I'll push the changes soon.

rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Show resolved Hide resolved
rmgpy/rmgobject.pyx Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
rmgpy/statmech/rotation.pyx Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
@amarkpayne
Copy link
Member Author

I have added the last of the changes that @mliu49 and I discussed here and offline. @alongd let me know if there are any other changes. Otherwise if you approve I will update the branch and squash the fixup commits in preparation for merging.

Copy link
Contributor

@mliu49 mliu49 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fixups look good and can be squashed. I made a couple minor formatting comments, which you can go ahead and squash as well if you choose to fix them.

arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
rmgpy/statmech/torsion.pyx Outdated Show resolved Hide resolved
@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 0a2a4c0 to 50736d7 Compare July 16, 2019 01:10
@amarkpayne
Copy link
Member Author

I added your formatting changes and squashed all of the fixup commits. Thanks again @mliu49 !

@amarkpayne
Copy link
Member Author

The builds are getting errors related to cairo. It looks like this package was updated on the RMG channel today. @mliu49 any idea if something went wrong with the new build?

@mliu49
Copy link
Contributor

mliu49 commented Jul 16, 2019

Sorry, I was playing around with updating a couple dependencies yesterday. The newest noarch cairocffi build by conda-forge apparently works on mac but not linux. The linux-64 build works though, so I just removed the noarch build.

@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 50736d7 to 2ba4c73 Compare July 16, 2019 14:59
@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch 2 times, most recently from b42180a to 1abb665 Compare July 17, 2019 16:26
Copy link
Member

@alongd alongd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the tests!!

arkane/common.py Show resolved Hide resolved
arkane/common.py Show resolved Hide resolved
arkane/common.py Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
rmgpy/rmgobject.pyx Outdated Show resolved Hide resolved
arkane/commonTest.py Show resolved Hide resolved
@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 1abb665 to 534ac86 Compare July 17, 2019 16:55
Should work no matter how many objects or dictionaries or lists are nested in each other. Most objects that inherits from RMGObject should work by default without  having to overwrite the make_object method
@amarkpayne amarkpayne force-pushed the recursive_as_dictionary branch from 534ac86 to 13b1cb6 Compare July 17, 2019 17:15
@amarkpayne
Copy link
Member Author

The last round of fixes/additions has been rebased/autosquashed. Let me know if this is ready to merge or if there are any other changes we missed.

Copy link
Member

@alongd alongd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this awesome contribution!

@alongd alongd merged commit 2a4bef9 into master Jul 18, 2019
@alongd alongd deleted the recursive_as_dictionary branch July 18, 2019 14:58
@amarkpayne
Copy link
Member Author

Thanks @alongd and @mliu49 for all of the hard work you put into this PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants