Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DCD XDR pickle tests #2911

Merged
merged 7 commits into from
Aug 17, 2020
Merged

Conversation

yuxuanzhuang
Copy link
Contributor

@yuxuanzhuang yuxuanzhuang commented Aug 15, 2020

Fixes #2878

Changes made in this Pull Request:


  • Tests?
  • Docs?
  • CHANGELOG updated?
  • Issue raised/referenced?

@pep8speaks
Copy link

pep8speaks commented Aug 15, 2020

Hello @yuxuanzhuang! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-08-16 14:55:01 UTC

@codecov
Copy link

codecov bot commented Aug 15, 2020

Codecov Report

Merging #2911 into develop will decrease coverage by 0.01%.
The diff coverage is 64.70%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2911      +/-   ##
===========================================
- Coverage    92.95%   92.94%   -0.02%     
===========================================
  Files          187      187              
  Lines        24579    24591      +12     
  Branches      3185     3185              
===========================================
+ Hits         22847    22855       +8     
- Misses        1686     1690       +4     
  Partials        46       46              
Impacted Files Coverage Δ
package/MDAnalysis/lib/formats/libmdaxdr.pyx 89.81% <62.50%> (-0.16%) ⬇️
package/MDAnalysis/lib/formats/libdcd.pyx 88.48% <66.66%> (-0.85%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22612de...5e9d11b. Read the comment docs.

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good that we had a look at this again. I think this looks right but please see comments. If it's the way that I think it is then we need to add a bit of documentation to the code and at least the DCDReader.

However, I am not really keen on making DCDFile.seek() behave differently from any other seek. Could we do the following: Make the second part of DCDFile.seek() starting at self.reached_eof = False a private method _seek() that only checks that its input frame is 0 <= frame <= self.n_frames. DCDFile.__setstate__ can call _seek() but DCDFile.seek() would check 0 <= frame < self.n_frames before calling _seek() and so continue to behave like all the other Reader.seek()` methods.

@@ -402,7 +401,7 @@ cdef class DCDFile:
seek the file to given frame (0-based)

"""
if frame >= self.n_frames:
if frame >= self.n_frames + 1:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to allow to seek to the end of the file? If so, add a comment, because under normal circumstances I expect a 0-based index for a sequence to fail when it hits len(sequence).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is possible to seek to the end of the file (after the last coordinate frame) with

dcd.seek(len(dcd))

then state it in the doc string. This is a fairly odd thing to allow so it should be documented.

if current_frame == self.offsets.size:
# cannot seek to self.offsets.size (like in DCD file)
# because here seeking depends on the offsets list size.
# Instead, we seek to one frame ahead and read the next frame.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean "previous frame" instead of "frame ahead"?

@@ -306,8 +306,14 @@ cdef class _XDRFile:

# where was I
current_frame = state[1]
self.seek(current_frame - 1)
self.current_frame = current_frame
if current_frame == self.offsets.size:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switch the decision in the if statement to make the more common situation come first

if current_frame < self.offsets.size:
     ...
elif current_frame == self.offsets.size:
    ...
else:
    #pragma: no cov
    raise RuntimeError("Invalid frame number {} > {} -- this should not happen.".format(current_frame, self.offsets.size)

and add defensive code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comment under which circumstances we have to do funny things – namely when we need to serialize at the end of the trajectory.

@orbeckst orbeckst added the GSoC GSoC project label Aug 16, 2020
@orbeckst orbeckst added this to the 2.0 milestone Aug 16, 2020
@yuxuanzhuang
Copy link
Contributor Author

I makes DCDFile to behave the same as XDRFile, i.e. not messing up with seek, but checking during __setstate__

@yuxuanzhuang
Copy link
Contributor Author

The pragma: no cover is not honered in cython files (cython/cython#3680)
Any reason we have to check for that here (aside from for the completeness), since it will be eventually checked in seek.

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a good solution, just doing the special casing inside __setstate__.

It's ok that "no cover" didn't work, thanks for finding out why.

@orbeckst orbeckst merged commit 23074c6 into MDAnalysis:develop Aug 17, 2020
PicoCentauri pushed a commit to PicoCentauri/mdanalysis that referenced this pull request Mar 30, 2021
- Fixes MDAnalysis#2878
- Changes made in this Pull Request:
    - add more comprehensive tests for pickling low-level dcd and xdr files
    - fix a bug that was introduced in MDAnalysis#2723 (wrong seeking in DCD and XDR, note: this bug was only in develop, never in released code, see discussion in PR MDAnalysis#2908)
- update CHANGELOG
- maybe backport in MDAnalysis#2908
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail to pickle last frame of DCD, XDR files
4 participants