use pickle of ag to transport information #65
Conversation
| """helper function to actually setup dask graph""" | ||
| with timeit() as b_universe: | ||
| u = mda.Universe(top, traj) | ||
| agroups = [u.atoms[idx] for idx in indices] | ||
| agroups = [pickle.loads(idx) for idx in pickles] |
There was a problem hiding this comment.
E RuntimeError: Couldn't find a suitable Universe to unpickle AtomGroup onto with Universe hash '63ae64d5-94f2-42e5-ad84-cb7f89970d36'. Available hashes: 303e8959-6b17-49ee-9910-9528cf5ee0d5@richardjgowers the simple pickle solution doesn't work. The universe Hash values are different for some reason.
There was a problem hiding this comment.
@kain88-de it's not actually hashing the Universe, it's just creating a uuid when the Universe is created. So when you create a new Universe from the same files it "hashes" different.
There was a problem hiding this comment.
So I also need to pickle the universe?
|
or use the anchor thingy, it lets you define the hash
…On Sep 20, 2018, 9:04 AM -0500, Max Linke ***@***.***>, wrote:
@kain88-de commented on this pull request.
In pmda/parallel.py:
> """helper function to actually setup dask graph"""
with timeit() as b_universe:
u = mda.Universe(top, traj)
- agroups = [u.atoms[idx] for idx in indices]
+ agroups = [pickle.loads(idx) for idx in pickles]
So I also need to pickle the universe?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I have no idea how to use the anchor thingy. The documentation isn't clear on this. |
|
So I cannot pickle a Universe object because the |
kain88-de
force-pushed
the
update-ag
branch
from
61df0c9 to
df3a7f4
Compare
|
Anchor thingy works about 90% of the time to unpickle an atomgroup. Unfortunately, the results are really off now. Did I make a mistake in using the anchors? I guessed my usage from the code in MDAnalysis. Also this looks like an error in MDAnalysis. RuntimeError: Couldn't find a suitable Universe to unpickle AtomGroup onto with Universe hash 'c66d4379-9abc-46d6-91da-b31dc1a37af0'. Available hashes: c66d4379-9abc-46d6-91da-b31dc1a37af0 |
|
@kain88-de yeah that looks right. The tests look like they're failing because the trajectory isn't being used That second one looks like a bug, maybe it's a bytes/str difference? |
|
what do you mean with trajectory not being used?
about bytes/strings I only use the MDAnalysis functions. But the python 2.7
test fail with the same error message. So maybe something more fundamental.
…On Thu, Sep 20, 2018 at 4:54 PM Richard Gowers ***@***.***> wrote:
@kain88-de <https://github.com/kain88-de> yeah that looks right. The
tests look like they're failing because the trajectory isn't being used
That second one looks like a bug, maybe it's a bytes/str difference?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#65 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEGnVodlFHaBGVJwjwgx4p-PMYuUPwjFks5uc6wzgaJpZM4Wxygc>
.
|
|
The contacts is coming out as ones, which (I’m guessing) means the same
frame is being used for all frames, so the contact is constant
On Thu, Sep 20, 2018 at 10:02 AM, Max Linke <notifications@github.com>
wrote:
… what do you mean with trajectory not being used?
about bytes/strings I only use the MDAnalysis functions. But the python 2.7
test fail with the same error message. So maybe something more fundamental.
On Thu, Sep 20, 2018 at 4:54 PM Richard Gowers ***@***.***>
wrote:
> @kain88-de <https://github.com/kain88-de> yeah that looks right. The
> tests look like they're failing because the trajectory isn't being used
>
> That second one looks like a bug, maybe it's a bytes/str difference?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#65 (comment)>, or
mute
> the thread
> <
https://github.com/notifications/unsubscribe-auth/AEGnVodlFHaBGVJwjwgx4p-PMYuUPwjFks5uc6wzgaJpZM4Wxygc
>
> .
>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#65 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AI0jByOjgSd3U_qWyj9fizu2DkZ7o0Skks5uc64VgaJpZM4Wxygc>
.
|
|
Yeah but I don't understand how this would happen. In the old code where I use the indices of the atomgroups everything works and the universe is iterated as expected. Using the anchors and pickling of atomgroups it fails. I don't see how the atomgroups are not linked to the trajectory. |
This makes functions unpacking this tuple easier to read.
|
looks like my rebase broke something |
|
@kain88-de I'm going to catch up with people at the workshop to understand how dask works here, so hopefully I'll understand this enough to help out.. |
Fixes #
Changes made in this Pull Request:
PR Checklist