deserialization error: Domains not defined

[vtjnash]

This text is repeatedly printed to the console after calling remote_call_fetch:
"deserialization error: Domains not defined"

I don't know exactly where this error comes from since it appears slowly and randomly after calling remote_call_fetch. Domains is a type that gets defined using @bcast load() on all nodes.

Where does this error get thrown? So how would I catch this error?

Strangely, it appears to work anyways, but I don't know what might have been lost, which is important in situations where I can recover from and repeat the action. Eventually too, Julia starts to throw more errors (could not connect to 18.68.147.198:9010, errno=113 and could not connect to 18.68.147.215:9009, errno=110) after several minutes and then afterwards I start seeing DisconnectException() and it crashes if I attempt to do anything parallel after that. But the commands that are still running are able to finish, so I still get typically get results. (I don't keep track of what fraction of the results that I eventually get)

Perhaps this is related to the other random failure I have seen where after calling @bcast load(), the contents of the file will still not be defined on the local system?

[JeffBezanson]

The connection errors are interesting; we seem to have

julia> strerror(110)
"Connection timed out"

julia> strerror(113)
"No route to host"

so maybe some of the needed connections are not happening.

Can this be reproduced with no network, say 2 local processes?

If you send me the code that causes this I can look deeper into it.

[vtjnash]

I doubt I could reproduce it locally, it seems to occur just with specific nodes, and only after some sort of timeout period has passed. I've stopped using some of the nodes and those connection issues seem to have ended.

I'm still getting the "deserialization error: Domains not defined" errors printing to the console. You can get my code at https://manwe.mit.edu/rhodecode/jqueens/files/7/jqueens.j However, it is probably not clear at all how to use it. I most likely need to create a reduced test case for this.

[StefanKarpinski]

Is that part of a parallel chess search program?

[JeffBezanson]

Just a quick note: you shouldn't use sleep since it will block the whole process. Instead there should be a RemoteRef you can wait on.

[vtjnash]

Can I wait simultaneously on any RemoteRef result? I only want results from the first one to finish, and then to kill the rest.

[JeffBezanson]

I'll consider this a general fault tolerance issue (#217).