expansion of singleton arguments with map/pmap

[ghost]

In some cases, map will "expand" singleton arguments, which is great:

julia> map(+,{1,1,1},{2,2,2})
3-element Any Array:
 3
 3
 3

julia> map(+,{1,1,1},2)
3-element Int64 Array:
 3
 3
 3

However, this doesn't work very systematically:

julia> map(+,{1,1,1},{2,2,2},{3,3,3})
3-element Any Array:
 6
 6
 6

julia> map(+,{1,1,1},2,{3,3,3})
no method map(Function,Array{Any,1},Int64,Array{Any,1})
 in method_missing at base.jl:70

julia> map(+,{1,1,1},{2,2,2},3)
no method map(Function,Array{Any,1},Array{Any,1},Int64)
 in method_missing at base.jl:70

And, with pmap, it doesn't seem to work at all:

julia> pmap(+,{1,1,1},{2,2,2})
3-element Any Array:
 3
 3
 3

julia> pmap(+,{1,1,1},2)
exception on 1: no method ref(Int64,Int64)
exception on 1: no method ref(Int64,Int64)
exception on 1: no method ref(Int64,Int64)
3-element Any Array:
 #undef
 #undef
 #undef

[JeffBezanson]

I think it would be better if map didn't do any kind of automatic expansion, since the arguments might be any containers and then it might not be clear what is and isn't being expanded. Can be done instead with map(x->x+2, [1,2,3]).

[StefanKarpinski]

I agree. The expansion thing in map/pmap seems like a misfeature to me.

[ghost]

I see the logic behind keeping this out of map. However, the realistic use case for doing this would be a large-scale optimization where one wants to map some function over a dataset, but needs to pass a structure of constants to each function evaluation as well. Setting up to explicitly copy that structure to each child process is somewhat tiring. I've tried using closures, but that seems to cause chaos with serialization for pmap... Workarounds are easy for using map without expansion-- pmap not so much because of serialization.

It seems like iterators would be another nice solution here... Do map/pmap support those?

[JeffBezanson]

Serializing closures is supposed to work; we do it all the time. It would be great if you could write up the case that caused chaos as an issue.

[ghost]

OK, will do. It is tricky to get a simple self-contained example, but I'll try to trace where the problems are coming from.