Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loosen signature of read! to accept views? #33035

Closed
giordano opened this issue Aug 22, 2019 · 1 comment · Fixed by #33046
Closed

Loosen signature of read! to accept views? #33035

giordano opened this issue Aug 22, 2019 · 1 comment · Fixed by #33046

Comments

@giordano
Copy link
Contributor

On slack it was pointed out that you can't write a binary file to a slice of an Array:

julia> x = rand(3, 3)
3×3 Array{Float64,2}:
 0.899822  0.0439746   0.961134
 0.748855  0.0687037   0.238223
 0.774079  0.00639737  0.224812

julia> open("test.bin", "w") do io
           write(io, [1.0, 2.0, 3.0])
       end;

julia> open("test.bin", "r") do io
           read!(io, x[:,3])
       end
3-element Array{Float64,1}:
 1.0
 2.0
 3.0

julia> x
3×3 Array{Float64,2}:
 0.899822  0.0439746   0.961134
 0.748855  0.0687037   0.238223
 0.774079  0.00639737  0.224812

That's of course because the slice creates a copy, so it would be great if we could call read!(io, @view x[:,3]), but the signature of read! doesn't accept it.

Would it make sense to loosen the signature of read! to be something like

read!(s::IO, a::Union{Array{T}, SubArray{T,N,<:Array{T}} where N}) where T

? In this way we could do:

julia> open("test.bin", "r") do io
           read!(io, @view x[:,3])
       end
3-element view(::Array{Float64,2}, :, 3) with eltype Float64:
 1.0
 2.0
 3.0

julia> x
3×3 Array{Float64,2}:
 0.899822  0.0439746   1.0
 0.748855  0.0687037   2.0
 0.774079  0.00639737  3.0

Are there perhaps other functions that could benefit from similar loosening?

@tlnagy
Copy link
Contributor

tlnagy commented Sep 13, 2019

Glad to see this brought up again. I ran into this about 2 years ago: https://discourse.julialang.org/t/efficient-way-to-read-large-array-from-binary-file-in-slices/6847

There's also this issue: #32524

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants