Implement getindex, firstindex, and lastindex for Generators

[uniment]

A generator like this:

(x^2 for x=(1,2,3))

should be indexable:

Base.getindex(g::Base.Generator, args...) = g.f(getindex(g.iter, args...))
Base.firstindex(g::Base.Generator, d...) = firstindex(g.iter, d...)
Base.lastindex(g::Base.Generator, d...) = lastindex(g.iter, d...)

especially considering that eachindex(::Generator) is implemented.

Currently, generators implement last but not lastindex:

julia> last(x^2 for x=(1,2,3))
9

julia> lastindex(x^2 for x=(1,2,3))
ERROR: MethodError: no method matching lastindex(::Base.Generator{Tuple{Int64, Int64, Int64}, var"#7#8"})

This contradicts the user manual:

Get the last element of an ordered collection, if it can be computed in O(1) time. This is accomplished by calling lastindex to get the last index.

So, to be consistent with eachindex, first, and last, generators should also implement getindex, firstindex, and lastindex.

[KristofferC]

eachindex works because keys is defined and this is now inconsistent as pointed out in #34678 (comment). I don't think the solution is to dump more index behavior into Generator though..

Also ref #36175

[uniment]

Reading through those, I see lots of support for this proposal. The motivation for pr #34678 is issue #34674, in which we find:

since generators are said to be the lazy variant of map and map preserves the indices it would be good to define that missing keys method.

The same argument applies here to getindex, firstindex, and lastindex (although I think it better to consider a generator as a lazy variant of a comprehension instead).

As you point out, in #34678 (comment) @tkf objected:

I'm against this patch unless values and getindex are defined for Generator

Despite that objection, #34678 was merged anyway; this proposal would help resolve that conflict. Further, in #36175, @tkf says:

I think it makes sense to define more methods for Base.Generator assuming the purity of the mapping function and then explain on which conditions certain interface can be used consistently.

This proposal is consistent with that intent, and we could probably implement values too to satisfy the original objection.

P.S. As seen in #48510, I think also eltype, adjoint, and permutedims could be implemented for generators too, but that will obviously be a different issue.

[uniment]

For multidimensional generators, it looks like indexing would need something like this too:

Base.getindex(itr::Iterators.ProductIterator, args...) = ntuple(length(args)) do i; itr.iterators[i][args[i]] end

[jishnub]

Is it possible to know that a generator doesn't have side effects? E.g, what about something like

julia> a = Int[];

julia> g = (push!(a, i) for i in 1:4)
Base.Generator{UnitRange{Int64}, var"#5#6"}(var"#5#6"(), 1:4)

With this,

julia> g[2]
1-element Vector{Int64}:
 2

julia> g[2]
2-element Vector{Int64}:
 2
 2

[uniment]

Note that we already observe this with functions with side-effects,

julia> a = Int[]
       g = (push!(a, i) for i=1:4)
       @show last(g)
       @show last(g);
last(g) = [4]
last(g) = [4, 4]

so this proposal adds no new surprise; I think the answer is, "know thy function and know thy iterator."