Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ZeroTangent vs array of arrays tests #257

Merged
merged 1 commit into from
Jul 25, 2022
Merged

Add ZeroTangent vs array of arrays tests #257

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/check_result.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ for (T1, T2) in
end

test_approx(::AbstractZero, x, msg=""; kwargs...) = test_approx(zero(x), x, msg; kwargs...)
test_approx(::AbstractZero, x::AbstractArray{<:AbstractArray}, msg=""; kwargs...) = test_approx(map(zero, x), x, msg; kwargs...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a recursive definition such as

Suggested change
test_approx(::AbstractZero, x::AbstractArray{<:AbstractArray}, msg=""; kwargs...) = test_approx(map(zero, x), x, msg; kwargs...)
test_approx(x::AbstractZero, y::AbstractArray{<:AbstractArray}, msg=""; kwargs...) = all(yi -> test_approx(x, yi, msg; kwargs...), y)

would be easier as it would pick up test_approx definitions for the elements automatically, without having to redefine the array of arrays method?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that the test_approx does not return a boolean, but rather does the @test. So the suggestion above results in something like

julia> test_approx(ZeroTangent(), [[0, 0.0], [0.0, 0.1], [[0.0, 0.0], [0.0, 0.0]]])
ERROR: TypeError: non-boolean (Test.Pass) used in boolean context
Stacktrace:
 [1] _all(f::ChainRulesTestUtils.var"#113#114"{Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}, ZeroTangent, String}, itr::Vector{Vector}, #unused#::Colon)
   @ Base ./reduce.jl:1161
 [2] all(f::Function, a::Vector{Vector}; dims::Function)
   @ Base ./reducedim.jl:902
 [3] all(f::Function, a::Vector{Vector})
   @ Base ./reducedim.jl:902
 [4] test_approx(z::AbstractZero, x::AbstractArray{<:AbstractArray}, msg::Any; kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ ChainRulesTestUtils ~/JuliaEnvs/ChainRulesTestUtils.jl/src/check_result.jl:47
 [5] test_approx (repeats 2 times)
   @ ~/JuliaEnvs/ChainRulesTestUtils.jl/src/check_result.jl:47 [inlined]
 [6] top-level scope
   @ REPL[12]:1

The alternative is to create a new testset, something like:

function test_approx(z::AbstractZero, x::AbstractArray{<:AbstractArray}, msg=""; kwargs...)
    @testset "test_approx($(typeof(z)), $(typeof(x)))" begin
        for el in x
            test_approx(el, z, msg; kwargs...)
        end
    end
end

It's quite verbose, but probably closer to what we want. (Though in principle this could result in very nested testsets)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that the test_approx does not return a boolean, but rather does the @test.

Ah sorry, I missed that. Is a separate test set actually needed? It seems there are already some definitions such as https://github.com/alyst/ChainRulesTestUtils.jl/blob/5b9ec32a49a26984112edc6102488538d5efe2ae/src/check_result.jl#L99 that loop over elements without wrapping the individual tests in a test set. But maybe they should?

(As a side remark, it seems the definitions for NoTangent-Tangent are wrong as they return a boolean?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, correct on both points - I've opened a PR to fix, would be grateful if you could review? #259

test_approx(x, ::AbstractZero, msg=""; kwargs...) = test_approx(x, zero(x), msg; kwargs...)
test_approx(x::AbstractArray{<:AbstractArray}, ::AbstractZero, msg=""; kwargs...) = test_approx(x, map(zero, x), msg; kwargs...)
test_approx(x::ZeroTangent, y::ZeroTangent, msg=""; kwargs...) = @test true
test_approx(x::NoTangent, y::NoTangent, msg=""; kwargs...) = @test true

Expand Down
4 changes: 4 additions & 0 deletions test/check_result.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ end

test_approx([1.0, 2.0], [1.0, 2.0])
test_approx([[1.0], [2.0]], [[1.0], [2.0]])
test_approx([[0.0], [0.0]], ZeroTangent())
test_approx(ZeroTangent(), [[0.0], [0.0]])
test_approx(Broadcast.broadcasted(identity, [1.0 2.0; 3.0 4.0]), [1.0 2.0; 3.0 4.0])

test_approx(@thunk(10 * 0.1 * [[1.0], [2.0]]), [[1.0], [2.0]])
Expand Down Expand Up @@ -108,6 +110,8 @@ end

@test fails(() -> test_approx([1.0, 2.0], [1.0, 3.9]))
@test fails(() -> test_approx([[1.0], [2.0]], [[1.1], [2.0]]))
@test fails(() -> test_approx([[0.0], [0.1]], ZeroTangent()))
@test fails(() -> test_approx(ZeroTangent(), [[0.1], [0.0]]))

@test fails(() -> test_approx(@thunk(10 * [[1.0], [2.0]]), [[1.0], [2.0]]))

Expand Down