Use newlines and indents to make large, complex structs more readable when printed

[BioTurboNick]

As shown in #43183 , printing of structs can get quickly out of hand. While there are nice solutions to better display structs, as noted there, they may be a bit too heavy for Base, or too similar to dump.

This PR uses newlines and indents to add just a bit of visual structure to complex structs, so they can remain compact but more human-readable.

The rule in use is that if a struct contains other structs, or if the type as printed takes a large fraction of the displaysize() columns, it gets its own line. Simple structs which when printed do not take up many columns are printed inline as before.

struct ComplexStruct2
       a::Vector{Int}
       b::Union{ComplexStruct2, Nothing}
       c::ComplexF64
       d::Bool
       e::Method
       f::Int
       g::String
       h::Bool
end

ComplexStruct2(x) = ComplexStruct2(collect(1:50), x, ComplexF64(3, 6), true, methods(read)[1], 5, "test", false)

ComplexStruct2(ComplexStruct2(ComplexStruct2(nothing)))
#=
Currently:

ComplexStruct2([1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  41, 42, 43, 44, 45, 46, 47, 48, 49, 50], ComplexStruct2([1, 2, 3, 4, 5,
6, 7, 8, 9, 10  …  41, 42, 43, 44, 45, 46, 47, 48, 49, 50], ComplexStruct2([1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  41, 42,
43, 44, 45, 46, 47, 48, 49, 50], nothing, 3.0 + 6.0im, true, read(::Union{Base.DevNull, Core.CoreSTDERR, Core.CoreSTDOUT
}, ::Type{UInt8}) in Base at coreio.jl:22, 5, "test", false), 3.0 + 6.0im, true, read(::Union{Base.DevNull, Core.CoreSTD
ERR, Core.CoreSTDOUT}, ::Type{UInt8}) in Base at coreio.jl:22, 5, "test", false), 3.0 + 6.0im, true, read(::Union{Base.D
evNull, Core.CoreSTDERR, Core.CoreSTDOUT}, ::Type{UInt8}) in Base at coreio.jl:22, 5, "test", false)



In this PR:

ComplexStruct2(
    [1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  41, 42, 43, 44, 45, 46, 47, 48, 49, 50],
    ComplexStruct2(
        [1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  41, 42, 43, 44, 45, 46, 47, 48, 49, 50],
        ComplexStruct2(
            [1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  41, 42, 43, 44, 45, 46, 47, 48, 49, 50],
            nothing, 3.0 + 6.0im, true,
            read(::Union{Base.DevNull, Core.CoreSTDERR, Core.CoreSTDOUT}, ::Type{UInt8}) in Base at coreio.jl:22,
            5, "test", false),
        3.0 + 6.0im, true,
        read(::Union{Base.DevNull, Core.CoreSTDERR, Core.CoreSTDOUT}, ::Type{UInt8}) in Base at coreio.jl:22,
        5, "test", false),
    3.0 + 6.0im, true,
    read(::Union{Base.DevNull, Core.CoreSTDERR, Core.CoreSTDOUT}, ::Type{UInt8}) in Base at coreio.jl:22,
    5, "test", false)
=#

[mcabbott]

This would be great.

What your example doesn't show is nested structs which are parameterised by their contents, as they often are. This often results in the types being about as long as the contents. Should they too be wrapped somehow? Or otherwise distinguished from the contents, perhaps by changing the colour?

julia> struct Mine{A,B} a::A; b::B; end

julia> x = reshape(0:0.2:1, 2,3)'; Mine((1,2,x), Mine(3,(4,x)))  # before
Mine{Tuple{Int64, Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}, Mine{Int64, Tuple{Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}}}((1, 2, [0.0 0.2; 0.4 0.6; 0.8 1.0]), Mine{Int64, Tuple{Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}}(3, (4, [0.0 0.2; 0.4 0.6; 0.8 1.0])))

julia> x = reshape(0:0.2:1, 2,3)'; Mine((1,2,x), Mine(3,(4,x)))  # after
Mine{Tuple{Int64, Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}, Mine{Int64, Tuple{Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}}}(
    (1, 2, [0.0 0.2; 0.4 0.6; 0.8 1.0]), 
    Mine{Int64, Tuple{Int64, Adjoint{Float64, Base.ReshapedArray{Float64, 2, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Tuple{}}}}}(3, 
        (4, [0.0 0.2; 0.4 0.6; 0.8 1.0])))

(Note also that in this example, 3 is pushed by the type, and perhaps would be better on a new line?)

[BioTurboNick]

Seems consistent to adopt the dimmed color from stacktraces

[stevengj]

Seems like this should only be used for the 3-argument show(::IO, ::MIME"text/plain", x).

[BioTurboNick]

Seems like this should only be used for the 3-argument show(::IO, ::MIME"text/plain", x).

Can you say a bit more, @stevengj ? If I understand right, this should only be called if no specialized method is defined or provided. Or is there an example I can test to make sure it doesn't occur in the situation you're envisioning?

[BioTurboNick]

New state for your example @mcabbott :

The 3 is on its own line, and I implemented a gray type list.

[stevengj]

As explained here you should generally define a 3-argument show method for multi-line output, and a 2-argument show method for single-line compact output.

That is, the old 2-argument show method should stay as-is, and the new multi-line output should only be for the 3-argument show(io, "text/plain", x).

[BioTurboNick]

As explained here you should generally define a 3-argument show method for multi-line output, and a 2-argument show method for single-line compact output.

That is, the old 2-argument show method should stay as-is, and the new multi-line output should only be for the 3-argument show(io, "text/plain", x).

Sure, but I'm not defining any show method, I'm just changing the show_default method already being used for a default display. What change are you suggesting?

What I'm asking is, in what situation would this not work, such that it has to be done differently?

[mcabbott]

I think that the default at present only has one path, both for display and inline printing. If I understood right, the suggestion is that an upgrade like this should make it take two paths, so that the multi-line unfolded printing is only triggered when the struct is shown at top level, like for Array.

How much code these two paths can share I don't know.

and I implemented a gray type list.

Cool. How hard would it be to delay the grey by one level, so that some type parameters are shown?

[BioTurboNick]

I think that the default at present only has one path, both for display and inline printing. If I understood right, the suggestion is that an upgrade like this should make it take two paths, so that the multi-line unfolded printing is only triggered when the struct is shown at top level, like for Array.

How much code these two paths can share I don't know.

Hmm... so show(io::IO, ::MIME"text/plain", x) = show(io, x) is already defined in multimedia.jl, and if I overwrite that, things start behaving unexpectedly.

The show_default path relies on the two-argument form downstream.

So... possible someone who knows the printing system better needs to help untangle this?

and I implemented a gray type list.

Cool. How hard would it be to delay the grey by one level, so that some type parameters are shown?

Moderate. One issue here is that the printing of types in stack traces is entirely separate logic. It's easier there to break down the parts; here, it's just calling show on the data type, which recursively jumps to the special methods and back to show. Currently, what I'm doing is printing it all to a separate buffer and then printing the composed buffer into the output IO stream with color. Maybe with IOContext information could be passed down, and then wrapped at the appropriate point?

[BioTurboNick]

Ah, just added detection of :compact == true, so that it displays entirely inline again where that's specified. So arrays look messy still, but not sprawling over too many lines.

And unfortunately the gray only applies to the default display - it wouldn't impact any custom type displays with type parameters.

[stevengj]

Oh, I see... yes, unfortunately it seems quite hard to use this only for 3-argument show because of the way the dispatch tree works here.

[BioTurboNick]

Just revisiting this - So it seems like it shouldn't be the default, but perhaps as an optional path it could work? So that on-demand a struct could be printed in a slightly nicer way.

What would the best way to do that be?

[vtjnash]

I think https://github.com/thautwarm/PrettyPrint.jl attempts to address this? I am unsure how it deals with the dispatch question. In theory we could have something like if whichapplicable(show, args...) == which(show, (IO, Any)) to detect whether it will call the fallback, and if so, to instead call the 3-arg show recursively.

But I think in this case, it may selectable with a boolean flag to the function, so let me try to write inline comments to suggest how to do that

[vtjnash]

Ah, sorry, I mixed myself up. I think what is needed is to change the existing mime-show fallback method with this:

show(io::IO, ::MIME"text/plain", x::Any) =
    showdefault(io, x,
         which(show, (typeof(IO), typeof(x))) == which(show, (IO, Any)))

and add an optional boolean parameter to showdefault which sets whether to print "pretty", recursively, or not

[BioTurboNick]

I checked out PrettyPrint.jl, and it's doing a bit too much IMO (see image). It's choosing to fill the screen vertically to show whole arrays (no truncation), and it failed on an UndefRefError with my example in the OP.

I was just looking to add minimal extra structure to the standard printing, which I was otherwise injecting manually. Just enough to make it easier to read and compare complex nested types.

[vtjnash]

Yes, take a look at the which calls in my last message for how to deal with the "is this overloaded" question