RFC: recshow

[toivoh]

A first attempt at a show function that works for self-referential data, per this discussion:
https://groups.google.com/forum/#!topic/julia-dev/qdHQYQb-0XQ

It should work for sets, dicts, composite types that use the default show,
and other types that call rshow rather than show within their own show methods.
I wasn't sure how to get it to work with arrays, since they seem to use some sprintf and showcompact trickery to show themselves.

I've replaced most calls to show within base/, on the premise that rshow should be the user facing function. All of these might not be strictly necessary, but they don't add any overhead to normal shows either.

There's probably still some open issues:

• Presentation
• Is it possible to make recshow print incrementally
  rather than consuming the whole object first?
• Names: show, rshow, recshow
• Array printing
• ...

Any feedback appreciated!

[toivoh]

Btw, is there a good way to include a test/example script with the pull request?

[toivoh]

Also, I should probably make show_composite(io, x) into a staged function, if those are still alive and well.
Good/bad idea?

[pao]

If its an actual test, add it to the test suite; for an example, I'd pop it in a gist and link that here.

[toivoh]

Ok, here's a small example then: https://gist.github.com/3304162

[StefanKarpinski]

Here's an interesting idea, regarding showing recursive data structures. If we've already seen something, then there's an expression for it. So we can use that expression as a representation of the thing itself. Example:

julia> x = {};

julia> push(x,x);

julia> x
{x}

This is a hypothetical example — but it would be an ideal way to show this. Even if the shown expression is something fairly complex, this would be possible because Julia is fully expression-oriented, so you can always take an expression that evaluates to something and place it somewhere else. So this would also work:

julia> x[1]
{x[1]}

Of course, this could be better written as {x}, but that would be much harder to implement, requiring a global search of things that are equal to some value — basically the sweep phase of a mark-and-sweep garbage collection, looking for the simplest way to express a specific object. The bigest hurdle to implementing this is the fact that indexing is user-defined, so you can't generally know that writing x[1] is the way to get at the first item in x.

[StefanKarpinski]

Also, this approach would clearly require access to the evaluated expression in the repl. That can certainly be arranged, however.

[toivoh]

I'm not sure that would suit my purposes, which is typically to inspect programmatically generated graphs from the REPL.

Also, I'd be a lot more comfortable with a general purpose show function that produces a self-contained result.
What if the show result is not going to the REPL, but to a log, or some other destination?

There could be a use for the kind of thing you're describing, for in-REPL experimentation. But then I think it should limit itself to naming objects by global variables that refer to them, so that you can type in the variable and see what it contains.
Still, there will be cases when it's necessary to name an object that has never appeared in the REPL,
so I think any recshow function must be able to refer within it's printout to other parts in the same printout.

[ViralBShah]

What is the latest on this? We certainly need a better way to show self-referential data, but what approach should we take?

[toivoh]

I'm not sure that there's been any progress on this.
I think that the major design issue is that it's impossible to achieve simultaneously all of the six desirable features below:

1. Each (mutable) object printed only once
2. Each (mutable) object that appears in several places referred to by a common name
3. Named objects named when they are printed
4. A nice, readable printout, that isn't cluttered with unnecessary names
5. Incremental printing: Start printing before the entire graph has been visited
6. Not going back to erase old printouts/reprinting a fresh one

Basically, this requires to know for each object whether it should be named before it is printed,
which requires to know whether it is referred to by more than one object,
which requires to visit the entire graph first.

We'll need to give up on at least one of these. This pull request gives up number 5.
I'd really like to see a discussion on this design issue.

Besides that, there are a number of decisions to be made (partially dependent on the first answer)

• What should a printout that contains shared (named) objects look like?
• How should this be integrated with the current system for show etc?

[toivoh]

Jumping ahead a little. As for the visual format, how about something like

julia> a = {}; push(a,a)
(a = {a}; a)

When the graph is acyclic, the output should be able to evaluate to something like the original, just as I think show aims to do now:

julia> a={}; {a,a}
(a={}; {a,a})

(Note: The examples above are mockups, not real julia output)

[ViralBShah]

What are we going to do about this? This pull request has been around for 7 months.

[StefanKarpinski]

I was originally very gung-ho about showing everything in a form that would evaluate back to the original version of something, but we ended up not doing that in a lot of places (e.g. displaying arrays), and I think it's better not to. So given that we don't generally do that, it seems fair to give it up. Giving up on property #5 and traversing an entire data structure before printing is probably the best way to go here. Unfortunately, this pull request is now rather old and bringing it up-to-date is likely a fair amount of work (even checking it out and building the state it was once in will take a while, although I might do it on julia.mit.edu with 60 cores or something). @tovioh, can you include an example of the output this produced?

[toivoh]

@StefanKarpinski: I agree that giving up on number 5 seems like the best way out.
About the output: is https://gist.github.com/toivoh/3304162 enough?

We should probably have a fresh discussion about the output format. It might very well be that it's easiest to start the implementation afresh as well. Unfortunately, I won't have much time for this for a good while.

[StefanKarpinski]

Ok, so it seems like we should close this pull-request in favor of a future one when you get the chance. No worries about the time, but I'm looking forward to it when you do get a chance.

[toivoh]

Ok, sounds fine!

[StefanKarpinski]

Closing to be resumed later.