RFC: replace all redirection operators with x |> redirect(y)

[JeffBezanson]

Using |> to chain function calls has become popular. There is also dissatisfaction with the use of .>, >>, and .>> for redirecting I/O. This change addresses that by proposing to use |> only for function calls, and replacing all redirection operators with the idiom x |> redirect(y).

The introduction of call overloading makes x |> y potentially confusing, since it's not clear whether y is a functor and we want y(x), or we are calling an overload of |>. For the same reason, call overloading makes a separate |> operator unnecessary, since if you really want x |> y to work you can define call for y.

Some examples:

`command` |> redirect("out.txt",stderr="errs.txt")

`command` |> redirect("out.txt") |> redirect(stderr="errs.txt",append=true)

Of course, this generalizes well too, since it's easy to add more options to redirect.

(ref #5349)

[JeffBezanson]

For more brevity, we could call this to instead, as in foo |> to("file").

[JeffBezanson]

The problem is that right now you can't predict when it will work to rewrite y(x) to x |> y, or vice versa. I also don't think we want to make strings "callable" so that one can redirect to a file using cmd |> "file". I would just as soon have everybody stop writing function calls with |>, but I know that isn't going to happen at this point. Piping mania has fully taken hold.

[StefanKarpinski]

Personally I dislike that too and would be just as happy to find another way for people to write pipelines.

[JeffBezanson]

I'm a bit skeptical of how important it is to have nice overly-fancy syntax for pipelines of processes. This seems fine to me:

pipe(`command`, "out.txt", stderr="errs.txt")

It's actually shorter than my example above, even without renaming redirect to pipe.

[StefanKarpinski]

I far prefer that, actually. It would also be possible to add pipe syntax inside of backticks, which would be more user-friendly anyway.

[StefanKarpinski]

How about instead of this redirect business we just have a pipe function that has stdin, stdout and stderr keyword arguments that take string and/or IO objects? It can take one or more command objects as arguments, and maybe allow file names or IO objects at the start and end that are implicitly hooked up to the stdin and stdout? It would kind of be the swiss-army knife function for creating pipelines.

[JeffBezanson]

Yes, that's pretty much what I was getting at. So far I think that's the best approach.

[StefanKarpinski]

Getting rid of the f |> x syntax can be a separate push ;-)

[JeffBezanson]

Ok, I tried this and I like it even more than I thought I would. It is just so simple and regular. Multi-step pipelines are bit awkward, but overall this is great.

[StefanKarpinski]

Yes, much better – the multistep case seems like it could be handled by a n-arg version of pipe where n is larger than 2.

[JeffBezanson]

+1

[tkelman]

Yes please. Can this be compat-ed easily?

[JeffBezanson]

Yes, very easily. I'll add deprecations.
Changing |> to function call syntax is hard to deprecate, but that's a separate change.

[StefanKarpinski]

Changing |> to function call syntax is hard to deprecate, but that's a separate change.

Can we please hold off on that? I'm not at all convinced that it's a good idea. I just don't feel like we're getting enough out of that syntax to do this.

[JeffBezanson]

Everybody ok to merge this?

[StefanKarpinski]

DO IT

[tkelman]

that travis failure's sorta scary but pretty sure it's unrelated https://travis-ci.org/JuliaLang/julia/jobs/51291717

[andrewcooke]

so |> is going to disappear as a syntactical element that can be overloaded?

i ask because it's used in ParserCombinator.jl. here's an example grammar:

sum = Delayed()
val = S"(" + sum + S")" | PFloat64()

neg = Delayed()             # allow multiple negations (eg ---3)
neg.matcher = val | (S"-" + neg > Neg)

mul = S"*" + neg
div = S"/" + neg > Inv
prd = neg + (mul | div)[0:end] |> Prd

add = S"+" + prd
sub = S"-" + prd > Neg
sum.matcher = prd + (add | sub)[0:end] |> Sum

all = sum + Eos()

[StefanKarpinski]

Let's not have that discussion on a long-merged-and-dead pull request.

[andrewcooke]

then please could you reply on the julia-users thread where you previously didn't address it?

[JeffBezanson]

See also #5571 #4963