How to use rich by redirecting stdout and stderr?

[ssbarnea]

I think it would prove very useful to have an example on how someone could start using rich Console without having to convert all his print() calls into console.print() ones.

I suspect that this has something to do with https://github.com/willmcgugan/rich/blob/36efcb5abe9ea8b6a7707243bec89a81e063c01a/rich/progress.py#L476-L509 but is not fully clear to me how this could be achieved as a generic solution, especially as this _FileProxy appears to be a private class.

Apparently the code below seems to be working but is has one serious downside: it uses a private class _FileProxy, which makes it unsafe to use as it would likely break on new releases:

from rich.console import Console
from rich.progress import _FileProxy

console = Console(record=True)
from rich.progress import _FileProxy
print("foo")  # <-- on purpose not using console.print()
assert foo in console.export_text()

I wonder if it would be possible to either make _FileProxy public, or even add a redirect=False option to Console constructor that does the redirection for us.

If user wants to also to the same for stderr, they would have to create a new console for stderr, something that is already the current way to deal with both streams.

[willmcgugan]

You could define a print function with the same signature as the builtin print, just like rich.print. Then inject that in to builtins.

import builtins
builtin.print = my_print

Or you could use the _FileProxy route. Since its private, it would be wise to copy it to your project.

[ssbarnea]

Based on my initial testing with builtins alteration it appears to be less reliable than alternating sys.stdout as I seen some python libraries being not using my proxy.

It may have something to so with the import order and in some cases I may not be able to do the swap before they are imported. I am still doing more tests in this area.

Altering sys.stdout directly has other benefits too, as some may decided not to use print() and just write to the stream.

BTW, I should tell you that I really love the record feature as it enables anyone to easily implement an --html-report feature for their command line tools.

[ssbarnea]

I think I may have detected an issue with export_html() which seems to be missing content written to the stream directly, which export_text() seems to be working file.

I wrote a small test around this at https://github.com/pycontribs/subprocess-tee/pull/13/checks?check_run_id=1336657368

ConsoleEx is effectively just using the FileProxy approach around sys.stdout/sys.stderr, as seen at https://github.com/pycontribs/subprocess-tee/blob/main/lib/subprocess_tee/rich.py#L58-L59

Sadly what I was looking for was the html export bit... I wonder if I did something wrong or there is a bug in rich.

[willmcgugan]

It's because export_text and export_html clears the internal buffer, unless you set clear=False. See reference

[ssbarnea]

Thanks that sorted the dilemma of missing output from HTML export. Now the only issue is that the HTML output does not render the ANSI output from the sys.stdout, the result is very different from what happens on console:

What happens on terminal:

What gets recorded as HTML

My guess is that sending data to the std stream bypasses rich ANSI filter. As I do happen to run subprocesses that produce a huge amount of output and the whole point was to transform it to html.

I am still looking into it but if you have a hint it would greatly be appreciated.

[willmcgugan]

The best solution would be to parse the strings in to a Text instance, with the ansi codes translated to styled spans. That would work for the most part, but you might also have to strip out more exotic ansi codes, and carriage return etc. It's a non-trivial task I'm afraid.

Second best solution (and easiest) would be to strip out all ansi codes. You won't get color and style, but at least you won't see garbage text. I see you have a draft PR for that.

[ssbarnea]

Yep. I went for the bandaid (stripping) as this was easy to implement. I do plan to do a proper conversion but as you said, it is not so easy.

I recently became co-maintainer of ansi2html package, which IMHO is the only python package that can do this as I used it in pytest-html. Still, this package is known to have some licensing limitations (used to be GPL, become LGPL but this still is not as liberal as MIT). It would likely be easier to reuse it if you want as the effort to implement ansi conversion would be considerably lower.

For example inside pytest-html (MIT) we avoided adding ansi2html as a direct dependency and only detected if it was installed and used it if so (falling back to stripping when missing).

[willmcgugan]

I've started work on https://github.com/willmcgugan/rich/tree/ansi-decoder

No promises on the time scale.

[ssbarnea]

In case anyone else is interested about the subject of redirecting stderr/stdout with rich Console, I released https://github.com/pycontribs/enrich which extends Console with this option.