[feature request] Multiple active progress bars

[pdimens]

First and foremost, what a wonderful package and accompanying documentation! I really like the simplicity of using the transient property for a ProgressBar and was surprised when it did not accommodate a nested loop, like so:

for i in track(1:3, description= "Total Progress", columns = :detailed)
    for j in track(1:5, description = "Inner Progress", transient = true)
        sleep(0.1)
    end
end

Yielding something like

Total Progress ● ━━━             ● 1/3  20 % ● elapsed:    0ms remaining:   4.0ms
Inner  Progress ● ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━                      ● 3/5  60 %

This seems like a very natural API for having a top-level progressbar that tracks the outermost loop and an inner-level progressbar that tracks the progress of the inner loop, clearing itself when the inner loop exhausts itself. The documentation describes why STDOUT is withheld until the end, so I was wondering if this will also be prohibitive for such an implementation.

[FedeClaudi]

Hi,

Yeah this is something that is going to happen for sure. It will look like's rich progress bars in python where you can have multiple "tasks" each showing up as a progress bar line and started/updated/stopped independently. It just takes a bit of work to get the functionality just right, but as you say it's a very natural thing to have.

Having nested calls to track on the other hand might not be possible. The way the progress bar display works, the line of text representing the progress bar needs to be erased and re-drawn. With multiple progress bars you need to know which line to erase. If you have a single progress bar object with multiple tasks you can keep track of which task is shown in which line, but track creates a whole new progress bar object.

So the code will look something like:

pbar = ProgressBar()
outer = addtask(pbar, "Total"; N=10)
for i in 1:10
    inner = addtask(pbar, "inner"; N=3)
    for j in 1:3
          update(inner)
    end
    update(outer)
end

which is a bit cumbersome. Though perhaps there's a way to have track spawn tasks as requested and avoid the extra code, I'll have to think about it!

[FedeClaudi]

Btw I'm currently working on making the code faster and cleaning up the codebase so the next release won't have any new feature. The one after that will include this

[FedeClaudi]

@pdimens this is coming: #62

Screen.Recording.2022-03-27.at.22.00.50.mov

[pdimens]

Ah, so exciting! Thank you for working on this!

[ChrisRackauckas]

Please don't make a Term.jl-only interface when one already exists and is built into Julia itself. The logging infrastructure specifies how progress information can be given. This is then be used by all downstream packages, such as VS Code, https://github.com/JuliaLogging/ProgressLogging.jl, https://github.com/JuliaLogging/TerminalLoggers.jl, Juno, etc. If you just accept that format, then progress bars from packages work. If you don't, then packages won't support it.

[FedeClaudi]

Hi Chris,

Could you elaborate?

The logging infrastructure specifies how progress information can be given.

Does this relate to logging progress info, or is it about printing this info to terminal? I haven't looked yet at how Term's packages look like in a log file, but I'm guessing they'll result in a bunch of repeated lines, I'm happy to try to avoid that if possible. What do they look like in e.g. ProgressLogging.jl?

If you just accept that format, then progress bars from packages work. If you don't, then packages won't support it.

Maybe I don't understand what 'interface' you're referring to above, but yes a package that wanted to use Term's progress bars would have to be modified to do so. Other than that I don't see what would prevent that from working.

I haven't used ProgerssLogging or TerminalLoggers so I'm not sure what you're referring to exactly. If it's about logging progress information I'm happy to try and conform, but could you give me more details about how to do that/what to look for? If it is about the progress API itself (as pertains progress information to the terminal), then I think I'll stick with what I'm working on. I need something capable of handling:

1. progress bars with multiple layouts and Term's styling
2. progressbars updated asynchronously from ongoing tasks: if you have something that does 10 iterations but each takes 5 minutes then you want a column shown elapsed time to be updated continunously, not once every 5 minutes.
3. Something capable of handling multiple progress bar related to distinct tasks (e.g. nested loops). Not only that, one should be able to add, start, pause and stop progress bars independently as required. As @pdimens and others have reported this is a feature people (and I) want to be able to use.

As far as I can tell neither ProgressBars.jl nor ProgressMeter.jl can handle 3, though I might be wrong there.

If there's a progress bar framework capable of handling all of this and more I can consider adapt Term to it. Otherwise I won't sacrifice functionality for it. Downstream packages would have to either adapt to it or not use Term for progress information (there's tons of other useful stuff fortunately).

[ChrisRackauckas]

Does this relate to logging progress info, or is it about printing this info to terminal?

It's just how the info is handled, not how it's printed. For example, OrdinaryDiffEq.jl can do this:

https://github.com/SciML/OrdinaryDiffEq.jl/blob/e06e744b2e290170154890dd3ecb075e77b2fe5f/src/integrators/integrator_utils.jl#L129-L135

which emits logging information for a progress bar named OrdinaryDiffEq (i.e. for multiple concurrent progress bars) without having a dependency on any of the progress monitoring packages. That is then picked up in Juno, VS Code, ProgressLogging.jl, etc.

[FedeClaudi]

@ChrisRackauckas I've moved this to a discussion because it's offtopic from the original issue. See here: #67