[Feature Request] Print progress information while downloading media

[mhogomchungu]

Currently, the app appears to just hang while its downloading media and then prints the name of the downloaded file and quits and this is not a good user experience.

It would be nice if the app will print progress info while its downloading something and useful info to print are:-

1. Download speed.
2. Estimated time remaining.
3. Final file size if it can be known or reasonably guessed.

It will also be nice if the file name is printed as the first output and not the last.

The "-v" option has info that are not useful to end users.

[mikf]

I'd guess >90% of all downloads made by gallery-dl take less than 1 second and >99% less than 5 seconds, so I wouldn't consider this all that necessary. Maybe in v1.18.0 or v2.0.0.

user experience

It's a command-line program that is meant to do its thing without user input.
What I'd actually consider terrible user experience is the current config file format, post processors, and lacking documentation.

It will also be nice if the file name is printed as the first output and not the last.

It does, depending on output.mode. Usually it prints the output filename when starting the download and then overwrites it with a slightly different version indicating it completed that download.
The default ("auto") uses "pipe" whenever stdout is not a TTY, which only prints the name when it's done.

The "-v" option has info that are not useful to end users.

Well, yeah, it is there to enable debug output and not for everyday use.

[mhogomchungu]

Reason of why i opened this feature request:-

I have a project called Media Downloader[1] and i was thinking of adding your project as one of its supported extensions[2] after discovering it a few days ago when it was mentioned here[3] and my project currently does not behave well with tools that do not print anything upfront.

I did not know the usual download times is only a few seconds

[1] https://github.com/mhogomchungu/media-downloader

[2] https://github.com/mhogomchungu/media-downloader/tree/main/extensions

[3] https://www.reddit.com/r/youtubedl/comments/n2oqjj/is_there_similar_program_for_instagram_need_one/

[TestPolygon]

Newgrounds' downloads are notable slow. For this it would be nice to have.

[mhogomchungu]

If i can add to this post, i downloaded a 3MB video file from instagram and the download took more than a minute and its not a good GUI experience to hang for that longwithout showing a user any progress.

This is not a good user experience even when running manually from the terminal.

[ink@localhost ~]$ date ; "/home/ink/.local/share/media-downloader/bin/gallery-dl.bin" "https://www.instagram.com/p/CPlDGhUnDu_/" ; date
Tue Jun  1 05:02:28 PM EAT 2021
./gallery-dl/instagram/kigogo__2014/2586487203229875135.mp4
Tue Jun  1 05:03:49 PM EAT 2021
[ink@localhost ~]$ stat ./gallery-dl/instagram/kigogo__2014/2586487203229875135.mp4
  File: ./gallery-dl/instagram/kigogo__2014/2586487203229875135.mp4
  Size: 3124634         Blocks: 6104       IO Block: 4096   regular file
Device: 807h/2055d      Inode: 39322482    Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  500/     ink)   Gid: (  500/     ink)
Access: 2021-06-01 17:03:49.863506317 +0300
Modify: 2021-06-01 16:16:19.000000000 +0300
Change: 2021-06-01 17:03:49.862439957 +0300
 Birth: 2021-06-01 17:02:30.439315418 +0300
[ink@localhost ~]$ 

The above output shows the download started at Tue Jun 1 05:02:28 PM EAT 2021 and finished more than a minute later at Tue Jun 1 05:03:49 PM EAT 2021

[mhogomchungu]

It does not work when std out is not a terminal and progress report does not show up there.

Tested with version 1.24.1.

example test

gallery-dl.bin https://2chen.moe/assets/images/src/1f58e66e7972cebece1d016342b7fb985eb23f1a.webm 2>err 1>out

Look inside the generated "out" text file and only the file name will be in it

[mikf]

Download progress goes to stderr and with default settings it only shows after 3 seconds.

gallery-dl/docs/gallery-dl.conf

Line 381 in 41bf236

"progress": 3.0,

[mhogomchungu]

There seems to be a problem with how std err is being used in combination with std out

My findings are below with default options.

URL = https://2chen.moe/assets/images/src/1f58e66e7972cebece1d016342b7fb985eb23f1a.webm

Below commands works as expected and progress is seen on the terminal and file name are both seen on the terminal.

gallery-dl.bin $URL

Below command works as expected, progress is added to err file and file name is printed on the terminal

gallery-dl.bin $URL 2>err

Below command does not work, nothing is written to err file and file name is written in out file.

gallery-dl.bin $URL 2>err 1>out

Below command does not work, progress is not written to the output file, only file name is added.

gallery-dl.bin $URL 1>out 2>&1

[mikf]

Below command does not work, nothing is written to err file and file name is written in out file.

gallery-dl.bin $URL 2>err 1>out

Below command does not work, progress is not written to the output file, only file name is added.

gallery-dl.bin $URL 1>out 2>&1

When stdout is not a TTY and output.mode is set to the default "auto", download progress is disabled since "mode": "pipe" gets used internally and that mode doesn't display any progress.

[mhogomchungu]

Below command seems to work

gallery-dl.bin" -o "output.mode=terminal"  $URL 1>out 2>&1

The out file now contains below output

  ./gallery-dl/directlink/2chen.moe_ass…66e7972cebece1d016342b7fb985eb23f1a.webm
 16%   1.01MB 337.19kB/s 
 16%   1.04MB 264.48kB/s 
 17%   1.08MB 248.24kB/s 
 18%   1.11MB 255.74kB/s 
 18%   1.14MB 263.25kB/s 
 19%   1.17MB 270.77kB/s 
 19%   1.21MB 278.28kB/s 
 20%   1.24MB 284.61kB/s 
 20%   1.27MB 286.94kB/s 
 21%   1.31MB 289.93kB/s 
 21%   1.34MB 292.43kB/s 
 22%   1.37MB 294.17kB/s 
 22%   1.40MB 294.08kB/s 
 23%   1.44MB 292.42kB/s 
 23%   1.47MB 283.18kB/s 
 24%   1.50MB 278.01kB/s 
 24%   1.54MB 284.03kB/s 
 25%   1.57MB 290.07kB/s 
 26%   1.60MB 294.82kB/s 
 26%   1.63MB 297.96kB/s 
 27%   1.67MB 298.45kB/s 
 27%   1.70MB 300.55kB/s 
 28%   1.73MB 301.72kB/s 
 28%   1.76MB 303.21kB/s 
 29%   1.80MB 303.46kB/s 
 29%   1.83MB 305.10kB/s 
 30%   1.86MB 306.53kB/s 
 30%   1.90MB 308.32kB/s 
 31%   1.93MB 308.32kB/s 
 31%   1.96MB 308.59kB/s 
 32%   1.99MB 308.73kB/s 
 32%   2.03MB 309.23kB/s 
 33%   2.06MB 310.40kB/s 
 33%   2.09MB 312.53kB/s 
 34%   2.12MB 314.56kB/s 
 35%   2.16MB 315.66kB/s 
 35%   2.19MB 317.34kB/s 
 36%   2.22MB 318.94kB/s 
 36%   2.26MB 319.17kB/s 
 37%   2.29MB 319.42kB/s 
 37%   2.32MB 318.24kB/s 
 38%   2.35MB 317.41kB/s 
 38%   2.39MB 312.11kB/s 
 39%   2.42MB 312.11kB/s 
 39%   2.45MB 312.10kB/s 
 40%   2.49MB 312.14kB/s 
 40%   2.52MB 312.48kB/s 
 41%   2.55MB 312.33kB/s 
 41%   2.58MB 312.41kB/s 
 42%   2.62MB 312.78kB/s 
 42%   2.65MB 314.36kB/s 
 43%   2.68MB 316.42kB/s 
 44%   2.71MB 317.93kB/s 
 44%   2.75MB 319.48kB/s 
 45%   2.78MB 320.58kB/s 
 45%   2.81MB 321.11kB/s 
 46%   2.85MB 321.38kB/s 
 46%   2.88MB 322.04kB/s 
 47%   2.91MB 321.11kB/s 
 47%   2.94MB 318.67kB/s 
 48%   2.98MB 318.77kB/s 
 48%   3.01MB 319.02kB/s 
 49%   3.04MB 319.91kB/s 
 49%   3.08MB 320.48kB/s 
 50%   3.11MB 319.72kB/s 
 50%   3.14MB 316.85kB/s 
 51%   3.17MB 317.65kB/s 
 52%   3.21MB 318.64kB/s 
 52%   3.24MB 311.89kB/s 
 53%   3.27MB 312.65kB/s 
 53%   3.30MB 312.89kB/s 
 54%   3.34MB 312.53kB/s 
 54%   3.37MB 312.46kB/s 
 55%   3.40MB 311.90kB/s 
 55%   3.44MB 311.87kB/s 
 56%   3.47MB 312.54kB/s 
 56%   3.50MB 313.18kB/s 
 57%   3.53MB 313.11kB/s 
 57%   3.57MB 313.15kB/s 
 58%   3.60MB 311.26kB/s 
 58%   3.63MB 310.74kB/s 
 59%   3.67MB 310.29kB/s 
 59%   3.70MB 310.22kB/s 
 60%   3.73MB 310.50kB/s 
 61%   3.76MB 310.56kB/s 
 61%   3.80MB 310.39kB/s 
 62%   3.83MB 311.13kB/s 
 62%   3.86MB 311.40kB/s 
 63%   3.89MB 311.63kB/s 
 63%   3.93MB 312.28kB/s 
 64%   3.96MB 312.53kB/s 
 64%   3.99MB 312.74kB/s 
 65%   4.03MB 313.64kB/s 
 65%   4.06MB 314.19kB/s 
 66%   4.09MB 315.01kB/s 
 66%   4.12MB 315.31kB/s 
 67%   4.16MB 315.73kB/s 
 67%   4.19MB 316.01kB/s 
 68%   4.22MB 316.72kB/s 
 69%   4.25MB 316.69kB/s 
 69%   4.29MB 316.53kB/s 
 70%   4.32MB 316.45kB/s 
 70%   4.35MB 316.36kB/s 
 71%   4.39MB 315.89kB/s 
 71%   4.42MB 315.66kB/s 
 72%   4.45MB 315.91kB/s 
 72%   4.48MB 316.38kB/s 
 73%   4.52MB 316.77kB/s 
 73%   4.55MB 317.03kB/s 
 74%   4.58MB 316.19kB/s 
 74%   4.62MB 316.29kB/s 
 75%   4.65MB 316.68kB/s 
 75%   4.68MB 316.62kB/s 
 76%   4.71MB 316.60kB/s 
 76%   4.75MB 315.82kB/s 
 77%   4.78MB 314.73kB/s 
 78%   4.81MB 314.21kB/s 
 78%   4.84MB 313.50kB/s 
 79%   4.88MB 313.99kB/s 
 79%   4.91MB 314.29kB/s 
 80%   4.94MB 313.69kB/s 
 80%   4.98MB 312.42kB/s 
 81%   5.01MB 311.82kB/s 
 81%   5.04MB 311.36kB/s 
 82%   5.07MB 311.20kB/s 
 82%   5.11MB 310.34kB/s 
 83%   5.14MB 309.44kB/s 
 83%   5.17MB 308.69kB/s 
 84%   5.21MB 308.29kB/s 
 84%   5.24MB 307.36kB/s 
 85%   5.27MB 307.13kB/s 
 85%   5.30MB 307.20kB/s 
 86%   5.34MB 307.29kB/s 
 87%   5.37MB 307.42kB/s 
 87%   5.40MB 307.72kB/s 
 88%   5.43MB 308.42kB/s 
 88%   5.47MB 308.92kB/s 
 89%   5.50MB 309.34kB/s 
 89%   5.53MB 309.45kB/s 
 90%   5.57MB 310.02kB/s 
 90%   5.60MB 310.38kB/s 
 91%   5.63MB 310.69kB/s 
 91%   5.66MB 309.98kB/s 
 92%   5.70MB 309.58kB/s 
 92%   5.73MB 309.10kB/s 
 93%   5.76MB 309.06kB/s 
 93%   5.79MB 308.92kB/s 
 94%   5.83MB 308.86kB/s 
 95%   5.86MB 308.58kB/s 
 95%   5.89MB 308.47kB/s 
 96%   5.93MB 308.71kB/s 
 96%   5.96MB 308.37kB/s 
 97%   5.99MB 306.31kB/s 
 97%   6.02MB 304.24kB/s 
 98%   6.06MB 304.41kB/s 
 98%   6.09MB 305.09kB/s 
 99%   6.12MB 305.66kB/s 
 99%   6.16MB 306.61kB/s 
100%   6.17MB 307.06kB/s 
✔ ./gallery-dl/directlink/2chen.moe_ass…66e7972cebece1d016342b7fb985eb23f1a.webm