Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make downloaded files nothing more than a cache so that all the features work #10542

Open
6 tasks done
JeromeA opened this issue Nov 5, 2023 · 18 comments
Open
6 tasks done
Labels
discussion This needs to be discussed before anything is done downloader Issue is related to the downloader feature request Issue is related to a feature in the app

Comments

@JeromeA
Copy link

JeromeA commented Nov 5, 2023

Checklist

  • I made sure that there are no existing issues - open or closed - which I could contribute my information to.
  • I have read the FAQ and my problem isn't listed.
  • I'm aware that this is a request for NewPipe itself and that requests for adding a new service need to be made at NewPipeExtractor.
  • I have taken the time to fill in all the required details. I understand that the feature request will be dismissed otherwise.
  • This issue contains only one feature request.
  • I have read and understood the contribution guidelines.

Feature description

Many features don't work on downloaded files, and there are several bugs already open for individual features (enqueue #9829, mark as watched #9334, already downloaded #7360), but the real problem is that downloaded files are considered a separate case with its own features that are independent from the rest of NewPipe.

Also, when playing a downloaded file, it stops playing if you switch to a different app, one can't play the audio only, etc.

If downloaded files were considered a cache (which obviously requires a lot of work), then all the feature would not have to be treated as a special case, with a different bug for each of them, and new features would take this case into account.

Why do you want this feature?

Before losing data connectivity, I want to download a bunch of videos, and then continue using NewPipe offline, with all the features still working (going to a channel, selecting a video to play, switching to another app, etc), as all the information NewPipe needs is available locally.

Additional information

No response

@JeromeA JeromeA added feature request Issue is related to a feature in the app needs triage Issue is not yet ready for PR authors to take up labels Nov 5, 2023
@SameenAhnaf SameenAhnaf added discussion This needs to be discussed before anything is done downloader Issue is related to the downloader and removed needs triage Issue is not yet ready for PR authors to take up labels Nov 5, 2023
@opusforlife2
Copy link
Collaborator

The location for this should probably be /Android/media/org.schabi.newpipe/, like Whatsapp does for its files. If the user messes around with this and gets errors in the app, that's their own fault.

Maybe there could be a dialog upon tapping Download to choose between 2 options:

  • "Download for offline play (This will download the file to the app's internal folder for playback within Newpipe)"
  • "Download to permanent storage (This will download the file to an external folder for playback by other media players)"

@SameenAhnaf
Copy link
Collaborator

SameenAhnaf commented Nov 6, 2023

This option in settings could be shown in a more simpler way. Enable Local Player in App with a sub-note Downloaded files will be played directly on Newpipe app but become not possible to be played on other apps

@SameenAhnaf
Copy link
Collaborator

There could be two checkboxes as well: Videos and Audios. Some users may have only video or only audio player app installed on their phone. Or, one of their used apps don't offer the function available on Newpipe app. These checkboxes will be helpful for them.

@jstetten
Copy link

jstetten commented Nov 7, 2023

This would be wonderful. I currently use a podcast app to play downloaded videos. This is less than ideal for several reasons:

  • The obvious hassle
  • Having to search the vid on NewPipe again, once done, so I can manually mark it as played. (As someone who consumes many short vids throughout the day [especially music], the played visual indicator is really important to me.)
  • Lack of metadata. Sometimes this is really needed, i.e. timestamps. It would be amazing to download the top X comments, as a lot of times that's where timestamps occur, too.

Please see this as mere brainstorming...I'm quite aware this all gets really tricky.

@opusforlife2
Copy link
Collaborator

Enable Local Player in App

That removes from the user the ability to choose where individual downloads go. I would understand having a checkbox like "Don't ask me again" for the Download alternatives dialog, but tightly coupling local playability with downloading to a specific location seems restrictive.


Another alternative is two completely separate buttons. The existing Download button stays as it is, and an additional "Save/Cache/Mark for offline playback"/"Play offline" can be added. I would suggest replacing the "Add to playlist" button with this one instead.

@opusforlife2
Copy link
Collaborator

@foxjaw Instead of making multiple comments if you have something to add, you can edit your existing comment as many times as you like.

@brunoais
Copy link

brunoais commented Nov 9, 2023

This is good but I'd want to be able to download to open with VLC for example. Maybe as an option which is off by default.

@Atemu
Copy link
Contributor

Atemu commented Nov 9, 2023

I'd advocate for two separate buttons:

  1. "Available offline" or "Download". This one would work similar to the one in the YT app (and many other vide streaming apps) making the whole video available offline, including the info page.
  2. "Save to storage" which puts a media file into the storage. If the video is available offline, it'd just show the file location dialog and copy/mux the cached video, if not, something similar to the download dialog would be shown; bypassing the offline cache.

This way, offline videos could also be put into the app's private storage's cache/ directory. They do not need to be user-accessible. Though clearing the app's cache would then also clear these offline videos which might not be desirable.

@opusforlife2
Copy link
Collaborator

Though clearing the app's cache would then also clear these offline videos which might not be desirable.

That's a tricky one. One straightforward answer to this is that we simply don't use the regular cache folder, but another one that persists across cache-clearing operations. This folder would only be deleted if the user specifically takes that action.

However, that means offline videos would start filling up the App Data instead of Cache metric. If anything goes wrong (as it often does with filesystem operations), then the only way to get rid of all that potential junk is to clear the app data. Very non-ideal.

I don't know if it's possible, but maybe the offline video folder could remain inside the regular cache folder, and using the cache-clearing operation from within the app would prevent that folder from being emptied. Or maybe show a dialog asking if the user wants to delete cached videos as well.

This would also mean that clearing cache from the OS App Info menu would remove the entire cache including videos, but we could explain that in the app settings, and maybe in our FAQ as well.

@opusforlife2
Copy link
Collaborator

In that case, any filesystem operations going wrong would cause junk to be left around, bloating storage.

@brunoais
Copy link

@opusforlife2 Not if the videos are stored in a well-designated directory and there's no database about what videos are supposed to be there. The name of the file already designates what the video is and how to link it to the original one so that one is executed instead of the original one. One straightforward way would be:

/storage/emulated/0/Android/data/org.schabi.newpipe/videos/offline/ha7/ha7zuYS7B0c_[videotitle].[ext]
/storage/emulated/0/Android/data/org.schabi.newpipe/videos/offline/ha7/ha7zuYS7B0c_[videotitle].txt

The txt file contains more metadata if that's needed
Usage of sub-directories is just a possibility if interested.
If you need multiple ways to reach a video, you can use symlinks and clean them periodically when the phone has more battery and not so busy. Ofc by then maybe it makes sense to have a database again. However, the source of truth must be the file being there, the database is more of a cache... In the cache directory.

@brunoais
Copy link

@foxjaw That information can be stored in the companion file. I wrote there txt but it can just be a json file or a CSV file maybe even making it gzipped or zsted compressed, if the saves make sense. Whatever format that would be useful. Maybe both streams are saved separately because that's how youtube does it, that's fine too. One silent mp4 and one m4a. Basically, 3 files per save.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion This needs to be discussed before anything is done downloader Issue is related to the downloader feature request Issue is related to a feature in the app
Projects
None yet
Development

No branches or pull requests

7 participants
@JeromeA @jstetten @brunoais @Atemu @opusforlife2 @SameenAhnaf and others