Disable cache for large files #79
Comments
|
This is currently not possible. I may add an option for it in the future, but it would have the same limitations as rclone when doing streaming writes. Files would have to be written sequentially (no random writes) and retries would not be supported in case of network issues. |
|
Currently RSAF would continue to upload when network is back, so I don't really understand why retries would be unavailable. |
|
BTW I'm quite curious if RSAF could show the approximate progress bar for upload and download in the notification, since large files currently will be cached first and there's no way to tell how much it had been uploaded. Also it would be nice to see the upload speed and how much time left to finish the task. If it's technically possible, then please add a pause action button as well. I know I'm requesting for too much, so just take them as an inspiration for new functions. |
This is only possible when caching is enabled. With caching disabled, retrying is not possible. This is an rclone limitation: https://rclone.org/commands/rclone_mount/#vfs-cache-mode-off rclone likely has this limitation because most backends don't actually support resuming uploads. "Retrying" is actually reuploading from scratch. Without the cached file, there's nowhere it can reread the data from. (RSAF cannot ask the client app to recreate the file either--the Android API does not support this.) Regarding the upload progress, I'm unfortunately very limited by rclone here. There's no way to monitor anything about that process or control how it behaves besides turning it on or off completely. I might be able to do something hacky to show the approximate progress, but pause/resume is definitely not possible. |
VFS caching can now be configured per-remote. Disabling it is equivalent to `rclone mount --vfs-cache-mode none`. This will disable support for random writes and prevent failed uploads from being retried. However, it allows uploads to begin immediately and allows the client to show more accurate file write progress. If a remote type does not support streaming, VFS caching will be force enabled and the UI option will be grayed out. This is preferable to allowing the user to set a broken configuration and not knowing until they try to create a file and having it fail. Internally, RSAF now maintains two sets of VFS instances. One set with caching VFSs and another set with streaming VFSs. This allows the caching option to be toggled immediately without waiting for open files to be closed (and fully uploaded). The downside is that if the user toggles the option while uploads are occurring, the directory listings may be inconsistent until the uploads complete. Issue: #79 Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
|
I've implemented support for disabling caching per-remote in #86 (equivalent to rclone's I'm planning to take a look at the upload notification soon to see if I can implement some hacky workaround to show the progress. |
VFS caching can now be configured per-remote. Disabling it is equivalent to `rclone mount --vfs-cache-mode none`. This will disable support for random writes and prevent failed uploads from being retried. However, it allows uploads to begin immediately and allows the client to show more accurate file write progress. If a remote type does not support streaming, VFS caching will be force enabled and the UI option will be grayed out. This is preferable to allowing the user to set a broken configuration and not knowing until they try to create a file and having it fail. Internally, RSAF now maintains two sets of VFS instances. One set with caching VFSs and another set with streaming VFSs. This allows the caching option to be toggled immediately without waiting for open files to be closed (and fully uploaded). The downside is that if the user toggles the option while uploads are occurring, the directory listings may be inconsistent until the uploads complete. Issue: #79 Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
VFS caching can now be configured per-remote. Disabling it is equivalent to `rclone mount --vfs-cache-mode none`. This will disable support for random writes and prevent failed uploads from being retried. However, it allows uploads to begin immediately and allows the client to show more accurate file write progress. If a remote type does not support streaming, VFS caching will be force enabled and the UI option will be grayed out. This is preferable to allowing the user to set a broken configuration and not knowing until they try to create a file and having it fail. Internally, RSAF now maintains two sets of VFS instances. One set with caching VFSs and another set with streaming VFSs. This allows the caching option to be toggled immediately without waiting for open files to be closed (and fully uploaded). The downside is that if the user toggles the option while uploads are occurring, the directory listings may be inconsistent until the uploads complete. Issue: #79 Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Unfortunately, rclone has no APIs, internal or otherwise, for querying the upload progress for the VFS writeback cache. We'll do it ourselves the hacky way by scanning all open file descriptors for open files in the VFS cache directory and grab the current file positions and sizes. Issue: #79 Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Unfortunately, rclone has no APIs, internal or otherwise, for querying the upload progress for the VFS writeback cache. We'll do it ourselves the hacky way by scanning all open file descriptors for open files in the VFS cache directory and grab the current file positions and sizes. Issue: #79 Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
|
The upload progress notification (for remotes with VFS caching enabled) has been updated to show the progress in #87. This will be included in the next version of RSAF as well. The notification shows the size uploaded so far and the total size to upload. Given how hacky this is, I'd prefer not to try and compute the speed and complicate it further. |
When uploading large files(e.g. videos>1G), is it possible to disable cache? Just read from the dir provided by the app and don't copy it to RSAF private dir. This will help reduce disk write and extremely useful when the device don't have enough space for cache
The text was updated successfully, but these errors were encountered: