[reddit] Support for saved posts

[kattjevfel]

I'm a bit unsure if I'm doing something wrong or if saved posts is simply not supported, if so then I'd like to ask for it to be included.

I have set client-id and user-agent like the docs said.

~ » gallery-dl -v https://www.reddit.com/user/redacted/saved
[gallery-dl][debug] Version 1.10.5-dev
[gallery-dl][debug] Python 3.7.4 - Linux-5.2.14-arch1-1-fsync-x86_64-with-arch-Arch-Linux
[gallery-dl][debug] requests 2.22.0 - urllib3 1.25.3
[gallery-dl][debug] Starting DownloadJob for 'https://www.reddit.com/user/redacted/saved'
[gallery-dl][debug] Updating urllib3 ciphers
[reddit][debug] Using RedditUserExtractor for 'https://www.reddit.com/user/redacted/saved'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): oauth.reddit.com:443
[urllib3.connectionpool][debug] https://oauth.reddit.com:443 "GET /user/redacted/saved/.json?limit=100&raw_json=1 HTTP/1.1" 403 38
[reddit][error] You do not have permission to access the resource at 'https://www.reddit.com/user/redacted/saved'

[mikf]

I have set client-id and user-agent like the docs said.

You need a refresh-token like the docs say: gallery-dl oauth:reddit

For accessing private stuff from OAuth sites, you only need to link your personal account with gallery-dl oauth:<site> and put the values it tells you to into your config. Getting your own client credentials is usually not necessary.

[kattjevfel]

Oh, right. Maybe that should be mentioned in the documentation? I did notice just now that it says how to do it in the main readme, but frankly I just assumed anything I'd needed for configuring gallery-dl would be in docs/configuration.rst.

Anyhow, I got myself a refresh-token and it still didn't work sadly, giving me the same error as before.

[Hrxn]

Isn't /user/<username>/saved some kind of new feature?

[kattjevfel]

@Hrxn to my knowledge saved posts have been a thing for a rather long time?

Here's a thread about it 4 years ago
https://www.reddit.com/r/help/comments/3j8k2o/how_to_view_saved_reddit_posts/

[Hrxn]

Ah, I remember. These kind of saved posts. Yes, that has been a thing for a long time, but you can or rather could only access your own saved posts from your own profile.

Maybe that has changed, with /user/<not-you-some-other-user>/saved?

(If that's what you are actually trying to do)

And BTW, I think the /user/<username> URL format is pretty new (since the big redesign), as opposed to just /u/<username>

[kattjevfel]

Yes my use case is basically this:
I save loads of stuff on my phone for downloading later on my pc (because franky, the reddit app sucks and the third party ones are no better)

https://www.reddit.com/user/me/saved/ redirects to https://www.reddit.com/user/*myusername*/saved/

Trying to access that as not logged in gives me this:

You do not have permission to access this resource

You can only look at your own saved posts and comments

For shits n giggles I tried https://www.reddit.com/user/me/saved/ with gallery-dl, which output makes it seem like this has been a supported feature, but is broken now.

~ » gallery-dl --verbose https://www.reddit.com/user/me/saved/
[gallery-dl][debug] Version 1.10.5-dev
[gallery-dl][debug] Python 3.7.4 - Linux-5.2.14-arch1-1-fsync-x86_64-with-arch-Arch-Linux
[gallery-dl][debug] requests 2.22.0 - urllib3 1.25.3
[gallery-dl][debug] Starting DownloadJob for 'https://www.reddit.com/user/me/saved/'
[gallery-dl][debug] Updating urllib3 ciphers
[reddit][debug] Using RedditUserExtractor for 'https://www.reddit.com/user/me/saved/'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): oauth.reddit.com:443
[urllib3.connectionpool][debug] https://oauth.reddit.com:443 "GET /user/me/saved/.json?limit=100&raw_json=1 HTTP/1.1" 400 40
[reddit][error] An unexpected error occurred: Exception - Bad Request. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[reddit][debug] 
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/gallery_dl/job.py", line 47, in run
    for msg in self.extractor:
  File "/usr/lib/python3.7/site-packages/gallery_dl/extractor/reddit.py", line 38, in items
    for url, data in self._urls(submissions):
  File "/usr/lib/python3.7/site-packages/gallery_dl/extractor/reddit.py", line 62, in _urls
    for submission, comments in submissions:
  File "/usr/lib/python3.7/site-packages/gallery_dl/extractor/reddit.py", line 287, in _pagination
    data = self._call(endpoint, params)["data"]
  File "/usr/lib/python3.7/site-packages/gallery_dl/extractor/reddit.py", line 278, in _call
    raise Exception(data["message"])
Exception: Bad Request

And to clarify, I have sorted out the login stuff, it even told me [reddit][info] Refreshing private access token

[mikf]

Turns out the Reddit refresh-token needs an additional "scope" to be able to access /user/<username>/... API endpoints. It worked just fine when I implemented this, but I might have only tested with /user/<username> URLs.

803d8f8 adds the needed history scope when requesting a token with gallery-dl oauth:reddit. Try getting a new token after applying that commit and see if this works. It did for me.

(/user/me/saved/ only "works" because it matches the same regex pattern as /user/<username>/saved/ and uses the same code path. Would've been nice if it actually worked "out of the box", but alas ...)

[kattjevfel]

Hooray, that worked! Thanks for the quick fix!