Checking up on using cache to store Oauth refresh-token

[biznizz]

Sorry to bother, but I'm just trying to keep top of things.

With the newest version of gallery-dl, it's possible (and now the only way?) to have oauth refresh-tokens written directly to cache files now.

So, I'm just checking in to see if I've missed anything while making changes.

1. In my deviantart extractor in my .conf, I have written "refresh-token": "cache",

2. In my oauth section in my .conf, I have

"oauth":
        {
            "browser": true,
            "cache": true,
            "port": 6414
        },

3. I've long since had my cache file moved to a different location to avoid accidental deletion, and is told to be written in a new location by .conf.

"cache": {
        "file": "%USERPROFILE%/.gallery-dl.cache"
    },

If I've made any mistakes, please let me know!

[KirbyFan102]

I'm a newbie at using at gallery-dl, could you explain the benefits of doing this? And how I should modify my own config file to do the same? I've been getting '429 too many requests' errors in spite of having my refresh-token in the config file, so maybe this might help.

[biznizz]

So, what I think the new cache/OAuth token thing does is reduce the amount of maintenance needed for sites that use refresh-tokens.

Previously, the refresh-token was stored in the cache file, but you also needed to manually insert the token value into the config file. So long as the cache file isn't deleted, OAuth refresh-tokens have a life of about 3 months.

What this new update has done is, by directing the refresh-token value to point to the cache, you won't have to manually write it in anymore; so it's closer to it being a "set it, and forget it" type of deal. All you have to do is run gallery-dl oauth:(site name) in command window, and it's automatically set for 3 months right away.

As for helping your setup, my suggestion is to list your refresh-token value to cache in the DA section as well, and having your OAuth information the same as mine. I only have a specific value to tell gallery-dl where write/read the cache because I had issues with my old cache getting deleted in my TEMP folder by CCleaner, so I moved it to a more secure location to avoid accidental deletion.

Anything else, just make sure you have a client-id, client-secret, and your cookies values working. If you keep getting too many requests, try adding one to five minutes in the wait-min value, just to see if that works.

[KirbyFan102]

I have made the modifications to the refresh-token and oauth configs. I would also like to change my cache location, but it doesn't seem to be working. The file doesn't appear in the location I put in.

[earthyrock]

biznizz, don't mean to highjack your question, but I've got the following .conf and

gallery-dl.txt

1. don't understand where I would put the oauth section as you've listed.

and

2. I understand how to download the cookies.txt but don't know how to properly implement in the .conf file.

Any correction you can provide would be helpful...?

[biznizz]

@KirbyFan102

My original cache file was originally located on C:\Users\USER\AppData\Local\Temp, but that might have changed a bit ago.

According to https://github.com/mikf/gallery-dl/blob/master/docs/configuration.rst#cache-file , the cache file should generate naturally, most likely as cache.sqlite3. Try looking for your cache there.

example.txt

The location of the "cache" argument should be ideally be located as it's line not within another group (like "extractor" or "output"). I've uploaded a very condensed version of my conf in txt to show how I've set it up.

So while your cache might be called cache.sqlite3 currently, the way my configuration is, it goes to my user profile folder, named as .gallery-dl.cache, and it has been working well as is. I haven't had any issues with DA downloading so far, but I just wanted to ask the dev if my set up is fine just to be 100% sure (I'm still a caveman when it comes to code).

@earthyrock

I've added the relevant info back into your txt file.
gallery-dl.txt

The important bit is that, whether you use the natural cookies.txt exported from your browser, or only copy site relevant cookies onto a new cookies.txt, you have to direct the conf file to where the txt you're using is.

Example, if you have the cookies.txt you want to use on your desktop (assuming you're using Windows), you'd put this value in: "cookies": C:\\Users\\USER\\Desktop\\cookies.txt

Edit: Forgot to double slash for proper parsing, added in. Also, addition to help Kirby.

[earthyrock]

You know that whole saying, "Reading is fundamental"? Uhhhh-yeah, it is. I saw somewhere else here on the site the use of the double-\ in the path, but totally forgot when I was pasting in the actual path. Kept getting an error on that line and thought maybe I had a hidden character or something. FINALLY, read all the way to the end, and it was alllllmmmmmost like you were warning me, "Make sure you do this..."

Me = Duh!

Thanks for the correction!! ;)

[mikf]

@biznizz
Your explanation in #807 (comment) is pretty much spot on. The idea is to automatically store OAuth tokens and not to have users manually copy-and-insert them into their config files. This is enabled by default and it is now even possible to do OAuth stuff without having a config file.

Setting oauth.cache to true (the default) instructs gallery-dl to write the tokens from gallery-dl oauth:… into your cache file, and setting refresh-token to "cache" (again, the default) tells it to look into your cache file for a token (but it won't fail if there is none there.)

If you don't like this behavior, you can of course disable it and do the old OAuth flow by manually inserting the tokens into your config file like before.

(And the default location of cache files on Windows got changed to
%APPDATA%\gallery-dl\cache.sqlite3 in da22ea8)

If I've made any mistakes, please let me know!

No mistakes there. Everything should be working as intended.