danbooru cloudflare 403 no matter what

[Hugs288]

tried with cookies, login, multiple user agents, etc, while i can view links fine in web browsers gallery-dl always gives me cloudflare challenge 403.

maybe this has something to do with tor? i use tor with socks5 to access danbooru.

[mikf]

Danbooru responds with 403 errors when using a standard/default User-Agent header in a script, which hasn't been the case for gallery-dl since v1.25.0 (cbe4769).

What's your output for gallery-dl --version?

[Hugs288]

1.27.7

[mikf]

Then it's probably tor.

edit:

$ gallery-dl --proxy socks5h://127.0.0.1:9050 https://danbooru.donmai.us/posts/686612
[danbooru][warning] Cloudflare challenge
[danbooru][error] HttpError: '403 Forbidden' for 'https://danbooru.donmai.us/posts/686612.json'

$ gallery-dl https://danbooru.donmai.us/posts/686612
./danbooru/danbooru_686612_206aee80734e0e5ab65aff450fdffa31.jpg

[Hugs288]

yeah i figured, but imgbrd grabber works fine with tor and socks5

[mikf]

As it turns out, accessing the API through Tor always triggers a Cloudflare challenge page, even in a browser.

Letting your browser solve the challenge by visiting an API page like https://danbooru.donmai.us/posts/686612.json, and using exported cookies and browser User-Agent would work,

$ gallery-dl --cookies-from firefox:/tmp/ --user-agent browser --proxy socks5h://127.0.0.1:9050 https://danbooru.donmai.us/posts/686612
[cookies][info] Extracted 2 cookies from Firefox
./danbooru/danbooru_686612_206aee80734e0e5ab65aff450fdffa31.jpg

but the danbooru code currently overwrites any custom User-Agent header.

gallery-dl/gallery_dl/extractor/danbooru.py

Line 26 in 00fe1c8

self.session.headers["User-Agent"] = util.USERAGENT

[Hugs288]

yep, removed that line of code from danbooru.py, works perfectly now.