flickr not downloading #6360
Comments
"access-secret": "myothersecret"This should be |
It does not indeed. I corrected the config and had the same results. |
|
Downloading without login download few images and then keep getting 403 forbidden errors. Formerly it didn't happen |
|
UPDATE: "flickr": {
"api-key": "myverylongapikey",
"api-secret": "myapisecret",
"access-token": "myverylongtoken",
"access-token-secret": "myothersecret"
}It decided to download some of the files. Some others, however, failed with either errors 403 or 429. [urllib3.connectionpool][debug] Resetting dropped connection: live.staticflickr.com
[urllib3.connectionpool][debug] https://live.staticflickr.com:443 "GET /65535/53131855411_51145ae91c_3k.jpg HTTP/11" 403 93
[downloader.http][warning] '403 Forbidden' for 'https://live.staticflickr.com/65535/53131855411_51145ae91c_3k.jpg'
[download][error] Failed to download flickr_53131855411.jpg
[urllib3.connectionpool][debug] https://api.flickr.com:443 "GET /services/rest/?photo_id=53131855436&method=flickr.photos.getSizes&format=json&nojsoncallback=1 HTTP/11" 200 None
/mypath/flickr/Hotwife Jo Watt/flickr_53131855436.jpg
[urllib3.connectionpool][debug] Resetting dropped connection: live.staticflickr.com
[urllib3.connectionpool][debug] https://live.staticflickr.com:443 "GET /65535/53132337418_0bb7087959_4k.jpg HTTP/11" 429 117
[downloader.http][warning] '429 Too Many Requests' for 'https://live.staticflickr.com/65535/53132337418_0bb7087959_4k.jpg'What in the world could be happening now? |
|
A few days ago I started having issues with Flickr images not downloading (403 errors) whereas before everything was working 100%. I am not a professional computer guy, just an image collector. I went through my conf file and redid all of the fields mentioned above by hammerheaddf but for my system, exported cookies for Flickr, correct User Agent (this field is very imported for Twitter), created my own thing in the Flickr App Garden, used OAuth to get the extra two access fields. I have delayed the timers, sleep is 5.1 seconds for instance. But when I start downloading I fetch a few images, then 403 errors alternate with fetches, then just endless '403 Forbidden'. I start a new session 10 minutes later and the same thing happens. The file names are 'funny' because I am trying out renaming rules to get more useful information than just a string of numbers. Bits from the log: Interestingly, when I copy and paste the Jpeg URLs into my Browser I get: 403 Forbidden |
|
I am also having problems with Flickr recently, I think they rate limiting the free API heavily now. After around 15 consecutive downloads I start to get errors, 403 but also 429
|
|
increasing even more the sleep time seems to have helped |
|
The "retry-codes": [403, 420],
"retries": 50, |
|
It seems that you cannot view all content on flickr anymore, at least that is what I am seeing here with my account. There used to be this setting in the account options which allows you to see restricted content (as in NSFW content), that setting is still there but I cannot activate it anymore, because apparently a flickr Pro account is now required. Not sure if this is just my flickr test account, or if you guys are seeing the same thing as well. |
|
I think I found a solution. Instead of using regular image URLs, transform them to "download" URLs by adding Patch: diff --git a/gallery_dl/extractor/flickr.py b/gallery_dl/extractor/flickr.py
index df252ee3..eb5c6418 100644
--- a/gallery_dl/extractor/flickr.py
+++ b/gallery_dl/extractor/flickr.py
@@ -45,7 +45,7 @@ class FlickrExtractor(Extractor):
self.log.debug("", exc_info=exc)
else:
photo.update(data)
- url = photo["url"]
+ url = self._file_url(photo)
yield Message.Directory, photo
yield Message.Url, url, text.nameext_from_url(url, photo)
@@ -57,6 +57,13 @@ class FlickrExtractor(Extractor):
def photos(self):
"""Return an iterable with all relevant photo objects"""
+ def _file_url(self, photo):
+ if "video" in photo:
+ return photo["url"]
+
+ path, _, ext = photo["url"].rpartition(".")
+ return path + "_d." + ext
+
class FlickrImageExtractor(FlickrExtractor):
"""Extractor for individual images from flickr.com"""
@@ -98,7 +105,7 @@ class FlickrImageExtractor(FlickrExtractor):
if isinstance(value, dict):
location[key] = value["_content"]
- url = photo["url"]
+ url = self._file_url(photo)
yield Message.Directory, photo
yield Message.Url, url, text.nameext_from_url(url, photo)
|
|
@mikf Is there a reason you haven't merged that patch into the main project? |
|
@mikf this is now unfortunately returning the following: |
|
@misteramazingyt |
Excuse me, I am using the standalone EXE version, how can I use this patch? |
|
|
It works now! Thank you very much! |
|
Working now! Thank you for the assistance! |
Running gallery-dl 1.27.6 on Ubuntu arm64 (Raspberry Pi).
Created custom api keys following documentation, and added them to .gallery-dl.conf.
When trying to download from user Jo Watt (NSFW), the API doesn't fetch anything:
Funny enough, when I access the same API entry point over Firefox on the same machine, the requests get the expected JSON. The only difference I spotted is that FF uses HTTP/2, when urllib goes on HTTP/1.1.
This is the back-and-forth from FF:
I configured gallery-dl to pull cookies from the local FF, and even to emulate it via the
browseroption. The results were the same as above. Here's my config snippet for flickr:Does anybody have an idea why flickr is acting up?
The text was updated successfully, but these errors were encountered: