You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Xhamster galleries fail to download due to KeyError - 'authorModel'. Console verbose output from Windows/cmd.exe and Debian/WSL using the URL from /test/results/xhamster.py included below.
Windows/cmd.exe:
gallery-dl --verbose https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946
[gallery-dl][debug] Version 1.28.4 - Executable (stable/windows)
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.22621
[gallery-dl][debug] requests 2.32.3 - urllib3 2.2.3
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[xhamster][debug] Using XhamsterGalleryExtractor for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): xhamster.com:443
[urllib3.connectionpool][debug] https://xhamster.com:443 "GET /photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946 HTTP/11" 200 None
[xhamster][error] An unexpected error occurred: KeyError - 'authorModel'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[xhamster][debug]
Traceback (most recent call last):
File "gallery_dl\job.pyc", line 152, in run
File "gallery_dl\extractor\xhamster.pyc", line 43, in items
File "gallery_dl\extractor\xhamster.pyc", line 53, in metadata
KeyError: 'authorModel'
Debian/WSL:
[gallery-dl][debug] Version 1.28.4 - Git HEAD: d98d3f5
[gallery-dl][debug] Python 3.10.5 - Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.28
[gallery-dl][debug] requests 2.28.2 - urllib3 1.26.11
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[xhamster][debug] Using XhamsterGalleryExtractor for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): xhamster.com:443
[urllib3.connectionpool][debug] https://xhamster.com:443 "GET /photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946 HTTP/1.1" 200 None
[xhamster][error] An unexpected error occurred: KeyError - 'authorModel'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[xhamster][debug]
Traceback (most recent call last):
File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/job.py", line 152, in run
for msg in extractor:
File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/extractor/xhamster.py", line 43, in items
data = self.metadata()
File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/extractor/xhamster.py", line 53, in metadata
user = self.data["authorModel"]
KeyError: 'authorModel'
It looks like the extractor is looking for the "authorModel" metadata, but Xhamster has changed the payload and no longer includes an authorModel containing the user info for the gallery. Checking the current window.initials value, it looks like some basic user info can be found in galleryPage.infoProps.authorInfoProps with the username itself at galleryPage.infoProps.authorInfoProps.authorName.
The user ID should be switched from: "id" : text.parse_int(user["id"]),
to "id" : text.parse_int(user["userID"]),
"url" is now at galleryPage.infoProps.authorInfoProps.authorLink
"name" is now at galleryPage.infoProps.authorInfoProps.authorName
"retired" may be gone entirely.
"verified" is now at galleryPage.infoProps.authorInfoProps.verified
"subscribers" value can be found at galleryPage.infoProps.subscribeButtonProps.subscribers
Also some slight modification to the pagination: pgntn = data["pagination"]
could be changed to pgntn = data["galleryPage"]["paginationProps"]
if pgntn["active"] == pgntn["maxPage"]:
should be changed to if pgntn["currentPageNumber"] == pgntn["lastPageNumber"]:
the pagination next value also needs to be determined differently. Maybe just adding 1 to pgntn["currentPageNumber"]?
In testing out potential fixes I noticed some issues with missing verified and url values, due to contexts where the user account isn't verified and when the account has been disabled (but gallery itself is still active). Probably need more graceful handling of missing values instead of rejecting the download when the gallery still exists, but I'm not a Pythonista and my suggestions on how to do that wouldn't be worth much.
The text was updated successfully, but these errors were encountered:
Xhamster galleries fail to download due to
KeyError - 'authorModel'
. Console verbose output from Windows/cmd.exe and Debian/WSL using the URL from /test/results/xhamster.py included below.Windows/cmd.exe:
Debian/WSL:
It looks like the extractor is looking for the "authorModel" metadata, but Xhamster has changed the payload and no longer includes an authorModel containing the user info for the gallery. Checking the current window.initials value, it looks like some basic user info can be found in
galleryPage.infoProps.authorInfoProps
with the username itself atgalleryPage.infoProps.authorInfoProps.authorName
.The user ID should be switched from:
"id" : text.parse_int(user["id"]),
to
"id" : text.parse_int(user["userID"]),
"url" is now at
galleryPage.infoProps.authorInfoProps.authorLink
"name" is now at
galleryPage.infoProps.authorInfoProps.authorName
"retired" may be gone entirely.
"verified" is now at
galleryPage.infoProps.authorInfoProps.verified
"subscribers" value can be found at
galleryPage.infoProps.subscribeButtonProps.subscribers
Also some slight modification to the pagination:
pgntn = data["pagination"]
could be changed to
pgntn = data["galleryPage"]["paginationProps"]
if pgntn["active"] == pgntn["maxPage"]:
should be changed to
if pgntn["currentPageNumber"] == pgntn["lastPageNumber"]:
the pagination next value also needs to be determined differently. Maybe just adding 1 to
pgntn["currentPageNumber"]
?In testing out potential fixes I noticed some issues with missing
verified
andurl
values, due to contexts where the user account isn't verified and when the account has been disabled (but gallery itself is still active). Probably need more graceful handling of missing values instead of rejecting the download when the gallery still exists, but I'm not a Pythonista and my suggestions on how to do that wouldn't be worth much.The text was updated successfully, but these errors were encountered: