Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xhamster gallery KeyError - 'authorModel' error #6818

Closed
hvhvhv-hvhvhv opened this issue Jan 12, 2025 · 0 comments
Closed

xhamster gallery KeyError - 'authorModel' error #6818

hvhvhv-hvhvhv opened this issue Jan 12, 2025 · 0 comments

Comments

@hvhvhv-hvhvhv
Copy link

hvhvhv-hvhvhv commented Jan 12, 2025

Xhamster galleries fail to download due to KeyError - 'authorModel'. Console verbose output from Windows/cmd.exe and Debian/WSL using the URL from /test/results/xhamster.py included below.

Windows/cmd.exe:

gallery-dl --verbose https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946
[gallery-dl][debug] Version 1.28.4 - Executable (stable/windows)
[gallery-dl][debug] Python 3.8.10 - Windows-10-10.0.22621
[gallery-dl][debug] requests 2.32.3 - urllib3 2.2.3
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[xhamster][debug] Using XhamsterGalleryExtractor for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): xhamster.com:443
[urllib3.connectionpool][debug] https://xhamster.com:443 "GET /photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946 HTTP/11" 200 None
[xhamster][error] An unexpected error occurred: KeyError - 'authorModel'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[xhamster][debug]
Traceback (most recent call last):
  File "gallery_dl\job.pyc", line 152, in run
  File "gallery_dl\extractor\xhamster.pyc", line 43, in items
  File "gallery_dl\extractor\xhamster.pyc", line 53, in metadata
KeyError: 'authorModel'

Debian/WSL:

[gallery-dl][debug] Version 1.28.4 - Git HEAD: d98d3f5
[gallery-dl][debug] Python 3.10.5 - Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.28
[gallery-dl][debug] requests 2.28.2 - urllib3 1.26.11
[gallery-dl][debug] Configuration Files []
[gallery-dl][debug] Starting DownloadJob for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[xhamster][debug] Using XhamsterGalleryExtractor for 'https://xhamster.com/photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): xhamster.com:443
[urllib3.connectionpool][debug] https://xhamster.com:443 "GET /photos/gallery/take-me-to-the-carwash-at-digitaldesire-15860946 HTTP/1.1" 200 None
[xhamster][error] An unexpected error occurred: KeyError - 'authorModel'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[xhamster][debug]
Traceback (most recent call last):
  File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/job.py", line 152, in run
    for msg in extractor:
  File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/extractor/xhamster.py", line 43, in items
    data = self.metadata()
  File "/home/xxx/.pyenv/versions/3.10.5/lib/python3.10/site-packages/gallery_dl/extractor/xhamster.py", line 53, in metadata
    user = self.data["authorModel"]
KeyError: 'authorModel'

It looks like the extractor is looking for the "authorModel" metadata, but Xhamster has changed the payload and no longer includes an authorModel containing the user info for the gallery. Checking the current window.initials value, it looks like some basic user info can be found in galleryPage.infoProps.authorInfoProps with the username itself at galleryPage.infoProps.authorInfoProps.authorName.
The user ID should be switched from:
"id" : text.parse_int(user["id"]),
to
"id" : text.parse_int(user["userID"]),
"url" is now at galleryPage.infoProps.authorInfoProps.authorLink
"name" is now at galleryPage.infoProps.authorInfoProps.authorName
"retired" may be gone entirely.
"verified" is now at galleryPage.infoProps.authorInfoProps.verified
"subscribers" value can be found at galleryPage.infoProps.subscribeButtonProps.subscribers

Also some slight modification to the pagination:
pgntn = data["pagination"]
could be changed to
pgntn = data["galleryPage"]["paginationProps"]

if pgntn["active"] == pgntn["maxPage"]:
should be changed to
if pgntn["currentPageNumber"] == pgntn["lastPageNumber"]:

the pagination next value also needs to be determined differently. Maybe just adding 1 to pgntn["currentPageNumber"]?

In testing out potential fixes I noticed some issues with missing verified and url values, due to contexts where the user account isn't verified and when the account has been disabled (but gallery itself is still active). Probably need more graceful handling of missing values instead of rejecting the download when the gallery still exists, but I'm not a Pythonista and my suggestions on how to do that wouldn't be worth much.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants