"write-metadata" not working for deviantart #252

ReneRanzinger · 2019-05-09T03:58:18Z

Using the following command:

gallery-dl.exe -u xxx -p xxx --write-metadata --verbose https://www.deviantart.com/cryptid-creations

This error happens:

[gallery-dl][debug] Version 1.8.3
[gallery-dl][debug] Python 3.7.3 - Windows-7-6.1.7601-SP1
[gallery-dl][debug] requests 2.21.0 - urllib3 1.24.3
[gallery-dl][debug] Starting DownloadJob for 'https://www.deviantart.com/cryptid-creations'
[gallery-dl][debug] updating default urllib3 ciphers
[deviantart][debug] Using DeviantartGalleryExtractor for 'https://www.deviantart.com/cryptid-creations'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.deviantart.com:443
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/user/profile/cryptid-creations HTTP/1.1" 200 3060
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/gallery/all?username=cryptid-creations&offset=0&limit=24&mature_content=true HTTP/1.1" 200 None
[deviantart][debug] Active postprocessor modules: [<gallery_dl.postprocessor.metadata.MetadataPP object at 0x031D0650>]
[urllib3.connectionpool][debug] https://www.deviantart.com:443 "GET /api/v1/oauth2/deviation/download/425FED0A-3860-06FA-1B94-ECCA3008A3EF?mature_content=true HTTP/1.1" 200 1246
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): s3.amazonaws.com:443
[urllib3.connectionpool][debug] https://s3.amazonaws.com:443 "GET /origin-orig.deviantart.net/ef50/f/2019/128/9/3/daily_paint_2360__webcam_by_cryptid_creations-dd6gyft.png?X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Security-Token=AgoJb3JpZ2luX2VjEFsaCXVzLXdlc3QtMiJHMEUCIHUF8OzUXilUhb3tyh%2Fz%2BDNCA7mbyA1h8%2FbmVBI%2BrNzYAiEApLcaLevIDhE7PfnVSIg1rbcj1HALuXG052vlVr58JoEq2gMIVBAAGgwwMDg5NzQzOTIxMzEiDLGJnGQCzSkiX5nJASq3A5JnEdLbIcIZMaEfE0oKJVXDht0yI3dZLsvmAdawxALLX2z0xFpiYvaWZsZB5kpE%2Fs0%2Ftxqg%2Bkd8UdnVg5ooVTei6psJIYg85DQALZvodrzGlU8sYnX5joYI1T1f%2FKnmcMCQzR6PLjNWWa2ctyUGz42BYaDLD8PKUj%2B9wY7BO5YkCGCJTFkVknfh%2B70At5EBgsYgRPSghkBRTdjWh9gkMlgOqnxYlTG9B6efWBP4yu11EUJVWOfUEG4Ue%2FHOXGmNiwwUt3vnHfhiVyRbHR4UXI%2BKPWgATtsJ7Q8%2BtXef%2FjftB4D51XOQkgJl4tAiaqAx43EsG%2F5oRbtXhhHMNYLZ51p%2F3K7PeeOia64J8WfAaRzE1R3c8jHzWh2Eb3ziaLHhcPhlXzf49pvTrAoPBaOubULWGCBt%2B%2Bee%2BdB31sO0hibvFLSUcxd2cu5LJxgtPzbmTMzFQZYg6W%2BvQlMZtyzPJp8OhHxp0TCYp6R%2FYIzNvc8tjC4E4QFyNo%2B6RupgPP50kvG%2FzHxC5jQ%2Fg9lfJ3W1d3k%2FbkreYCw53DOjxx4rbcPOptLlFIbyFB7XxYutbZut%2F0fAcOdohBwwwa3O5gU6tAGF7qFsocXW21S9wLQ2ciwdFoiS5BOxtxRmyf%2BZRPQK8P093k7pFZrKnW%2Fq7hLNcFENgGMROmTnZ2Lu47oBHHLbFQDzde%2BjdM1r7ZrMKzZhvSHDM%2BsSWWVbD6JUnh66GoAva843wkHo5IkW22Z6HEsZOa6koXNTonBmtPvYLDiyPahkZkPbVprpRp7x2vjOWvuTwJePez5OaVqd9uLxAICHcSHxrX2lmAcaMm5cPi79jD2pzlQ%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQEFXKLNB5Y27WGZY%2F20190509%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190509T034046Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Signature=dbe2fe6b9453a655e61db3bee19fc9a7f1bd3639bcb4173a2040e9e99ee555a3 HTTP/1.1" 416 None
[deviantart][error] An unexpected error occurred: TypeError - Object of type datetime is not JSON serializable. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[deviantart][debug] 
Traceback (most recent call last):
  File "gallery_dl\job.py", line 55, in run
  File "gallery_dl\job.py", line 99, in dispatch
  File "gallery_dl\job.py", line 230, in handle_url
  File "gallery_dl\postprocessor\metadata.py", line 40, in run
  File "gallery_dl\postprocessor\metadata.py", line 69, in _write_json
  File "json\__init__.py", line 179, in dump
  File "json\encoder.py", line 431, in _iterencode
  File "json\encoder.py", line 405, in _iterencode_dict
  File "json\encoder.py", line 438, in _iterencode
  File "json\encoder.py", line 179, in default
TypeError: Object of type datetime is not JSON serializable

The text was updated successfully, but these errors were encountered:

iamleot · 2019-05-09T08:05:50Z

Hello Rene, Rene Ranzinger writes:

Using the following command: `gallery-dl.exe -u xxx -p xxx --write-metadata --verbose https://www.deviantart.com/cryptid-creations ` This error happens: [...] [deviantart][debug] Traceback (most recent call last): File "gallery_dl\job.py", line 55, in run File "gallery_dl\job.py", line 99, in dispatch File "gallery_dl\job.py", line 230, in handle_url File "gallery_dl\postprocessor\metadata.py", line 40, in run File "gallery_dl\postprocessor\metadata.py", line 69, in _write_json File "json\__init__.py", line 179, in dump File "json\encoder.py", line 431, in _iterencode File "json\encoder.py", line 405, in _iterencode_dict File "json\encoder.py", line 438, in _iterencode File "json\encoder.py", line 179, in default TypeError: Object of type datetime is not JSON serializable ` [...]

Pull request #251 is a possible fix for it (the problem is not specific to `deviantart` extractor but to extractors that have `datetime` in metadata).

Simplified universal serialization support in json.dump() can be achieved by passing 'default=str', which was already the case in DataJob.run() for -j/--dump-json, but not for the 'metadata' post-processor. This commit introduces util.dump_json() that (more or less) unifies the JSON output procedure of both --write-metadata and --dump-json. (#251, #252)

ReneRanzinger · 2019-05-09T14:57:32Z

Thanks for the quick fix. I am actual not a Python guy and only use the Windows compilation. Do you know when you plan to make the next release version? Best, René

________________________________________ From: Leonardo Taccari <notifications@github.com> Sent: Thursday, May 9, 2019 4:08:54 AM To: mikf/gallery-dl Cc: Rene Ranzinger; Author Subject: Re: [mikf/gallery-dl] "write-metadata" not working for deviantart (#252) Hello Rene, Rene Ranzinger writes:

Using the following command: `gallery-dl.exe -u xxx -p xxx --write-metadata --verbose https://www.deviantart.com/cryptid-creations ` This error happens: [...] [deviantart][debug] Traceback (most recent call last): File "gallery_dl\job.py", line 55, in run File "gallery_dl\job.py", line 99, in dispatch File "gallery_dl\job.py", line 230, in handle_url File "gallery_dl\postprocessor\metadata.py", line 40, in run File "gallery_dl\postprocessor\metadata.py", line 69, in _write_json File "json\__init__.py", line 179, in dump File "json\encoder.py", line 431, in _iterencode File "json\encoder.py", line 405, in _iterencode_dict File "json\encoder.py", line 438, in _iterencode File "json\encoder.py", line 179, in default TypeError: Object of type datetime is not JSON serializable ` [...]

Pull request #251 contain a possible fix for it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#252 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AARJT6B37OJZ23BUUPVNE6LPUPLZNANCNFSM4HLW55YA>.

mikf · 2019-05-09T15:33:35Z

Do you know when you plan to make the next release version?

The next "regular" release would be Friday next week, but I can do one this weekend as well.

As a side note: -u and -p have no effect for DeviantArt. You need to use OAuth if you want to be "logged in", but that is only neccessary if you want to access "private/hidden" content.

ReneRanzinger · 2019-05-09T20:31:53Z

Thanks for the OAuth hint. I will have to look into this more. Just starting to explore the configuration file. Release: If you can make a new release that is appreciated but I can also wait one more week.

…

________________________________________ From: Mike Fährmann <notifications@github.com> Sent: Thursday, May 9, 2019 11:33:37 AM To: mikf/gallery-dl Cc: Rene Ranzinger; Author Subject: Re: [mikf/gallery-dl] "write-metadata" not working for deviantart (#252) Do you know when you plan to make the next release version? The next "regular" release would be Friday next week, but I can do one this weekend. As a side note: -u and -p have no effect for DeviantArt. You need to use OAuth<https://github.com/mikf/gallery-dl#oauth> if you want to be "logged in", but that is only neccessary if you want to access "private/hidden" content. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#252 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AARJT6A5D2WE4CU7GJR62ZDPUQ75DANCNFSM4HLW55YA>.

mikf · 2019-05-17T22:06:03Z

New release:

mikf added the bug label May 9, 2019

mikf closed this as completed May 17, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"write-metadata" not working for deviantart #252

"write-metadata" not working for deviantart #252

ReneRanzinger commented May 9, 2019 •

edited by mikf

Loading

iamleot commented May 9, 2019 via email •

edited

Loading

ReneRanzinger commented May 9, 2019 via email

mikf commented May 9, 2019 •

edited

Loading

ReneRanzinger commented May 9, 2019 via email

mikf commented May 17, 2019

"write-metadata" not working for deviantart #252

"write-metadata" not working for deviantart #252

Comments

ReneRanzinger commented May 9, 2019 • edited by mikf Loading

iamleot commented May 9, 2019 via email • edited Loading

ReneRanzinger commented May 9, 2019 via email

mikf commented May 9, 2019 • edited Loading

ReneRanzinger commented May 9, 2019 via email

mikf commented May 17, 2019

ReneRanzinger commented May 9, 2019 •

edited by mikf

Loading

iamleot commented May 9, 2019 via email •

edited

Loading

mikf commented May 9, 2019 •

edited

Loading