[deviantart] add extractor for status updates #3541
Conversation
|
Well, this is even more complicated than I thought, so I think I'll call it a day. As I explained in #3539 (comment), the |
ClosedPort22
force-pushed
the
da-status
branch
from
d0a7da5 to
f43dab7
Compare
extract user status updates using the '/user/statuses/' endpoint
- recursively yield statuses - ignore items with missing or unexpected field(s)
ClosedPort22
force-pushed
the
da-status
branch
from
f43dab7 to
013733c
Compare
| alphabet = "0123456789abcdefghijklmnopqrstuvwxyz" | ||
| if "index" not in deviation: | ||
| try: | ||
| deviation["index"] = text.parse_int( | ||
| deviation["url"].rpartition("-")[2]) | ||
| if deviation["url"].startswith("https://sta.sh"): | ||
| filename = deviation["content"]["src"].split("/")[5] | ||
| deviation["index_base36"] = filename.partition("-")[0][1:] | ||
| deviation["index"] = \ | ||
| util.bdecode(deviation["index_base36"], alphabet) | ||
| else: | ||
| deviation["index"] = text.parse_int( | ||
| deviation["url"].rpartition("-")[2]) | ||
| except KeyError: | ||
| deviation["index"] = 0 | ||
| deviation["index_base36"] = "0" | ||
| if "index_base36" not in deviation: | ||
| deviation["index_base36"] = \ | ||
| util.bencode(deviation["index"], alphabet) |
There was a problem hiding this comment.
Why is this part of this PR? From what I can tell, this is not at all necessary for status updates.
There was a problem hiding this comment.
This is used to handle shared sta.sh deviations in status updates (see #3539 (comment)). I moved the code to the parent extractor because it's also possible to extract embedded sta.sh deviations in journals, and I plan to add this feature in the future.
| def comments(self, id, target, offset=0): | ||
| """Fetch comments posted on a target""" | ||
| endpoint = "/comments/{}/{}".format(target, id) |
There was a problem hiding this comment.
I'd have preferred to have one DeviantartOAuthAPI method per API endpoint, but I guess this works as well.
There was a problem hiding this comment.
This method can also be used to fetch comments posted on a user's profile.
- relax regex pattern - handle invalid 'items' field - add a test for shared sta.sh item Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
| @@ -802,7 +809,7 @@ def deviations(self): | |||
| yield from self.status(status) | |||
|
|
|||
| def status(self, status): | |||
| for item in status.get("items", ()): # do not trust is_share | |||
| for item in status.get("items") or (): # do not trust is_share | |||
There was a problem hiding this comment.
What's the point of this change ?
There was a problem hiding this comment.
See #3541 (comment)
This crashes (TypeError: 'NoneType' object is not iterable)
status = {"items": None}
for item in status.get("items", ()):
passThis does not
status = {"items": None}
for item in status.get("items") or ():
passThere was a problem hiding this comment.
Do I even wanna know why the Py core devs thought this was a good idea ? Surely they coulda included the or ... functionality in the .get.
Partially resolves #3539
1/3 done