@handle_urls() with item return type #84
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #84 +/- ##
=======================================
Coverage 99.81% 99.82%
=======================================
Files 17 18 +1
Lines 553 583 +30
=======================================
+ Hits 552 582 +30
Misses 1 1
|
|
hey @BurnzZ! I think we can solve "Handling the case of not needing all fields (recommendation for frameworks like scrapy-poet)" separately. These seem to be two independent features. |
BurnzZ
force-pushed
the
handle_urls-with-item
branch
from
4cc42c2 to
3a5499c
Compare
BurnzZ
force-pushed
the
handle_urls-with-item
branch
from
e38bd52 to
de15a86
Compare
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Co-authored-by: Adrián Chaves <adrian@chaves.io>
web_poet/rules.py
Outdated
| If ``instead_of=None``, this simply means that the Page Object assigned in | ||
| the ``use`` parameter will be utilized for all URLs matching the URL pattern | ||
| in ``for_patterns``. However, if ``instead_of=ReplacedPageObject``, then it |
There was a problem hiding this comment.
This doesn't look correct: if instead_of=None, then Page Object won't be used to replace another Page Object, regardless of URL pattern. It might be used on the provided URL pattern if to_return is set, but this has nothing to do with instead_of=None, as it should work regardless of instead_of value.
So, in the example below:
@handle_urls("example.com")
class MyPage(ItemPage):
# ...handle_urls is a pure documentation. You can't use it instead of any other page object: instead_of is not set, so it is unknown which POs can MyPage replace. You also can't use it to return a certain data type, because the item class MyPage can return is unknown.
There was a problem hiding this comment.
@BurnzZ let me send a PR with some doc-related suggestions
I've removed the incorrect parts, and also removed some of the correct parts :) It seems this kind of documentation belongs somewhere else - it describes how the information stored in ApplyRule is used by other components.
Co-authored-by: Kevin Lloyd Bernal <kevin@zyte.com>
Documentation improvements for overrides (apply rules)
|
Thanks @BurnzZ, great work! |
Addresses approach 1 of #77
TODO:
to_returnitem using@handle_urlsOverrideRuleto something else (candidate:ApplyRule)Consider renaming(hopefully to match theOverrideRule.instead_ofto something else@handle_urlsparameter). Or the other way around: renaming@handle_urls'soverridesparameter.handle_urls'sdata_typeto something else (ensure that it's consistent withOverrideRuleas well)Handling the case of not needing all fields (recommendation for frameworks likescrapy-poet)