Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving away from upcoming deprecated functionalities of web-poet==0.6.0 #89

Merged
merged 9 commits into from
Nov 21, 2022

Conversation

BurnzZ
Copy link
Contributor

@BurnzZ BurnzZ commented Oct 14, 2022

Migration away from the deprecated functionalities of web-poet as introduced in scrapinghub/web-poet#84.

This PR should not receive any DeprecationWarning from web-poet.

TODO:

  • Update tox.ini and setup.py with the new web-poet version in PyPI

@BurnzZ BurnzZ requested review from Gallaecio, kmike and wRAR October 14, 2022 10:28
@BurnzZ BurnzZ mentioned this pull request Oct 14, 2022
7 tasks
@codecov
Copy link

codecov bot commented Oct 14, 2022

Codecov Report

Merging #89 (5cbd70a) into master (83a3cd8) will decrease coverage by 0.10%.
The diff coverage is 100.00%.

❗ Current head 5cbd70a differs from pull request most recent head 4881d6f. Consider uploading reports for the commit 4881d6f to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #89      +/-   ##
==========================================
- Coverage   97.88%   97.78%   -0.11%     
==========================================
  Files          12       10       -2     
  Lines         473      451      -22     
==========================================
- Hits          463      441      -22     
  Misses         10       10              
Impacted Files Coverage Δ
scrapy_poet/api.py 100.00% <ø> (ø)
scrapy_poet/injection.py 98.95% <ø> (ø)
scrapy_poet/cache.py 92.30% <100.00%> (ø)
scrapy_poet/overrides.py 95.12% <100.00%> (+0.12%) ⬆️
scrapy_poet/__init__.py
scrapy_poet/utils.py

@BurnzZ BurnzZ changed the title Moving away from upcoming deprecated functionalities of web-poet Support web-poet's new "to_return" parameter in ApplyRule Oct 14, 2022
@BurnzZ BurnzZ changed the title Support web-poet's new "to_return" parameter in ApplyRule Moving away from upcoming deprecated functionalities of web-poet Oct 14, 2022
CHANGELOG.rst Outdated Show resolved Hide resolved
@BurnzZ BurnzZ changed the title Moving away from upcoming deprecated functionalities of web-poet Moving away from upcoming deprecated functionalities of web-poet==0.6.0 Nov 21, 2022
@@ -16,7 +16,8 @@ def __getitem__(self, fingerprint: str) -> Any:
def __setitem__(self, fingerprint: str, value) -> None:
pass

def close(self):
@abc.abstractmethod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why should it be an abstractmethod? Empty close seems like a good default, in case a backend doesn't need to close anything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted it. 4a8807a


logger = logging.getLogger(__name__)

RuleAsTuple = Union[Tuple[str, Callable, Callable], List]
RuleFromUser = Union[RuleAsTuple, OverrideRule]
PageObject = Type[ItemPage]
Copy link
Member

@kmike kmike Nov 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks a bit confusing, with x: PageObject annotation I'd expect x to be a page object instance, not a subclass of ItemPage. It seems that removing this shortcut (not even renaming it) can make the code easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 4a8807a

tox.ini Outdated
@@ -38,7 +38,7 @@ deps =

[testenv:mypy]
deps =
mypy==0.790
mypy==0.982
Copy link
Member

@kmike kmike Nov 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it make sense to update directly to the latest mypy version (0.991)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, as the new version picked up some typing inconsistencies. 4881d6f

@kmike kmike merged commit c3686a9 into master Nov 21, 2022
@kmike
Copy link
Member

kmike commented Nov 21, 2022

Thanks @BurnzZ!

@wRAR wRAR deleted the web-poet-update branch October 31, 2023 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants