Skip to content

Commit

Permalink
Add wrapper for parse method that moved
Browse files Browse the repository at this point in the history
This is due to scrapy/scrapy#4254

Sollution copied from meilisearch/docs-scraper#67
Thanks @renehernandez!
  • Loading branch information
nflx committed Sep 19, 2022
1 parent 0e480ea commit cdb65b4
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions scraper/src/documentation_spider.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,9 @@ def start_requests(self):
},
errback=self.errback_alternative_link)

def parse(self, response, **kwargs):
return super()._parse(response, **kwargs)

def add_records(self, response, from_sitemap):
records = self.strategy.get_records_from_response(response)
self.typesense_helper.add_records(records, response.url, from_sitemap)
Expand Down

0 comments on commit cdb65b4

Please sign in to comment.