Skip to content

A high speed spider-bot that crawls through web pages within a web domain. Adapter interfaces are defined to help scale horizontally in distributed systems

Notifications You must be signed in to change notification settings

roumak/web-crawler

About

A high speed spider-bot that crawls through web pages within a web domain. Adapter interfaces are defined to help scale horizontally in distributed systems

Topics

Stars

Watchers

Forks

Languages