Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

一个 Golang 实现的相对智能、无需规则维护的通用新闻网站数据提取工具库。含域名探测、网页编码语种识别、网页链接分类提取、网页新闻要素抽取以及新闻正文抽取等组件。

Linkrot checks for broken links on a given website

GoCrawler is a web crawling framework written in Go, inspired by Scrapy. Support this project at https://ko-fi.com/gocrawl

从网页上捉取美女图片

A web spider written in Golang to Scrap Webpages and Index them. Concurrency is highly used