A web scraping platform for the AI era.
git clone https://github.com/Datasilk/Collector
git submodule init
git submodule update --init
cd Collector/Collector.App
npm install
I started this project in 2015 by building Charlotte along with a web UI in ASP.NET Core using C#. I've built this project from the ground up several times, eventually transforming it into a plugin for Saber (a website builder), and so now I am turning it into a set of tools.
The new Collector app will be a .NET command-line-based web server with a web UI for managing all your collections of data.
A common .NET library that contains all the common functionality of the Collector App so that you can build your own app to collect data from the web.
A .NET command-line tool for scraping YouTube based on what kind of content you are looking for. Uses AI to build search queries for YouTube.