Skip to content

srinivasreddy/companies-list

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

43 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

About

This project extracts Fortune - 1000 companies data from a hidden url of the Fortune website.

Usage

import { Crawler } from "companies-list";
async function start() {
    const crawler = new Crawler(2019, "./srinivas.csv");
    await crawler.generateCSVFile();
}
start();

TODO

  1. Implement for any year. Does API support this? And year should be configureable too.
  2. Option to implement proxy server(s)
  3. csv fields generated should be configurable?
  4. Implement persistance to crawl where it left off?
  5. Dockerize this application

License

MIT