Skip to content

Latest commit

 

History

History
71 lines (49 loc) · 1.43 KB

README.md

File metadata and controls

71 lines (49 loc) · 1.43 KB

Web Scraping Project with Node.js and Playwright

Table of Contents

  1. Introduction
  2. Features
  3. Requirements
  4. Installation
  5. Usage
  6. Contributing
  7. License

Introduction

This project scrapes content from a Shopee merchant page and individual product pages. The scraped data is then posted to our server via an API.

Features

  • Automates login to Shopee
  • Handles captcha via a Python script
  • Scrapes data from a Shopee merchant page
  • Scrapes details from individual product pages
  • Updates scraped data to the server using an API

Requirements

  • Node.js
  • Playwright
  • Python (for captcha handling)
  • account to scrap

Installation

  1. Clone this repository:

    git clone https://github.com/nsanzimfura-eric/web-scraping.git
  2. Navigate into the project directory:

    cd web-scraping
  3. Install dependencies:

    npm install
  4. Add your .env variables:

    cp .env.sample .env
  5. Update .env with your API endpoints and Shopee account details.

Usage

  1. To start the scraper:
    npm start

Contributing

I, Nsanzimfura Eric contributed to this web-scraping app, and an author.

License

This project is licensed under the MIT License - see the LICENSE.md file for details.