Libretexts.org scraper

This scraper downloads libretexts.org courses and puts them in ZIM files, a clean and user friendly format for storing content for offline usage.

Installation

There are three main ways to install and use libretexts2zim from most recommended to least:

Install using a pre-built container

1. Download the image using docker:

   docker pull ghcr.io/openzim/libretexts

Build your own container

1. Clone the repository locally:

   git clone https://github.com/openzim/libretexts.git && cd libretexts

2. Build the image:

   docker build -t ghcr.io/openzim/libretexts .

Run the software locally using Hatch

1. Clone the repository locally:

   git clone https://github.com/openzim/libretexts.git && cd libretexts

2. Install Hatch:

   pip3 install hatch

3. Start a hatch shell to install software and dependencies in an isolated virtual environment.

   hatch shell

4. Run the libretexts2zim command:

   libretexts2zim --help

Usage

Warning

This project is still a work in progress and isn't ready for use yet, the commands below are examples only.

# Get help
docker run -v output:/output ghcr.io/openzim/libretexts libretexts2zim --help

# Create a ZIM for https://geo.libretexts.org
docker run -v output:/output ghcr.io/openzim/libretexts libretexts2zim --library-slug geo --library-name Geosciences

Developing

Use the commands below to set up the project once:

# Install hatch if it isn't installed already.
❯ pip install hatch

# Local install (in default env) / re-sync packages
❯ hatch run pip list

# Set-up pre-commit
❯ pre-commit install

The following commands can be used to build and test the scraper:

# Show scripts
❯ hatch env show

# linting, testing, coverage, checking
❯ hatch run lint:all
❯ hatch run lint:fixall

# run tests on all matrixed' envs
❯ hatch run test:run

# run tests in a single matrixed' env
❯ hatch env run -e test -i py=3.12 coverage

# run static type checks
❯ hatch env run check:all

# building packages
❯ hatch build

Contributing

This project adheres to openZIM's Contribution Guidelines.

This project has implemented openZIM's Python bootstrap, conventions and policies v1.0.3.

See details for contributions in CONTRIBUTING.md.