README.org

The collector

This project has served its purpose (which was to help making PersianQA Dataset). Therefore, although its not perfect, there won’t be much changes to it (unless someone has a particular issue with it and only I can help with).

Starting a local version

To clone the repo and initialize it (using Poetry), run:

git clone https://github.com/davoodeh/thecollector
cd thecollector
poetry install

To set required environmental variables, simply use the env.example.json file as below:

cp thecollector/env.example.json thecollector/env.json

Then replace the SECRET with a value created by something like the command below:

python -c 'import os, base64; print(base64.b64encode(os.urandom(16)).decode("utf-8"))'

Finally, the server can be run -m:

poetry run python -m thecollector

After that, your instance should be on http://127.0.0.1:5000/.

For more tools/testing you can try checking out local- branches, like so:

git checkout local-$branch