CONTRIBUTING.md

How to contribute to huggingface_hub, the GitHub repository?

Everyone is welcome to contribute, and we value everybody's contribution. Code is thus not the only way to help the community. Answering questions, helping others, reaching out and improving the documentations are immensely valuable to the community.

It also helps us if you spread the word: reference the library from blog posts on the awesome projects it made possible, shout out on Twitter every time it has helped you, or simply star the repo to say "thank you".

Whichever way you choose to contribute, please be mindful to respect our code of conduct.

The client library, huggingface_hub

This repository hosts the client library huggingface_hub, which is a frontend to the Hugging Face Hub. this part lives in src/huggingface_hub and tests.

There are many ways you can contribute to this client library:

• Fixing outstanding issues with the existing code;
• Contributing to the examples or to the documentation;
• Submitting issues related to bugs or desired new features.

Submitting a new issue or feature request

Do your best to follow these guidelines when submitting an issue or a feature request. It will make it easier for us to come back to you quickly and with good feedback.

Did you find a bug?

The huggingface_hublibrary is robust and reliable thanks to the users who notify us of the problems they encounter. So thank you for reporting an issue.

First, we would really appreciate it if you could make sure the bug was not already reported (use the search bar on Github under Issues).

Did not find it? :( So we can act quickly on it, please follow these steps:

• Include your OS type and version, the versions of Python, PyTorch and Tensorflow when applicable;
• A short, self-contained, code snippet that allows us to reproduce the bug in less than 30s;
• Provide the full traceback if an exception is raised by copying the text from your terminal in the issue description.

Do you want a new feature?

A good feature request addresses the following points:

1. Motivation first:

• Is it related to a problem/frustration with the library? If so, please explain why and provide a code snippet that demonstrates the problem best.
• Is it related to something you would need for a project? We'd love to hear about it!
• Is it something you worked on and think could benefit the community? Awesome! Tell us what problem it solved for you.

2. Write a full paragraph describing the feature;
3. Provide a code snippet that demonstrates its future use;
4. In case this is related to a paper, please attach a link;
5. Attach any additional information (drawings, screenshots, etc.) you think may help.

If your issue is well written we're already 80% of the way there by the time you post it.

Submitting a pull request (PR)

Before writing code, we strongly advise you to search through the existing PRs or issues to make sure that nobody is already working on the same thing. If you are unsure, it is always a good idea to open an issue to get some feedback.

You will need basic git proficiency to be able to contribute to huggingface_hub. git is not the easiest tool to use but it has the greatest manual. Type git --help in a shell and enjoy. If you prefer books, Pro Git is a very good reference.

Follow these steps to start contributing:

1. Fork the repository by clicking on the 'Fork' button on the repository's page. This creates a copy of the code under your GitHub user account.

2. Clone your fork to your local disk, and add the base repository as a remote. The following command assumes you have your public SSH key uploaded to GitHub. See the following guide for more information.

   $ git clone git@github.com:<your Github handle>/huggingface_hub.git
   $ cd huggingface_hub
   $ git remote add upstream https://github.com/huggingface/huggingface_hub.git

3. Create a new branch to hold your development changes, and do this for every new PR you work on.

   Start by synchronizing your main branch with the upstream/main branch (ore details in the GitHub Docs):

   $ git checkout main
   $ git fetch upstream
   $ git merge upstream/main

   Once your main branch is synchronized, create a new branch from it:

   $ git checkout -b a-descriptive-name-for-my-changes

   Do not work on the master branch.

4. Set up a development environment by running the following command in a virtual environment a conda or a virtual environment you've created for working on this library:

   $ pip install -e ".[dev]"

   (If huggingface_hub was already installed in the virtual environment, remove it with pip uninstall huggingface_hub before reinstalling it in editable mode with the -e flag.)

5. Develop the features on your branch.

   As you work on the features, you should make sure that the test suite passes. You should run the tests impacted by your changes like this (see below an explanation regarding the environment variable):

   $ pytest tests/<TEST_TO_RUN>.py

   For the following commands leveraging the make utility, we recommend using the WSL system when running on Windows. More information here.

   You can also run the full suite with the following command.

   $ make test

   hugginface_hub relies on black and isort to format its source code consistently. You can install pre-commit hooks so that these styles are applied and checked on files that you have touched in each commit:

   pip install pre-commit
   pre-commit install

   You only need to do the above once in your repository's environment. If for any reason you would like to disable pre-commit hooks on a commit, you can pass -n to your git commit command to temporarily disable pre-commit hooks.

   To permanently disable hooks, you can run the following command:

   pre-commit uninstall

   Alternatively, you can apply automatic style corrections and code verifications manually with the following command:

   $ make style

   huggingface_hub also uses flake8 and a few custom scripts to check for coding mistakes. Quality control runs in CI, however you can also run the same checks with:

   $ make quality

   Once you're happy with your changes, add changed files using git add and make a commit with git commit to record your changes locally:

   $ git add modified_file.py
   $ git commit

   Please write good commit messages.

   It is a good idea to sync your copy of the code with the original repository regularly. The following document covers it in length: github documentation

   And here's how you can do it quickly from your git commandline:

   $ git fetch upstream
   $ git rebase upstream/master

   Push the changes to your account using:

   $ git push -u origin a-descriptive-name-for-my-changes

6. Once you are satisfied (and the checklist below is happy too), go to the webpage of your fork on GitHub. Click on 'Pull request' to send your changes to the project maintainers for review.

7. It's ok if maintainers ask you for changes. It happens to core contributors too! So everyone can see the changes in the Pull request, work in your local branch and push the changes to your fork. They will automatically appear in the pull request.

8. Once your changes have been approved, one of the project maintainers will merge your pull request for you.

Checklist

1. The title of your pull request should be a summary of its contribution;
2. If your pull request addresses an issue, please mention the issue number in the pull request description to make sure they are linked (and people consulting the issue know you are working on it);
3. To indicate a work in progress please prefix the title with [WIP], or mark the PR as a draft PR. These are useful to avoid duplicated work, and to differentiate it from PRs ready to be merged;
4. Make sure existing tests pass;
5. Add high-coverage tests. No quality testing = no merge.
6. Due to the rapidly growing repository, it is important to make sure that no files that would significantly weigh down the repository are added. This includes images, videos and other non-text files. We prefer to leverage a hf.co hosted dataset like the ones hosted on hf-internal-testing in which to place these files and reference them by URL. We recommend putting them in the following dataset: huggingface/documentation-images. If an external contribution, feel free to add the images to your PR and ask a Hugging Face member to migrate your images to this dataset.

Tests

An extensive test suite is included to test the library behavior and several examples. Library tests can be found in the tests folder.

The huggingface_hub library's normal behavior is to work with the production Hugging Face Hub. However, for tests, we prefer to run on a staging version. In order to do this, the HUGGINGFACE_CO_STAGING environment variable to 1 when running tests (see setup.cfg config file).

We use pytest in order to run the tests for the library . From the root of the repository they can be run with the following:

$ python -m pytest ./tests

In fact, that's how make test is implemented (without the pip install line)!

You can specify a smaller set of tests in order to test only the feature you're working on.

For example, the following will only run the tests in the test_repository.py file:

$ python -m pytest ./tests/test_repository.py

And the following will only run the tests that include tag in their name:

$ python -m pytest ./tests -k tag

A corner case: testing Spaces

Fully testing Spaces is not possible on staging. We need to use the production environment for it (e.g huggingface.co). To do so, a personal User Access Token has to be set as HUGGINGFACE_PRODUCTION_USER_TOKEN environment variable, specifically for these tests. This value is configured in the Github CI but you need to set it on your machine to run the tests locally. The token requires write permission and a credit card must be set on your account.

Note that if the token is not find, the related tests are skipped.