diff --git a/doc/source/conda-forge-guide.rst b/doc/source/conda-forge-guide.rst index 840aad2..6c79e81 100644 --- a/doc/source/conda-forge-guide.rst +++ b/doc/source/conda-forge-guide.rst @@ -6,13 +6,32 @@ How to release conda-forge package .. _create-feedstock: +I already have a conda-forge feedstock. I want to release a new package version. How do I do that? +-------------------------------------------------------------------------------------------------- + +Please skip to :ref:`here ` + I am new to conda-forge. How do I create a conda package? --------------------------------------------------------- -Here, you will learn how to release a conda package distributed through the conda-forge channel in 10 to 15 minutes. This guide assumes you are familiar with a basic clone, fork, and pull request (PR) workflow on GitHub. +Here, you will learn how to release a conda package distributed through the conda-forge channel in 10 to 15 minutes. This guide assumes you are familiar with a basic clone, fork, and pull request workflow on GitHub. + +Overview +^^^^^^^^ + +The process is divided into three steps: + + +1. :ref:`Prepare recipe: ` You will learn to prepare package information in a file called ``meta.yaml`` using our group's cookiecutting template. The file serves as a recipe for building your conda package. The recipe contains the package version, the source code, the dependencies, the license, etc. + +2. :ref:`Upload therecipe: ` Once you have the ``meta.yaml`` generated, you will create a pull request the staged-recipe repository in the conda-forge repository `here `_ + +3. :ref:`Recipe review: ` One of the community members of conda-forge will review your ``meta.yaml`` and provide feedback. Once the recipe is approved, you will have a package available for ``conda install`` automatically, and you will have your own designated feedstock repository that contains ``meta.yaml`` in ``https://github.com/conda-forge/-feedstock``. -Step 1. Prepare ``meta.yaml``. See Appendix 1 to learn more about ``meta.yaml`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. _conda-forge-recipe-prepare: + +1. Prepare conda package recipe in ``meta.yaml`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To generate a package, we first need to generate a "recipe" for the package. The recipe contains the type of programming language, the package version, the source code, the dependencies, and license, etc. This recipe is stored in a file called ``meta.yaml``. @@ -24,42 +43,44 @@ If you are interested in learning more about each component within ``meta.yaml`` 2. Answer the following questions. Default values in parentheses are used if no value is provided. - :github_org: The GitHub organization name. For example, ``diffpy``. + :github_org: The GitHub organization name. For example, ``diffpy``. - :repo_name: The name of the repository. + :repo_name: The name of the repository. - :module_name: The name of the module. + :module_name: The name of the module. - :version: The version of the package. + :version: The version of the package. - :Select: Choose PyPI. PyPI's ``sdist`` containing requirements files, src/tests, and ``pyproject.toml`` + :Select: Choose PyPI. PyPI's ``sdist`` containing requirements files, src/tests, and ``pyproject.toml`` - :project_short_description: A short description of the project + :short_description: A short description of the project - :project_full_description: A full description of the project + :full_description: A full description of the project - :license_file: The license file that is located in your project repository. i.g., ``LICENSE.rst``. + :license_file: The license file that is located in your project repository. i.g., ``LICENSE.rst``. - :recipe_maintainers: You may have multiple maintainers ``sbillinge, bobleesj`` or just ``sbillinge`` + :maintainers: You may have multiple maintainers ``sbillinge, bobleesj`` or just ``sbillinge`` - :build_requirements: copy ``requirements/build.txt`` from the project repo. It should be empty for pure Python packages, otherwise compilers will be required. + :build_requirements: copy ``requirements/build.txt`` from the project repo. It should be empty for pure Python packages, otherwise compilers will be required. - :host_requirements: Copy the following for the ``python >=3.11, setuptools, setuptools-git-versioning >=2.0, pip``. Copy ``requirements/host.txt`` + :host_requirements: Copy the following for the ``python >=3.11, setuptools, setuptools-git-versioning >=2.0, pip``. Copy ``requirements/host.txt`` - :runtime_requirements: copy ``requirements/conda.txt``: + :runtime_requirements: copy ``requirements/conda.txt``: - :testing_requirements: copy ``requirements/test.txt`` + :testing_requirements: copy ``requirements/test.txt`` +Now, you have ``recipes//meta.yaml`` generated. -Now, you have ``recipes//meta.yaml`` is generated. +.. important:: + - For a pure python package, have you removed the ``build`` section under the ``requirements``? See https://github.com/conda-forge/diffpy.utils-feedstock/blob/main/recipe/meta.yaml for example. -- [ ] For a pure python package, have you removed the ``build`` section under the ``requirements``? See https://github.com/conda-forge/diffpy.utils-feedstock/blob/main/recipe/meta.yaml for example. + - Have you double-checked the license file name in ``meta.yaml`` against the license files in the project repository. If you are unsure, please confirm with the repository owner (Prof. Billinge). -- [ ] Have you double-checked the license file name in ``meta.yaml`` against the license files in the project repository. If you are unsure, please confirm with the repository owner (Prof. Billinge). +.. _conda-forge-recipe-upload: -Step 2. Upload ``meta.yaml`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +2. Upload ``meta.yaml`` to conda-forge for initial review +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. Fork https://github.com/conda-forge/staged-recipes and clone your forked repository @@ -81,53 +102,58 @@ Step 2. Upload ``meta.yaml`` 10. After the CI passes, create a new comment: ``@conda-forge/help-python Hello Team, ready for review!`` -Step 3. Wait for review and merge +.. _conda-forge-recipe-review: + + +3. Wait for recipe review +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. Wait for a ``conda-forge`` volunteer reviewer to review your submission. It may take up to one week. 2. Once the PR is merged by the reviewer (1) your package is available on conda-forge, and (2) a new repository will be created under https://github.com/conda-forge/package-name-feedstock/. Example: https://github.com/conda-forge/diffpy.structure-feedstock. -I already have a conda package. How do I distribute a new package version? --------------------------------------------------------------------------- +.. _conda-forge-feedstock-release: + +Use the conda-forge feedstock to release a new version +------------------------------------------------------ -This step assumes there is a new version of Python package released to PyPI. We will use that PyPI source code to generate a new conda package. +We release a new package once we have the ``version`` and ``SHA256`` sections in ``meta.yaml`` in ``https://github.com/conda-forge/-feedstock`` located in the ``main`` branch. The conda-forge team asks to only modify ``meta.yaml``. -Obtain the ``SHA256`` value from `pypi.org `_: +First, we will attain th ``SHA256`` value from `pypi.org `_: -1. Visit the project on PyPI at ``https://pypi.org/project/`` +#. Visit the project on PyPI at ``https://pypi.org/project/`` -2. Click ``Download files`` under ``Navigation`` +#. Click ``Download files`` under ``Navigation`` -3. Click ``view hashes`` under ``Source Distribution`` +#. Click ``view hashes`` under ``Source Distribution`` -4. Copy the ``SHA256`` value +#. Copy the ``SHA256`` value -Create a PR to your conda-forge feedstock: +#. Create a PR to the feedstock repository. -1. Fork the feedstock repository i.g. https://github.com/conda-forge/diffpy.utils-feedstock. +#. If you haven't, fork and clone the forked feedstock repository. -2. Clone the forked repository. +#. Run ``git checkout main && git pull upstream main`` to sync with the main branch. -3. Run ``git checkout main && git pull upstream main`` to sync with the main branch. +#. Run ``git checkout -b `` to create a new branch. -4. Run ``git checkout -b `` to create a new branch. +#. Open ``recipe/meta.yaml``, modify ``set version`` and ``sha256``. -5. Open ``recipe/meta.yaml``, modify 1) ``set version`` and 2) ``sha256``. +#. Run ``git add recipe/meta.yaml && git commit -m "release: ready for "``. -6. Run ``git add recipe/meta.yaml && git commit -m "Release "``. +#. Run ``git push --set-upstream origin ``. -7. Run ``git push --set-upstream origin ``. +#. Create a PR to ``main``, complete the relevant checklists generated in the PR comment. -8. Create a PR to ``main``, complete the relevant checklists generated in the PR comment, and modify ``meta.yaml`` as needed. +#. Wait for the CI to pass and tag Project Owner for review. -9. Wait for the CI to pass and tag Prof. Billinge for review. +#. Once the PR is merged, in 20 to 30 minutes, verify the latest conda-forge package version from the README badge or by visiting ``https://anaconda.org/conda-forge/``. i.e.g, ``https://anaconda.org/conda-forge/diffpy.utils``. -10. Once the PR is merged, verify the latest conda-forge package version from the README badge. -.. _conda-pre-release: +.. _conda-forge-pre-release: -I am familiar with the regular conda release process. How do I do pre-release? +Appendix 1. How do I do pre-release? ------------------------------------------------------------------------------ Generate ``meta.yaml`` by following ``Step 1`` and ``Step 2`` under ``conda-forge: release for the first time`` above. Here are two differences required for pre-release: @@ -147,9 +173,15 @@ To install the pre-release build:: For more, read the documentation for pre-release: https://conda-forge.org/docs/maintainer/knowledge_base/#pre-release-builds +Appendix 2. Add a new admin to the conda-forge feedstock +-------------------------------------------------------- + +Check whether you are an admin listed in the ``meta.yaml`` in the feedstock repository. Create an issue with the title/comment: ``@conda-forge-admin, please add user @username``. Please see an example issue `here `_. + + .. _meta-yaml-info: -Appendix 1. Background info on ``meta.yml`` +Appendix 3. Background info on ``meta.yml`` ------------------------------------------- The ``meta.yaml`` file contains information about dependencies, the package version, the license, the documentation link, and the maintainer(s) of the package. In ``meta.yaml``, there are 3 important keywords under the ``requirements`` section: ``build``, ``host``, and ``run`` that are used to specify dependencies. @@ -163,3 +195,5 @@ The ``meta.yaml`` file contains information about dependencies, the package vers To avoid any confusion, there is a separate YAML section called ``build`` above the ``requirements`` section. This section is for setting up the entire operating system. For more information, please refer to the official documentation: https://conda-forge.org/docs/maintainer/adding_pkgs/#build-host-and-run + +.. _conda-forge-add-admin: diff --git a/doc/source/cookiecutter-guide.rst b/doc/source/cookiecutter-guide.rst index caf8469..0f96b8a 100644 --- a/doc/source/cookiecutter-guide.rst +++ b/doc/source/cookiecutter-guide.rst @@ -60,75 +60,85 @@ We offer the following ways to help guide you through the cookiecutting process: .. Important:: Skip this section and go to :ref:`2. Cookiecutter main workflow ` if you are starting a new project! -1. Fork and clone the the repository. +#. Fork and clone the repository. -2. ``cd`` into the top-level directory of that project. +#. ``cd`` into the top-level directory of that project. -3. Type ``git pull upstream main`` to sync with the main branch. +#. Type ``git pull upstream main`` to sync with the main branch. -4. Double-check that no bug-fix etc. pull-requests are waiting to be merged. If you are a member, check with the project repository owner if you are unsure. +#. Double-check that no bug-fix etc. pull-requests are waiting to be merged. If you are a member, check with the project repository owner if you are unsure. -5. Create a new branch called ``black``. +#. Create a new branch called ``black``. -6. Create ``pyproject.toml``. Copy and paste the following to ``pyproject.toml``. +#. Create ``pyproject.toml``. Copy and paste the following to ``pyproject.toml``. -.. code-block:: bash + .. code-block:: bash + + [tool.black] + line-length = 115 + include = '\.pyi?$' + exclude = ''' + /( + \.git + | \.hg + | \.mypy_cache + | \.tox + | \.venv + | \.rst + | \.txt + | _build + | buck-out + | build + | dist + + # The following are specific to Black, you probably don't want those. + | blib2to3 + | tests/data + )/ + +#. Run ``black src`` in your Terminal. If your source code is in a different directory, replace ``src`` with the appropriate directory path. This will automatically format your code to PEP8 standards given the line-length provided under ``line-length`` above in ``pyproject.toml``. + +#. Add and commit the automatic changes by ``black``. The commit message can be ``git commit -m "style: apply black to src directory with black configured in pyproject.toml"``. + +#. Run ``black .`` Here, you are running black across the entire package directory. Then, test your package with unit tests locally. - [tool.black] - line-length = 115 - include = '\.pyi?$' - exclude = ''' - /( - \.git - | \.hg - | \.mypy_cache - | \.tox - | \.venv - | \.rst - | \.txt - | _build - | buck-out - | build - | dist +#. git add and commit the changes. The commit message can be ``git commit -m "style: apply black to all files in the project directory"``. - # The following are specific to Black, you probably don't want those. - | blib2to3 - | tests/data - )/ +#. Create a pull request into ``main``. The pull request title can be ``cookiecut: Apply black to project directory with no manual edits``. -7. Run ``black src`` in your Terminal. If your source code is in a different directory, replace ``src`` with the appropriate directory path. This will automatically format your code to PEP8 standards given the line-length provided under ``line-length`` above in ``pyproject.toml``. +#. After the ``black`` branch has been merged to ``main``, type ``git checkout main && git pull upstream main`` and create a new branch called ``precommit`` by typing ``git checkout -b precommit``. -8. Add add commit the automatic changes by ``black``. The commit message can be ``git commit -m "style: apply black to src directory after configuring black in pyproject.toml"``. +#. Copy and paste two files of ``.flake8`` `here `_ and ``.pre-commit-config.yaml`` `here `_ to your project directory. Cross-check with https://github.com/diffpy/diffpy.utils. -9. Run ``black .`` Here, you are running black across the entire package directory. Again, ``git add`` and commit the changes. The commit message can be ``git commit -m "style: apply black to all files in the project directory"``. Then, create a pull request into a new branch called ``pre-commit``. +#. Run ``pre-commit run --all-files`` in your Terminal. This will attempt to lint your code such as docstrings, extra spaces, across all file types such as ``.yml``, ``.md``, ``.rst``, etc. However, most likely, you will have to manually fix some of the errors raised by ``flake8``. -10. After the ``black`` branch has been merged, run unit tests to ensure your tests, if there are any, pass locally. +#. Before manually editing, let's first take a look at the changes made by running ``git status`` to get an overview of the files modified and then by running ``git diff `` to see the specific changes. If you do not want the new changes, you can run ``git restore `` to revert the changes. -11. Type ``git checkout main && git pull upstream main`` and create a new branch called ``precommit``. + .. note:: -12. Copy and paste the ``.flake8`` and ``.pre-commit-config.yaml`` files from ``{{ cookiecutter.repo_name }}`` to the top directory level. Cross-check with https://github.com/diffpy/diffpy.utils. + Q1. Do you want to prevent certain automatic modifications on files? You can add the folder or extension to the ``exclude`` section in ``.pre-commit-config.yaml`` with an example shown `here `_. -13. Run ``pre-commit run --all-files``. + Q2. Do you want to ignore certain spelling recommendations by Codespell? Please refer to this section in the FAQ :ref:`here `. -14. Create a PR to ``main``. Mention in the PR that you need to address flake8 errors. +#. At this point, you may have flake8 errors but we want to address them in a separate pull request. Hence, git add and commit and push the automatic changes made by ``precommit`` and create a pull request to ``main``. The commit message can be ``style: apply pre-commit without manual modification`` and the pull request title can be ``cookiecut: Apply pre-commit to project directory with no manual edits``. -15. After the ``precommit`` branch has been merged, sync with ``main`` in Step 11, create a new branch called ``flake8``. +#. After the ``precommit`` branch has been merged to ``main``, run ``git checkout main && git pull upstream main`` and create a new branch called ``flake8`` by typing ``git checkout -b flake8``. If you have many flake8 errors and types, feel free to create one branch for each specific type of error, like ``flake8-length``. -16. Fix flake8 errors manually: +Here are some tips to reduce cognitive overload: - - Tip 1: Start with easier error types to fix, such as line-lengths and "module imported not used", etc. + 1. Start with easier error types to fix, such as line lengths and "module imported but not used", etc. - - Tip 2: Submit periodic commits within a single PR. + 2. Create multiple PRs, each containing a specific theme (e.g., "Fix docstring line-length flake8 errors" using the ``flake8-length`` branch, etc.) to reduce cognitive overload for the reviewer. - - Tip 3: Create multiple PRs, each containing a specific theme (e.g., "Fix docstring line-length flake8 errors" using ``flake8-length`` branch, etc.) to reduce cognitive overload for the reviewer (Simon). + 3. If you are unsure, suppress the flake8 error by adding ``# noqa: `` at the end of the line. For example, ``import numpy as np # noqa: E000`` but make sure you create an issue for this so that you can revisit them after cookiecutting. - - Tip 4: Don't hesitate to reach out for help. +For each `flake8` branch, create a PR request to ``main``. Since you are fixing flake8 errors, the commit message can be ``style: fix flake8 errors`` and the pull request title can be ``cookiecut: Fix flake8 errors``. In each PR, feel free to communicate the remaining flake8 issues in each pull request to track progress. -Once all flake8 errors are fixed, create a pull request to ``main``. Mention in the PR that you need to address the ``pre-commit`` errors. +Congratulations! You have successfully completed the pre-commit workflow. You may proceed to the section to now transform your package structure! .. _cookiecutter-workflow-main: -2. Cookiecutter main workflow +1. Cookiecutter main workflow ----------------------------- If you are migrating an existing project, @@ -169,7 +179,7 @@ If you are here starting a new project, the :ref:`1, Pre-commit workflow ` below. +.. Important:: Skip the rest of Cookiecutter main workflow if you are starting a new project! Proceed to the :ref:`3. API documentation workflow` below. Otherwise, please continue! 3. cd into the new ``diffpy./`` directory (e.g., in our example ``pwd`` would return ``~/dev/diffpy.pdfmorph/diffpy.pdfmorph``) (we will refer to the nested directory as the "**cookiecutter**" directory and ``~/dev/diffpy.pdfmorph/`` as the "**main**" directory). @@ -240,17 +250,17 @@ Make a PR! It will be merged, trust! 4. Final sign-off ----------------- -1. For the ``cookierelease`` activity make a ``.rst`` file by copying ``TEMPLATE.rst`` in the news folder and under "fixed" put ``Repo structure modified to the new diffpy standard`` +#. For the ``cookierelease`` activity make a ``.rst`` file by copying ``TEMPLATE.rst`` in the news folder and under "fixed" put ``Repo structure modified to the new diffpy standard`` -2. If a new Python version has been added under "added" add `Python 3.xx, 3,xx support`. If a previous version has been removed, under "fixed", add a new item `Python 3.xx, 3.xx, support`. +#. If a new Python version has been added under "added" add `Python 3.xx, 3,xx support`. If a previous version has been removed, under "fixed", add a new item `Python 3.xx, 3.xx, support`. -3. Check the `README` and make sure that all parts have been filled in and all links resolve correctly. +#. Check the `README` and make sure that all parts have been filled in and all links resolve correctly. -4. Run through the documentation online and do the same, fix any last typos and make all the links work. To do this the documentation must have been correctly built on a merge to main and enabled on the github.io website. Instructions are [here](https://gitlab.thebillingegroup.com/resources/group-wiki/-/wikis/Maintaining-and-Deploying-Documentation). +#. Run through the documentation online and do the same, fix grammar and make sure all links work. -5. When you are are happy to sign off on the release send a Slack message to Simon saying something like "`OK to release diffpy.`" +#. Follow the instructions on setting up GitHub pages here. -6. Make sure that the codecov secret is set in the GH actions repository secrets. +.. _test-package-locally: Appendix 1. How to test your package locally -------------------------------------------- @@ -296,6 +306,11 @@ To run as a single command: cd doc && make html && open build/html/index.html && cd .. +.. _build-documentation-preview-real-time: + +Real-time preview with Visual Studio Code +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Alternatively, you may render the Sphinx documentation by installing the `Esbonio `_ extension in VS Code. This will allow you to see the changes in real-time and increase productivity. .. _news-file-guide: @@ -388,4 +403,9 @@ To configure ``pre-commit CI``, follow the simple steps below: 2. Select the repository(s). 3. Done! -.. _test-package-locally: +Appendix 6. How to setup GitHub Pages for your package +------------------------------------------------------ + +You have API doc built and tested locally. Now you want to deploy your doc via ``https://org-name/github.io/repo-name`` i.e., https://diffpy.github.io/diffpy.utils using GitHub Pages. + +Go to `Settings` in your repository. diff --git a/doc/source/frequently-asked-questions.rst b/doc/source/frequently-asked-questions.rst index ba1fa23..0bddc8d 100644 --- a/doc/source/frequently-asked-questions.rst +++ b/doc/source/frequently-asked-questions.rst @@ -36,9 +36,9 @@ To ignore a specific line, add it to ``.codespell/ignore_lines.txt``. See the ex .. code-block:: text - ;; src/translation.py - ;; The following single-line comment is written in German. - # Hallo Welt + ;; src/translation.py + ;; The following single-line comment is written in German. + # Hallo Welt To ignore a specific file extension, add ``*.ext`` to the ``skip`` section under ``[tool.codespell]`` in ``pyproject.toml``. For example, to ignore ``.cif`` and ``.dat`` files, use ``skip = "*.cif,*.dat"``. @@ -50,7 +50,7 @@ Release How can I change who is authorized to release a package? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In ``.github/workflows/build-wheel-release-upload.yml``, modify ``github_admin_username`` to the desired GitHub username. This username will be able to authorize the release. +In ``.github/workflows/build-wheel-release-upload.yml``, modify ``github_admin_username`` to the desired GitHub username. This username will be able to authorize the release by pushing the tag as instructed :ref:`here `. How is the package version set and retrieved? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -66,23 +66,23 @@ To include all files under a folder, use ``graft``: .. code-block:: text - graft src - graft tests + graft src + graft tests To include specific file(s), use ``include``: .. code-block:: text - include AUTHORS.txt LICENSE*.txt README.rst + include AUTHORS.txt LICENSE*.txt README.rst To exclude files globally, use ``globally-exclude``: .. code-block:: text - global-exclude *.py[cod] # Exclude all .pyc, .pyo, and .pyd files. - global-exclude .DS_Store # Exclude Mac filesystem artifacts. - global-exclude __pycache__ # Exclude Python cache directories. - global-exclude .git* # Exclude git files and directories. + global-exclude *.py[cod] # Exclude all .pyc, .pyo, and .pyd files. + global-exclude .DS_Store # Exclude Mac filesystem artifacts. + global-exclude __pycache__ # Exclude Python cache directories. + global-exclude .git* # Exclude git files and directories. Why have we decided to include test files in the PyPI source distribution? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -91,6 +91,34 @@ We decided to include test files in the PyPI source distribution to facilitate u The conda-forge CI uses the source code distributed via PyPI to build a Conda package. After building the package, we want to run pytest to ensure all unit tests pass before release. Therefore, test files must be included in the source code. In contrast, no documentation is distributed with the package, as it is already accessible from the GitHub repository and does not serve a practical purpose in the distribution package itself. +Documentation +------------- + +How can I preview documentation in real-time? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You may use Visual Studio Code. Please refer to the following section :ref:`here `. + +How do I re-deploy online documentation without another release? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Visit the following URL of your package: ``https://github.com///actions/workflows/publish-docs-on-release.yml`` i.e., https://github.com/diffpy/diffpy.utils/actions/workflows/publish-docs-on-release.yml. + +Click ``Run workflow`` and select the ``main`` branch. Your online documentation will be updated with the latest changes without a new release. + +conda-forge +----------- + +How do I add a new admin to the conda-forge feedstock? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Please refer to the admin section in the conda-forge release guide :ref:`here `. + +How do I do pre-release for conda-forge? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Please read our pre-release section in the conda-forge release guide :ref:`here `. + GitHub Actions -------------- @@ -103,40 +131,40 @@ The default is Python 3.13 for ``_tests-on-pr.yml`` and ``_publish-docs-on-relea .. code-block:: yaml - jobs: - tests-on-pr: - uses: Billingegroup/release-scripts/.github/workflows/_tests-on-pr.yml@v0 - with: - project: package-name - c_extension: false - headless: false - python_version: 3.12 - secrets: - CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} + jobs: + tests-on-pr: + uses: Billingegroup/release-scripts/.github/workflows/_tests-on-pr.yml@v0 + with: + project: package-name + c_extension: false + headless: false + python_version: 3.12 + secrets: + CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} 2. Add ``python_version`` in ``.github/workflows/_publish-docs-on-release.yml``: .. code-block:: yaml - jobs: - docs: - uses: Billingegroup/release-scripts/.github/workflows/_tests-on-pr.yml@v0 - with: - project: package-name - c_extension: false - headless: false - python_version: 3.12 + jobs: + docs: + uses: Billingegroup/release-scripts/.github/workflows/_tests-on-pr.yml@v0 + with: + project: package-name + c_extension: false + headless: false + python_version: 3.12 3. Add ``python_versions`` in ``.github/workflows/_matrix-and-codecov-on-merge-to-main.yml``: .. code-block:: yaml - jobs: - matrix-coverage: - uses: Billingegroup/release-scripts/.github/workflows/_matrix-and-codecov-on-merge-to-main.yml@v0 - with: - ... - python_versions: "3.11, 3.12" + jobs: + matrix-coverage: + uses: Billingegroup/release-scripts/.github/workflows/_matrix-and-codecov-on-merge-to-main.yml@v0 + with: + ... + python_versions: "3.11, 3.12" What is the difference between ``pull_request`` and ``pull_request_target``? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -145,25 +173,173 @@ For the current GitHub CI for checking a news item, ``pull_request_target`` is u .. code-block:: yaml - name: Check News Item + name: Check News Item - on: - pull_request_target: - branches: - - main + on: + pull_request_target: + branches: + - main - ``pull_request``: This event configures the ``GITHUB_TOKEN`` with read-only permissions by default, especially when triggered by forks. - ``pull_request_target``: This event grants the ``GITHUB_TOKEN`` write permissions, enabling it to perform actions that modify the repository, such as posting comments, updating pull request statuses, or merging code. The news CI creates a comment when an additional news ``.rst`` is not found under the ``news`` folder. Hence, ``pull_request_target`` is used. Another key difference is that with ``pull_request_target``, the ``.yml`` file **must already be merged** in the base branch at the time the pull request is opened or updated. For more, please refer to `GitHub docs `_. - Dependency management --------------------- Why are both pip.txt and conda.txt provided? -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Our preferred choice for installing the cookiecuttered package is as a Conda package, as outlined in the template ``README.rst`` file. With Conda, the end user can install all associated dependencies by running ``conda create --name new_env ``. Additionally, the environment is tested via conda-forge CI before the Conda package is released, which helps ensure the package's compatibility with its dependencies. Hence, we list conda package dependencies in ``conda.txt``. However, we also want to allow users to install the package via ``pip``. To support this, we provide a separate file for pip dependencies, ``pip.txt``. In most cases, the dependencies listed in ``conda.txt`` and ``pip.txt`` will be identical. However, there can be exceptions. For example, ``matplotlib-base`` is preferred for Conda installations, while ``matplotlib`` is used for pip installations. + +GitHub workflow +--------------- + +I am new to GitHub. Why do we use Git/GitHub? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +GitHub allows multiple contributors to work on a software project simultaneously under an organization like ``Billingegroup`` or ``diffpy``. There are two primary needs. First, we want to ensure that any changes under this organization are reviewed by the organization's project owner. Second, we want to ensure we add new changes from the latest version of the code, particularly when working with multiple contributors across different time zones. Hence, we use GitHub to serve the needs with a specific workflow below. Please see below for an overview of the GitHub workflow. + +.. _github-workflow-overview: + +What is the general the workflow? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Since cookiecutting requires a basic understanding of GitHub's workflow, we will provide you with a brief overview and how to set up your repository. + +First, if you are working on a package from an organization like ``github.com/diffpy`` or ``github.com/Billingegroup``, you first copy the repository of the organization to your GitHub user account. This process is called ``forking``. + +Then, you will download the forked repository in your GitHub account to your local machine. This process is called ``cloning``. + +In the cloned repository on your local machine, you will make edits. You want to first add a description for the changes by "committing" with a message describing the changes. Then you will upload these changes to the ``forked`` repository in your account. This process of updating code from the local computer to the repository hosted by GitHub is called ``pushing``. + +From the forked repository, you then want to upload changes to the repository under ``github.com/Billingegroup/cookiecutter``, for example. This process is done through a process called ``pull request``. The Project Owner reviews this pull request and merges it into the Billinge group's repository. If you are the contributor as well as the Project Owner, you would be the one who reviews your own code and merges your changes. + +I have a general understanding of fork, clone, commit, push, and pull request. How do I set up my repository for cookiecutting? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Please be familiar with the terminology such as "fork", "clone", "push", and "pull request" :ref:`above `. + +You may fork the repository using the "Fork" button on the top right corner of the repository page. This will copy the repository to your GitHub account. e.g., ``github.com/Billingegroup/cookiecutter`` to ``github.com/sbillinge/cookiecutter``. + +Then download the forked repository under your account to the local machine by cloning: + +.. code-block:: bash + + git clone https://github.com// + +Now, you also want to link with the repository of the organization by adding the URL. Recall, we want to make changes from the latest state of the source code. + +.. code-block:: bash + + git remote add upstream https://github.com// + +.. note:: + + What is ``upstream``? The repository that you forked from, e.g. ``Billingegroup/cookiecutting`` is referred to as the ``upstream`` repository. + +Verify that you have the ``upstream`` URL set up as the organization. + +.. code-block:: bash + + git remote -v + +Notice that you also have ``origin`` with an URL linking to your forked repository under your account. This is another GitHub jargon that refers to your forked repository. + +.. note:: + + What is ``remote``? The term ``remote`` is the opposite of ``local``. In other words, ``remote`` refers to the repository that is hosted by GitHub. e.g., ``github.com/Billingegroup/cookiecutter`` or ``github.com/sbillinge``. + +Do you have a general summary of each term used in the GitHub workflow? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:fork: The process of copying a repository from an organization to your GitHub account. e.g., ``github.com/Billingegroup/cookiecutter`` to ``github.com/sbillinge/cookiecutter``. + +:upstream: The repository of the original source code. e.g., ``github.com/Billingegroup/cookiecutter``. + +:origin: The forked repository under your account. e.g., ``github.com/sbillinge/cookiecutter``. + +:remote: The repository that is hosted by GitHub. e.g., ``github.com/Billingegroup/cookiecutter`` or ``github.com/sbillinge/cookiecutter``. + +:branch: The branch serves as a folder that contains the files of the repository. The ``main`` branch is the branch that is used for the final version of the code. Many branches can be created for different features or bug fixes that are later merged into the ``main`` branch. + +:git clone: The process of locally downloading a repository from GitHub (``remote``) to your local machine. + +:git push: The process of updating code from the local computer to the GitHub remote repository. Push can be made to the ``origin`` or ``upstream`` repository. But, in our workflow, we push to the ``origin`` repository, and then we create a pull request to merge the changes from ``origin`` to the ``upstream`` repository. + +:git commit: The process of adding a description for the changes made in the files that are ready to be pushed. + +:git add: The process of selecting files to be included within a single commit. + +I have cloned and added ``upstream``. What is the next step? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +We want to first sync our local folder with the ``upstream`` repository. This process is called ``pulling``. + +.. code-block:: bash + + git checkout main + git pull upstream main + +Above, we checkout the ``main`` branch of your cloned folder. We then download all the latest changes from the ``upstream`` repository. Recall that a GitHub repository is contributed by multiple contributors. Hence, we want to ensure that we are working with the latest version of the code in the ``main`` branch. + +Once we are fully synced with the ``upstream`` repository, we can now start making changes to the code. + +Instead of directly working in the ``main`` branch of your cloned repository, you will create a copy of ``main`` by "branching" it from ``main``. Think of a tree. You can name it anything you want like ``docs-faq``, etc. + +.. code-block:: bash + + git checkout -b docs-faq + +The above command not only creates a new branch but also switches to the new branch. You can verify that you are in the new branch by running: + +.. code-block:: bash + + git branch + +Of course, you can always switch back to the ``main`` branch by using ``git checkout main``. + +Now, you are ready to make changes to the code in the branch. If you have a README file in your project, try to modify it. Once you are done, you want to add the changes to a hidden folder called ``.git``. This process is called ``staging``. + +.. code-block:: bash + + git add README.rst + +Then, now you want to commit the changes with a message describing the changes. + +.. code-block:: bash + + git commit -m "docs: added a FAQ section in the README" + +Now, you want to push the changes to the ``origin`` repository under your account. Recall ``origin`` refers to the forked repository under your account hosted by GitHub. + +.. code-block:: bash + + git push --set-upstream origin docs-FAQ + +Go to your forked repository under your account on GitHub. You will see a green button that says "Compare & pull request". Click on it. You will see the changes you made in the branch. Click on "Create pull request". Add a description of the changes you made. Click on "Create pull request". + +The reviewer will review the changes and merge them into the ``upstream`` repository. You have successfully made your first contribution to the organization's repository. + +I still need to make another pull request. How do I do that? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Now, you want to make another pull request. You want to make sure that you are working with the latest version of the code in the ``main`` branch. + +.. code-block:: bash + + git checkout main + git pull upstream main + +The command above will sync your local folder with the ``upstream`` repository. It should download the changes made by other contributors as well as the recent commit you made in the ``docs-FAQ`` branch, for example. + +Again, you checkout a new branch from the ``main`` branch. You can name it anything you want, e.g. ``docs-typo``. + +.. code-block:: bash + + git checkout -b docs-typo + +You repeat the process of git add, commit, push to your ``origin`` (your forked repository) and then make a PR to the ``upstream`` repository (the organization's repository). diff --git a/doc/source/img/github-pages.png b/doc/source/img/github-pages.png new file mode 100644 index 0000000..c79128b Binary files /dev/null and b/doc/source/img/github-pages.png differ diff --git a/doc/source/img/release-issue.png b/doc/source/img/release-issue.png new file mode 100644 index 0000000..5221723 Binary files /dev/null and b/doc/source/img/release-issue.png differ diff --git a/doc/source/release-guide.rst b/doc/source/release-guide.rst index 39c8e1e..2ba52b1 100644 --- a/doc/source/release-guide.rst +++ b/doc/source/release-guide.rst @@ -8,53 +8,55 @@ How to release Python package =============================== -FAQ for PyPI/GitHub release -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Overview +~~~~~~~~ -Q1. No conda-forge feedstock yet? To release a conda-package, you first need to create one. Follow the instructions :ref:`here `. +This guide is written for those who have just cookiecuttered. -Q2. Not a feedstock admin? Create an issue with the title/comment: ``@conda-forge-admin, please add user @username``. Example `issue `_. +If you already have a conda package and if you are interested in just release a new version, read the instruction here. -Q3. Interested in pre-release? Please follow the instructions under the conda-forge: pre-release section :ref:`here `. +PyPI/GitHub release +~~~~~~~~~~~~~~~~~~~~ -Instructions for GitHub repository contributors -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. _release-instructions-contributor: -When you are ready for PyPI and GitHub release, please follow these steps: +Make sure you have your package cookiecuttered. Otherwise, please start from the Getting started page :ref:`here ` -1. In the repository, create an issue on GitHub with the "Release" option. +#. In the repository, create an issue on GitHub with the "Release" option as shown below: -2. Check off all items in the first checklist for PyPI/GitHub release. + .. image:: ./img/add-personal-access-token.png + :alt: add-personal-access-token + :width: 600px -3. After PyPI/GitHub release, create a PR into the conda-forge feedstock. While doing so, check off items in the second checklist. See Appendix 1. - -4. After conda release, complete the third checklist and close the issue. +#. Check off all items in the first checklist for PyPI/GitHub release. Instructions for Project Owner for release ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -1. Review the checklist prepared by the contributor in the issue. +.. _release-instructions-project-owner: + +#. Review the checklist prepared by the contributor in the issue. -2. Ensure ``PYPI_TOKEN`` and ``PAT_TOKEN`` are configured at the org level. See Appendix :ref:`1 `, :ref:`2 `, respectively. +#. Ensure ``PYPI_TOKEN`` and ``PAT_TOKEN`` are configured at the org level. See Appendix :ref:`1 `, :ref:`2 `, respectively. -3. Check the ``github_admin_username`` section in ``.github/workflows/build-wheel-release-upload.yml`` is that of the project owner. +#. Check the ``github_admin_username`` section in ``.github/workflows/build-wheel-release-upload.yml`` is that of the project owner. -4. In your terminal, run ``git checkout main && git pull upstream main`` to sync with the main branch. +#. In your terminal, run ``git checkout main && git pull upstream main`` to sync with the main branch. -5. Run the following:: +#. Run the following:: # For pre-release, use *.*.*rc* i.g. 1.0.0rc0 # For release, use *.*.* i.g. 1.0.0 git tag git push upstream -6. Done! Once the tag is pushed, visit the ``Actions`` tab in the repository to monitor the CI progress. +#. Done! Once the tag is pushed, visit the ``Actions`` tab in the repository to monitor the CI progress. -7. Comment ``GitHub/PyPI release done!`` or ``GitHub/PyPI pre-release done!`` in the release GitHub issue. +#. Comment ``GitHub/PyPI release done!`` or ``GitHub/PyPI pre-release done!`` in the release GitHub issue. -8. The contributor will make a PR into the conda-forge feedstock. Review and approve the PR. +#. The contributor will make a PR into the conda-forge feedstock. Review and approve the PR. -9. Wait for the contributor to test the released package. The issue will be closed by the contributor. +#. Wait for the contributor to test the released package. The issue will be closed by the contributor. .. _pypi-token-setup: @@ -63,25 +65,25 @@ Appendix 1. Setup ``PYPI_TOKEN`` Generate a PyPI API token from ``pypi.org``: -1. Visit https://pypi.org/manage/account/ and log in. +#. Visit https://pypi.org/manage/account/ and log in. -2. Scroll down to the ``API tokens`` section and click ``Add API token``. +#. Scroll down to the ``API tokens`` section and click ``Add API token``. -3. Set the ``Token name`` to ``PYPI_TOKEN``. +#. Set the ``Token name`` to ``PYPI_TOKEN``. -4. Choose the appropriate ``Scope`` for the token. +#. Choose the appropriate ``Scope`` for the token. -5. Click ``Create token`` and copy the generated token. +#. Click ``Create token`` and copy the generated token. Add the generated token to GitHub: -1. Navigate to the ``Settings`` page of the org (or repository). +#. Navigate to the ``Settings`` page of the org (or repository). -2. Click the ``Actions`` tab under ``Secrets and variables``. +#. Click the ``Actions`` tab under ``Secrets and variables``. -3. Click ``New org secret``, name it ``PYPI_TOKEN``, and paste the token value. +#. Click ``New org secret``, name it ``PYPI_TOKEN``, and paste the token value. -4. Done! +#. Done! .. image:: ./img/add-pypi-secret.png :alt: add-pypi-secret @@ -90,7 +92,7 @@ Add the generated token to GitHub: .. _pat-token-setup: Appendix 2. Setup ``PAT_TOKEN`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The GitHub workflow needs to link with this privilege through a personal access token (PAT) of the project owner. @@ -121,3 +123,18 @@ Copy and paste the ``PAT_TOKEN`` to your GitHub organization: 3. Click ``New organization secret`` and add a new secret and name it as ``PAT_TOKEN``. 4. Done! + +.. _gh-pages-setup: + +Appendix 3. How to host your package documentation online +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You have API doc built and tested locally. Now you want to deploy your doc via i.e., ``https://diffpy.github.io/diffpy.utils`` using GitHub Pages. + +#. Go to your ``Settings`` page in your repository and and click ``pages`` under ``Code and automation``. +#. Click ``Deploy from a branch`` under ``Source``. Below, choose ``gh-pages`` branch and ``/(root)`` and click ``Save``. + +.. image:: ./img/github-pages.png + :alt: setup-github-pages-from-branch + +Done! Wait a few minutes and visit your GitHub Pages URL!