Skip to content

Commit

Permalink
Merge pull request #18 from ornlneutronimaging/qa
Browse files Browse the repository at this point in the history
advance main to qa
  • Loading branch information
KedoKudo authored Jun 27, 2024
2 parents d41f17a + 5adb666 commit 384a7e6
Show file tree
Hide file tree
Showing 69 changed files with 5,883 additions and 472 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
echo "versioningit $(versioningit ../)"
# build the package
VERSION=$(versioningit ../) conda mambabuild --channel conda-forge --output-folder . .
conda verify noarch/mypackagename*.tar.bz2
conda verify noarch/bm3dornl*.tar.bz2
- name: upload conda package to anaconda
shell: bash -l {0}
if: startsWith(github.ref, 'refs/tags/v')
Expand All @@ -44,4 +44,4 @@ jobs:
CONDA_LABEL="main"
if [ "${IS_RC}" = "true" ]; then CONDA_LABEL="rc"; fi
echo pushing ${{ github.ref }} with label $CONDA_LABEL
anaconda upload --label $CONDA_LABEL conda.recipe/noarch/mypackagename*.tar.bz2
anaconda upload --label $CONDA_LABEL conda.recipe/noarch/bm3dornl*.tar.bz2
30 changes: 27 additions & 3 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ jobs:
echo "installing additional dependencies if cannot be installed from conda"
- name: run unit tests
run: |
echo "running unit tests"
python -m pytest --cov=src --cov-report=xml --cov-report=term-missing tests/
echo "running unit tests (CPU)"
python -m pytest --cov=src --cov-report=xml --cov-report=term-missing -m "not cuda_required"
- name: upload coverage to codecov
uses: codecov/codecov-action@v4
with:
Expand All @@ -43,4 +43,28 @@ jobs:
# below to build the conda package in your local machine
CHANNELS="--channel mantid/label/main --channel conda-forge"
VERSION=$(versioningit ../) conda mambabuild $CHANNELS --output-folder . .
conda verify noarch/mypackagename*.tar.bz2
conda verify noarch/bm3dornl*.tar.bz2
linux_gpu:
runs-on: github-gpu-builder
defaults:
run:
shell: bash -l {0}
steps:
- uses: actions/checkout@v3
- uses: conda-incubator/setup-miniconda@v2
with:
miniforge-version: "latest"
auto-update-conda: true
channels: conda-forge,defaults
mamba-version: "*"
environment-file: environment.yml
cache-environment-key: ${{ runner.os }}-env-${{ hashFiles('**/environment.yml') }}
cache-downloads-key: ${{ runner.os }}-downloads-${{ hashFiles('**/environment.yml') }}
- name: install additional dependencies
run: |
echo "installing additional dependencies if cannot be installed from conda"
- name: run unit tests
run: |
echo "running unit tests (GPU)"
python -m pytest --cov=src --cov-report=xml --cov-report=term-missing -m "cuda_required"
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,8 @@ cython_debug/
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
.envrc
src/bm3dornl/_version.py
.vscode/settings.json
tmp/*
dev*.ipynb
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "tests/bm3dornl-data"]
path = tests/bm3dornl-data
url = git@code.ornl.gov:sns-hfir-scse/infrastructure/test-data/bm3dornl-data.git
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ repos:
- id: trailing-whitespace
exclude: "tests/cis_tests/.*"
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.3.7
rev: v0.4.10
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
Expand Down
121 changes: 121 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Contributing to BM3D-ORNL

Thank you for considering contributing to the BM3D-ORNL project!
We welcome contributions from the community and are grateful for your help in improving this library.
This guide provides instructions on how to contribute to the project.

## Table of Contents

- [Code of Conduct](#code-of-conduct)
- [Getting Started](#getting-started)
- [Development Workflow](#development-workflow)
- [Coding Standards](#coding-standards)
- [Testing](#testing)
- [Submitting Changes](#submitting-changes)
- [Reporting Issues](#reporting-issues)
- [Contact](#contact)

## Code of Conduct

By participating in this project, you agree to abide by the [Code of Conduct](CODE_OF_CONDUCT.md).

## Getting Started

1. **Fork the Repository**: Fork the [bm3dornl repository](https://github.com/ornlneutronimaging/bm3dornl) to your GitHub account.

2. **Clone the Fork**: Clone your forked repository to your local machine.

```bash
git clone https://github.com/your-username/bm3dornl.git
cd bm3dornl
```

3. **Set Upstream Remote**: Add the original repository as an upstream remote.

```bash
git remote add upstream https://github.com/ornlneutronimaging/bm3dornl.git
```

4. **Create a Virtual Environment**: Set up a virtual environment to manage dependencies.

```bash
micromamba create -f environment.yml
micromamba activate bm3dornl
```

## Development Workflow

- **Create a Branch**: Create a new branch for your feature or bugfix.

```bash
git checkout -b feature/your-feature-name
```

- **Make Changes**: Make your changes in the codebase. Use `pre-commit` to help you format your code and check for common issues.

```bash
pre-commit install
```

> Note: you only need to run `pre-commit install` once. After that, the pre-commit checks will run automatically before each commit.

- **Write Tests**: Write tests for your changes to ensure they are well-tested. See the [testing](#testing) section for more details.

- **Commit Changes**: Commit your changes with a meaningful commit message.

```bash
git add .
git commit -m "Description of your changes"
```

- **Push Changes**: Push your changes to your forked repository.

```bash
git push origin feature/your-feature-name
```

- **Open a Pull Request**: Open a pull request (PR) from your forked repository to the `next` branch of the original repository. Provide a clear description of your changes and any relevant information.

## Coding Standards

- **PEP 8**: Follow the PEP 8 style guide for Python code.
- **Docstrings**: Use `numpy` docstrings style to document all public modules, classes, and functions.
- **Type Annotations**: Use type annotations for function signatures.
- **Imports**: Group imports into standard library, third-party, and local module sections. Use absolute imports.

## Testing

We use `pytest` for testing. Ensure that your changes are covered by tests.

- **Run Tests**: Run the tests using `pytest`.

```bash
pytest -v
```

- **Check Coverage**: Check the test coverage.

```bash
pytest --cov=src/bm3dornl
```

## Submitting Changes

1. **Ensure Tests Pass**: Make sure all tests pass and the coverage is satisfactory.

2. **Update Documentation**: If your changes affect the documentation, update the relevant sections.

3. **Open a Pull Request**: Open a pull request with a clear description of your changes. Reference any related issues in your PR description.

4. **Review Process**: Your pull request will be reviewed by the maintainers. Be prepared to make changes based on feedback.

## Reporting Issues

If you find a bug or have a feature request, please open an issue on the [GitHub issues page](https://github.com/ornlneutronimaging/bm3dornl/issues).
Provide as much detail as possible, including steps to reproduce the issue if applicable.

## Contact

If you have any questions or need further assistance, please contact the [repo maintainer](zhangc@ornl.gov).

Thank you for contributing to BM3D-ORNL!
121 changes: 10 additions & 111 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,114 +1,13 @@
# python_project_template
This repository is a template repository for Python projects under neutrons.
After you create a new repository using this repo as template, please follow the following steps to adjust it for the new project.
[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/ornlneutronimaging/bm3dornl/next.svg)](https://results.pre-commit.ci/latest/github/ornlneutronimaging/bm3dornl/next)
[![Documentation Status](https://readthedocs.org/projects/bm3dornl/badge/?version=latest)](https://bm3dornl.readthedocs.io/en/latest/?badge=latest)

## Codebase Adjustments
BM3D ORNL
=========

1. Adjust the branch protection rules for the new repo. By default, we should protect the `main` (stable), `qa` (release candidate), and `next` (development) branches.
This repository contains the BM3D ORNL code, which is a Python implementation of the BM3D denoising algorithm. The BM3D algorithm was originally proposed by K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian in the paper "Image Denoising by Sparse 3D Transform-Domain Collaborative Filtering" (2007).
The BM3D algorithm is a state-of-the-art denoising algorithm that is widely used in the image processing community.
The BM3D ORNL code is a Python implementation of the BM3D algorithm that has been optimized for performance using both `Numba` and `CuPy`.
The BM3D ORNL code is designed to be easy to use and easy to integrate into existing Python workflows.
The BM3D ORNL code is released under an open-source license, and is freely available for download and use.

1.1 Go to the `Settings` tab of the new repo.

1.2 Click on `Branches` on the left side.

1.3 Click on `Add rule` button.

1.4 Follow the instructions from Github.


2. Change the License if MIT license is not suitable for you project. For more information about licenses, please
refer to [Choose an open source license](https://choosealicense.com/).


3. Update the environment dependency file `environment.yml`, which contain both runtime and development dependencies.
For more information about conda environment file, please refer to [Conda environment file](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually).

3.1 Specify environment 'name' field to match package name

3.2 We strongly recommended using a single `environment.yml` file to manage all the dependencies,
including the runtime and development dependencies.

3.3 Please add comments to the `environment.yml` file to explain the dependencies.

3.4 Please prune the dependencies to the minimum when possible,
we would like the solver to figure out the dependency tree for us.


4. Adjust pre-commit configuration file, `.pre-commit-config.yaml` to enable/disable the hooks you need.
For more information about pre-commit, please refer to [pre-commit](https://pre-commit.com/).


5. Having code coverage, `codecov.yaml` is **strongly recommended**,
please refer to [Code coverage](https://coverage.readthedocs.io/en/coverage-5.5/) for more information.


6. Adjust the demo Github action yaml files for CI/CD. For more information about Github action,
please refer to [Github action](https://docs.github.com/en/actions).

6.1 Specify package name at: .github/workflows/package.yml#L34

6.2 Specify package name at: .github/workflows/package.yml#L46


7. Adjust the conda recipe, `conda-recipe/meta.yaml` to provide the meta information for the conda package.
For more information about conda recipe, please refer to [Conda build](https://docs.conda.io/projects/conda-build/en/latest/).

7.1 Specify package name at: conda.recipe/meta.yaml#L15

7.2 Update license family, if necessary: conda.recipe/meta.yaml#L42


8. Adjust `pyproject.toml` to match your project. For more information about `pyproject.toml`,
please refer to [pyproject.toml](https://www.python.org/dev/peps/pep-0518/).

8.1 Specify package name at: pyproject.toml#L2

8.2 Specify package description at: pyproject.toml#L3

8.3 Specify package name at: pyproject.toml#L39

8.4 Specify any terminal entry points (terminal commands) at : pyproject.toml#48.
In the example, invoking `packagename-cli` in a terminal is equivalent to running the python script
`from packagenamepy.packagename.import main; main()"

8.5 Projects will use a single `pyproject.toml` file to manage all the project metadata,
including the project name, version, author, license, etc.

8.6 Python has moved away from `setup.cfg`/`setup.py`, and we would like to follow the trend for our new projects.


10. Specify package name at src/packagenamepy


11. Specify package name at: src/packagenamepy/packagename.py

12. If a GUI isn't used, delete the MVP structure at src/packagenamepy:
11.1: mainwindow.py
11.2: home/
11.3: help/


11. Clear the content of this file and add your own README.md as the project README file.
We recommend putting badges of the project status at the top of the README file.
For more information about badges, please refer to [shields.io](https://shields.io/).

## Repository Adjustments

### Add an access token to anaconda

Here we assume your intent is to upload the conda package to the [anaconda.org/neutrons](https://anaconda.org/neutrons) organization.
An administrator of _anaconda.org/neutrons_ must create an access token for your repository in the [access settings](https://anaconda.org/neutrons/settings/access).

After created, the token must be stored in a _repository secret_:
1. Navigate to the main page of the repository on GitHub.com.
2. Click on the "Settings" tab.
3. In the left sidebar, navigate to the "Security" section and select "Secrets and variables" followed by "Actions".
4. Click on the "New repository secret" button.
5. Enter `ANACONDA_TOKEN` for the secret name
6. Paste the Anaconda access token
7. Click on the "Add secret" button
8. Test the setup by creating a release candidate tag,
which will result in a package built and uploaded to https://anaconda.org/neutrons/mypackagename

### Add an access token to codecov
Follow the instructions in the [Confluence page](https://ornl-neutrons.atlassian.net/wiki/spaces/NDPD/pages/103546883/Coverage+reports)
to create the access token.
For more information, check out our [FAQ](docs/FAQ.md).
2 changes: 1 addition & 1 deletion conda.recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
{% set build_number = 0 %}

package:
name: mypackagename
name: bm3dornl
version: {{ version_number }}

source:
Expand Down
Loading

0 comments on commit 384a7e6

Please sign in to comment.