Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implemented Hydra Usage #477

Merged
merged 139 commits into from
May 2, 2024
Merged

Implemented Hydra Usage #477

merged 139 commits into from
May 2, 2024

Conversation

Pale-Blue-Dot-97
Copy link
Owner

@Pale-Blue-Dot-97 Pale-Blue-Dot-97 commented May 2, 2024 •
edited
Loading

This PR introduces hydra to minerva, allowing it to supplant inbuilt config handling and arg-parsing

Major Updates

  • Closes Use hydra to handle configs #459
  • Most of config_load.py is removed as redundant.
  • runner.distributed_runis now a decorator to be used in conjunction withhydra.main`.
  • GENERIC_PARSER removed as CLI is now handled by hydra.
  • Config is now longer stored as a global variable in utils. This improves reliability and transparency. References to CONFIG, DATA_CONFIG and IMAGERY_CONFIG have been removed throughout minerva.

Important

Many functions will now require additional arguments that were provided by the CONFIG before,

Note

Not all documentation has been updated yet to reflect many of these changes.

pre-commit-ci bot and others added 30 commits January 29, 2024 20:49
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate
6ad1c2d 
updates:
- [github.com/psf/black: 23.12.1 → 24.1.1](psf/black@23.12.1...24.1.1)
- [github.com/PyCQA/bandit: 1.7.6 → 1.7.7](PyCQA/bandit@1.7.6...1.7.7)
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
f0490ec 
for more information, see https://pre-commit.ci
@Pale-Blue-Dot-97
Merge pull request #427 from Pale-Blue-Dot-97/pre-commit-ci-update-co…
0a83023 
…nfig

[pre-commit.ci] pre-commit autoupdate
@dependabot
Bump starlette from 0.36.0 to 0.36.1 in /requirements
f217086 
Bumps [starlette](https://github.com/encode/starlette) from 0.36.0 to 0.36.1.
- [Release notes](https://github.com/encode/starlette/releases)
- [Changelog](https://github.com/encode/starlette/blob/master/docs/release-notes.md)
- [Commits](encode/starlette@0.36.0...0.36.1)

---
updated-dependencies:
- dependency-name: starlette
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump argcomplete from 3.2.1 to 3.2.2 in /requirements
bbeb77f 
Bumps [argcomplete](https://github.com/kislyuk/argcomplete) from 3.2.1 to 3.2.2.
- [Release notes](https://github.com/kislyuk/argcomplete/releases)
- [Changelog](https://github.com/kislyuk/argcomplete/blob/develop/Changes.rst)
- [Commits](kislyuk/argcomplete@v3.2.1...v3.2.2)

---
updated-dependencies:
- dependency-name: argcomplete
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump mlflow from 2.9.2 to 2.10.0 in /requirements
4622ef7 
Bumps [mlflow](https://github.com/mlflow/mlflow) from 2.9.2 to 2.10.0.
- [Release notes](https://github.com/mlflow/mlflow/releases)
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md)
- [Commits](mlflow/mlflow@v2.9.2...v2.10.0)

---
updated-dependencies:
- dependency-name: mlflow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump types-requests in /requirements
ac33333 
Bumps [types-requests](https://github.com/python/typeshed) from 2.31.0.20240106 to 2.31.0.20240125.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@Pale-Blue-Dot-97
Merge pull request #432 from Pale-Blue-Dot-97/dependabot/pip/requirem…
da6d7d8 
…ents/types-requests-2.31.0.20240125

Bump types-requests from 2.31.0.20240106 to 2.31.0.20240125 in /requirements
@Pale-Blue-Dot-97
Merge pull request #431 from Pale-Blue-Dot-97/dependabot/pip/requirem…
0c35d2e 
…ents/mlflow-2.10.0

Bump mlflow from 2.9.2 to 2.10.0 in /requirements
@Pale-Blue-Dot-97
Merge pull request #430 from Pale-Blue-Dot-97/dependabot/pip/requirem…
21b2115 
…ents/argcomplete-3.2.2

Bump argcomplete from 3.2.1 to 3.2.2 in /requirements
@Pale-Blue-Dot-97
Merge pull request #428 from Pale-Blue-Dot-97/dependabot/pip/requirem…
9671d3a 
…ents/starlette-0.36.1

Bump starlette from 0.36.0 to 0.36.1 in /requirements
@dependabot
Bump starlette from 0.36.1 to 0.37.0 in /requirements
98b7399 
Bumps [starlette](https://github.com/encode/starlette) from 0.36.1 to 0.37.0.
- [Release notes](https://github.com/encode/starlette/releases)
- [Changelog](https://github.com/encode/starlette/blob/master/docs/release-notes.md)
- [Commits](encode/starlette@0.36.1...0.37.0)

---
updated-dependencies:
- dependency-name: starlette
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump tox from 4.12.0 to 4.12.1 in /requirements
7547c09 
Bumps [tox](https://github.com/tox-dev/tox) from 4.12.0 to 4.12.1.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.12.0...4.12.1)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@Pale-Blue-Dot-97
Merge pull request #436 from Pale-Blue-Dot-97/dependabot/pip/requirem…
843fd6f 
…ents/starlette-0.37.0

Bump starlette from 0.36.1 to 0.37.0 in /requirements
@dependabot
Bump codecov/codecov-action from 3 to 4
576d34b 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 3 to 4.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](codecov/codecov-action@v3...v4)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
@Pale-Blue-Dot-97
Merge pull request #437 from Pale-Blue-Dot-97/dependabot/pip/requirem…
2264c41 
…ents/tox-4.12.1

Bump tox from 4.12.0 to 4.12.1 in /requirements
@Pale-Blue-Dot-97
Merge pull request #438 from Pale-Blue-Dot-97/dependabot/github_actio…
c5957eb 
…ns/codecov/codecov-action-4

Bump codecov/codecov-action from 3 to 4
@dependabot
Bump torch from 2.1.2 to 2.2.0 in /requirements
00503ba 
Bumps [torch](https://github.com/pytorch/pytorch) from 2.1.2 to 2.2.0.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md)
- [Commits](pytorch/pytorch@v2.1.2...v2.2.0)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@Pale-Blue-Dot-97
Bump torchvision from 0.16.2-->0.17.0
abed3a9
@Pale-Blue-Dot-97
Merge pull request #434 from Pale-Blue-Dot-97/dependabot/pip/requirem…
4eb1bdb 
…ents/torch-2.2.0

Bump torch from 2.1.2 to 2.2.0 in /requirements
@dependabot
Bump ipykernel from 6.29.0 to 6.29.2 in /requirements
75eac2f 
Bumps [ipykernel](https://github.com/ipython/ipykernel) from 6.29.0 to 6.29.2.
- [Release notes](https://github.com/ipython/ipykernel/releases)
- [Changelog](https://github.com/ipython/ipykernel/blob/main/CHANGELOG.md)
- [Commits](ipython/ipykernel@v6.29.0...v6.29.2)

---
updated-dependencies:
- dependency-name: ipykernel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump starlette from 0.37.0 to 0.37.1 in /requirements
d52879e 
Bumps [starlette](https://github.com/encode/starlette) from 0.37.0 to 0.37.1.
- [Release notes](https://github.com/encode/starlette/releases)
- [Changelog](https://github.com/encode/starlette/blob/master/docs/release-notes.md)
- [Commits](encode/starlette@0.37.0...0.37.1)

---
updated-dependencies:
- dependency-name: starlette
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump tqdm from 4.66.1 to 4.66.2 in /requirements
49a91c4 
Bumps [tqdm](https://github.com/tqdm/tqdm) from 4.66.1 to 4.66.2.
- [Release notes](https://github.com/tqdm/tqdm/releases)
- [Commits](tqdm/tqdm@v4.66.1...v4.66.2)

---
updated-dependencies:
- dependency-name: tqdm
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@pre-commit-ci
[pre-commit.ci] pre-commit autoupdate
8ebff3b 
updates:
- [github.com/psf/black: 24.1.1 → 24.2.0](psf/black@24.1.1...24.2.0)
- [github.com/adrienverge/yamllint.git: v1.33.0 → v1.35.1](https://github.com/adrienverge/yamllint.git/compare/v1.33.0...v1.35.1)
@Pale-Blue-Dot-97
Merge pull request #445 from Pale-Blue-Dot-97/pre-commit-ci-update-co…
b3ea272 
…nfig

[pre-commit.ci] pre-commit autoupdate
@Pale-Blue-Dot-97
Merge pull request #444 from Pale-Blue-Dot-97/dependabot/pip/requirem…
c12a870 
…ents/tqdm-4.66.2

Bump tqdm from 4.66.1 to 4.66.2 in /requirements
@Pale-Blue-Dot-97
Merge pull request #443 from Pale-Blue-Dot-97/dependabot/pip/requirem…
80eec98 
…ents/starlette-0.37.1

Bump starlette from 0.37.0 to 0.37.1 in /requirements
@Pale-Blue-Dot-97
Merge pull request #442 from Pale-Blue-Dot-97/dependabot/pip/requirem…
af4709c 
…ents/ipykernel-6.29.2

Bump ipykernel from 6.29.0 to 6.29.2 in /requirements
@dependabot
Bump setuptools from 69.0.3 to 69.1.1 in /requirements
25e27a0 
Bumps [setuptools](https://github.com/pypa/setuptools) from 69.0.3 to 69.1.1.
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst)
- [Commits](pypa/setuptools@v69.0.3...v69.1.1)

---
updated-dependencies:
- dependency-name: setuptools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot
Bump ipykernel from 6.29.2 to 6.29.3 in /requirements
aedab75 
Bumps [ipykernel](https://github.com/ipython/ipykernel) from 6.29.2 to 6.29.3.
- [Release notes](https://github.com/ipython/ipykernel/releases)
- [Changelog](https://github.com/ipython/ipykernel/blob/main/CHANGELOG.md)
- [Commits](ipython/ipykernel@v6.29.2...v6.29.3)

---
updated-dependencies:
- dependency-name: ipykernel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@Pale-Blue-Dot-97 Pale-Blue-Dot-97 added documentation Improvements or additions to documentation enhancement New feature or request generalise Code needs generalising testing New tests needed restructure Restructure package python Pull requests that update Python code yaml Updates to yaml files labels May 2, 2024
@Pale-Blue-Dot-97 Pale-Blue-Dot-97 self-assigned this May 2, 2024
@Pale-Blue-Dot-97
Copy link
Owner Author

Tests failing due to known old issue with mismatching outputs sizes from SimConv. This is resolved in 458-downstream-tasks so fail will be ignored here.

There is also an issue with MacOS with a segmentation fault in test_trainer_1. This will be resolved in the future.

@Pale-Blue-Dot-97 Pale-Blue-Dot-97 merged commit f3b9f37 into beta May 2, 2024
2 of 12 checks passed
@Pale-Blue-Dot-97 Pale-Blue-Dot-97 deleted the 459-hydra-dev branch May 2, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Pale-Blue-Dot-97 Pale-Blue-Dot-97

Labels
documentation Improvements or additions to documentation enhancement New feature or request generalise Code needs generalising python Pull requests that update Python code restructure Restructure package testing New tests needed yaml Updates to yaml files
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Pale-Blue-Dot-97