Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit which lfs assets are pulled by default #139

Merged
44 commits merged into from
Jun 8, 2022

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Jun 2, 2022

  • Adds helper script scripts/fetch_data.py to pull additional assets.
  • Update README.md & CONTRIBUTING.md docs to reflect changes
  • Updated the hammah unittest to no longer depend on training data.
  • Pinned Neo dep to 22.04.00a to work-around incompatible neo version
$ scripts/fetch_data.py -h
usage: Fetches data not included in the repository by default [-h] {fetch,check} ...

optional arguments:
  -h, --help     show this help message and exit

Subcommands:
  valid subcommands

  {fetch,check}
    fetch        Fetch datasets
    check        Check download status of large files. Displays a True/False whether all files are downloaded.

Closes #88

@dagardner-nv dagardner-nv requested review from a team as code owners June 2, 2022 16:33
@dagardner-nv dagardner-nv added non-breaking Non-breaking change feature request New feature or request 3 - Ready for Review labels Jun 2, 2022
@dagardner-nv dagardner-nv added DO NOT MERGE PR should not be merged; see PR for details 4 - Waiting on Author and removed 3 - Ready for Review labels Jun 3, 2022
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of comments. I think the script should be moved to ./scripts filter instead of CI. Also, printing to the console didnt work for me out of the box.

docker/conda/environments/cuda11.4_dev.yml Outdated Show resolved Hide resolved
ci/scripts/fetch_data.py Outdated Show resolved Hide resolved
ci/scripts/fetch_data.py Outdated Show resolved Hide resolved
ci/scripts/fetch_data.py Outdated Show resolved Hide resolved
ci/scripts/fetch_data.py Outdated Show resolved Hide resolved
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check command is good but needs better output.

docker/conda/environments/cuda11.4_dev.yml Outdated Show resolved Hide resolved
scripts/fetch_data.py Outdated Show resolved Hide resolved
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Great addition

@mdemoret-nv
Copy link
Contributor

@gpucibot merge

@ghost ghost merged commit 1a1be1a into nv-morpheus:branch-22.06 Jun 8, 2022
@dagardner-nv dagardner-nv deleted the david-lfs-config branch February 12, 2024 23:20
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Improve LFS with Scripts and Optional Download
3 participants