[RELEASE] Morpheus v22.06 #237
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fixes typos in README.md removes one remaining gitlab merge-request term in CONTRIBUTING.md
Add trailing underscore to collection name
Updated README.md to fix URL to NGC Morpheus collection
Install new apt key in docker
Fix type-o in log parsing example & small formatting fix
…and CONTRIBUTING.md Fix typos in README.md & Change GitLab-style reference in CONTRIBUTING.md
* Remove ref to git submodule * Fix path to labels file * Remove buffer stages * Add `--truncation=True` to the preprocess stage * Morpheus stops on its own now
This reverts commit f1de88f.
Rename mlflow conda env file
Updated the `README.md` file as below. - Removed reference to getting started on NGC (was causing confusion) - Reworked language on getting started on GitHub - Minor formatting issues in the Kafka quick start section Authors: - Bartley Richardson (https://github.com/BartleyR) Approvers: - Pete MacKinnon (https://github.com/pdmack) URL: #91
Looks like a duplicated line. Authors: - Pete MacKinnon (https://github.com/pdmack) Approvers: - https://github.com/gbatmaz - Michael Demoret (https://github.com/mdemoret-nv) URL: #90
Apply HTML theme from `Enterprise Docs` so we don't have to switch theme for every release. <img width="1504" alt="Screen Shot 2022-05-09 at 5 17 25 PM" src="https://user-images.githubusercontent.com/48110516/167500219-c603542b-503e-48ad-80c5-eef363adec97.png"> Authors: - Eli Fajardo (https://github.com/efajardo-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #97
This MR allows python classes created in C++ to show up in IDEs and work with linters. It works by using pybind11-stubgen to create .pyi files that are then included in the package. The result is this:  Only downside is the pybind11-stubgen isnt too actively maintained (but its also not complex). And the option for generating the output location is limited. So in our output we get: ``` common/ __init__.pyi cudf_helpers/ __init__.pyi messages/ __init__.pyi stages/ __init__.pyi common.so cudf_helpers.so messages.so stages.so ``` It would be better to be organized like the following: ``` common.pyi common.so cudf_helpers.pyi cudf_helpers.so messages.pyi messages.so stages.pyi stages.so ``` The only alternative is to use mypy. However, they dont support docstrings according to this issue: python/mypy#11965 Closes #99 Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) - Devin Robison (https://github.com/drobison00) - Bartley Richardson (https://github.com/BartleyR) URL: #100
This refactors all of the python package files to improve the directory structure, separate classes into their own files, improve readability and reduce merge conflicts. The main morpheus package will now be subdivided into the following submodules:
```
morpheus/
_lib/
io/
messages/
pipelines/
stages/
general/
inference/
input/
output/
postprocess/
preprocess
utils/
```
The largest differences are coming from separating the `morpheus/pipeline.py` into separate files and breaking all stages into their own package (resulting in quite a few new files).
Remaining questions:
- [x] How to handle package reloads for `morpheus.messages`
- [ ] What to call utilities? `utils`? or `utilities`?
Closes #98
Authors:
- Michael Demoret (https://github.com/mdemoret-nv)
Approvers:
- David Gardner (https://github.com/dagardner-nv)
- Devin Robison (https://github.com/drobison00)
- Bartley Richardson (https://github.com/BartleyR)
URL: #98
gbatmaz
approved these changes
Jun 29, 2022
Added missing abstract implementation function to `log_parsing` example preproc and postproc stages. Authors: - https://github.com/bsuryadevara - https://github.com/bsuryadev Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #239
This was referenced Jun 29, 2022
Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: #246
* Testing why LFS isnt working * Removing the morpheus/data files from git LFS * Removing tensorrt * Removing duplicate conda requriements * Adding git=2.35.2 and git-lfs to the ci dependencies to fix an issue with the build * Trying to use fetchinclude still * Trying to pin cuda-python to <=11.7.0 * Go back to exclude * Override exclude Co-authored-by: David Gardner <dagardner@nvidia.com>
Fixes #244 The kafka stage needs to be a pass through node. Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #245
Merged
The release of `cuda-python=11.7.1` has broken both the builds and the CI. See PR #246 which fixed the CI. This PR fixes the conda build. Issue #251 has been created to remove this dependency in the future. Fixes #250 Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #252
Fixes style checks preventing #237 from passing Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - https://github.com/raykallen - David Gardner (https://github.com/dagardner-nv) URL: #249
Fixes #254 This re-enables the check on the number of messages received before trying to process to cudf. Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) - Pete MacKinnon (https://github.com/pdmack) URL: #257
|
rerun tests |
A small regression in PR #257 that would duplicate messages in the `from-kafka` stage when `--use_cpp=False`. If you used 3 threads you would process each message in Kafka 3 times instead of dividing the messages between threads. This also improves the shutdown behavior of the `from-kafka` stage when pressing Ctrl+C to close the pipeline. Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: #262
Some remaining style errors that need to be cleaned up before 22.06 Authors: - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - https://github.com/bsuryadevara URL: #261
mdemoret-nv
removed request for
cwharris,
pdmack,
BartleyR,
bsuryadevara,
drobison00,
raykallen,
efajardo-nv,
tzemicheal and
dagardner-nv
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-22.06andv22.06releaseWhat does this mean?
Only critical/hotfix level issues should be merged into
branch-22.06until release (merging of this PR).What is the purpose of this PR?
branch-22.06into main for the release