Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create tests for examples with custom stages #885

Merged
merged 90 commits into from
Apr 28, 2023
Merged

Create tests for examples with custom stages #885

merged 90 commits into from
Apr 28, 2023

Commits on Mar 29, 2023

  1. First 20 lines of examples/abp_pcap_detection/pcap_out.jsonlines pull… 

    …ed out here since the original is large and not part of the dataset pulled by the test stage in CI
    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    9e9222d View commit details
    Browse the repository at this point in the history

  2. Remove flow_id and rollup_time cols

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    16de6f7 View commit details
    Browse the repository at this point in the history

  3. Test data

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    a239f41 View commit details
    Browse the repository at this point in the history

  4. Fix type-o

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    59dc4f9 View commit details
    Browse the repository at this point in the history

  5. Add a get_stage_class method to StageInfo

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    d7e23be View commit details
    Browse the repository at this point in the history

  6. Fixture to reset the PluginManager and GlobalStageRegistry

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    fb7829e View commit details
    Browse the repository at this point in the history

  7. Add helper method to assist pulling a stage from the plugin manager

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    454eeea View commit details
    Browse the repository at this point in the history

  8. End-to-end test for AbpPcapPreprocessingStage

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    e3007b7 View commit details
    Browse the repository at this point in the history

  9. Test asserting that needed columns are defined

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    c354361 View commit details
    Browse the repository at this point in the history

  10. Request pre-allocation

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    3fcb19a View commit details
    Browse the repository at this point in the history

  11. Fix setting get_stage_class in LazyStageInfo

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    dd41ddc View commit details
    Browse the repository at this point in the history

  12. Merge branch 'branch-23.03' into pcap_preprocessing-pre-allocating-797

    @dagardner-nv
    dagardner-nv committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    1781bd6 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2023

  1. Merge branch 'branch-23.07' into pcap_preprocessing-pre-allocating-797

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    5d4eb0a View commit details
    Browse the repository at this point in the history

  2. reload_modules fixture reloads requested modules before and after the… 

    … test
    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    493e7fb View commit details
    Browse the repository at this point in the history

  3. Move test_abp_pcap_preprocessing.py to an examples subdir

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    81002ac View commit details
    Browse the repository at this point in the history

  4. Fixture to restore sys.path

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    ad9e578 View commit details
    Browse the repository at this point in the history

  5. Add type hints

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    664b2d7 View commit details
    Browse the repository at this point in the history

  6. Add test for pass-thru stages

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    c8992cd View commit details
    Browse the repository at this point in the history

  7. Fix invocation of reload_modules fixture

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    f460f66 View commit details
    Browse the repository at this point in the history

  8. Deleting redundant copies of bert hash files

    @dagardner-nv
    dagardner-nv committed Apr 5, 2023
    Configuration menu
    Copy the full SHA
    7c3a2bd View commit details
    Browse the repository at this point in the history

Commits on Apr 6, 2023

  1. Update references to files bert-base-cased-hash.txt and bert-base-unc… 

    …ased-hash.txt to the version in the morpheus/data dir
    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    e0185a2 View commit details
    Browse the repository at this point in the history

  2. Revert "Deleting redundant copies of bert hash files" 

    This reverts commit 7c3a2bd.
    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    219e911 View commit details
    Browse the repository at this point in the history

  3. Remove redundant copies of bert-base-cased-hash.txt and bert-base-unc… 

    …ased-hash.txt
    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    c49c039 View commit details
    Browse the repository at this point in the history

  4. Remove redundant copue of bert-base-cased-vocab.txt

    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    af42993 View commit details
    Browse the repository at this point in the history

  5. Subset of models/log-parsing-models/log-parsing-config-20220418.json … 

    …with just the id2label actually used by LogParsingPostProcessingStage
    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    59b1e88 View commit details
    Browse the repository at this point in the history

  6. Fetch the bert cased vocab file for tests

    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    6b52af3 View commit details
    Browse the repository at this point in the history

  7. Test for log parsing post proc example

    @dagardner-nv
    dagardner-nv committed Apr 6, 2023
    Configuration menu
    Copy the full SHA
    85d2924 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2023

  1. Move to sub-dir

    @dagardner-nv
    dagardner-nv committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    4df19b0 View commit details
    Browse the repository at this point in the history

  2. Test for https://github.com/rapidsai/cudf/issues/130850

    @dagardner-nv
    dagardner-nv committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    7a7f4d5 View commit details
    Browse the repository at this point in the history

  3. Explicitly set the encoding as a work-around for nv-morpheus#859

    @dagardner-nv
    dagardner-nv committed Apr 7, 2023
    Configuration menu
    Copy the full SHA
    5985006 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2023

  1. Add kw_only to C++ impls of tensor memory classes to match Python API

    @dagardner-nv
    dagardner-nv committed Apr 10, 2023
    Configuration menu
    Copy the full SHA
    6956143 View commit details
    Browse the repository at this point in the history

  2. wip

    @dagardner-nv
    dagardner-nv committed Apr 10, 2023
    Configuration menu
    Copy the full SHA
    49b31e4 View commit details
    Browse the repository at this point in the history

  3. wip

    @dagardner-nv
    dagardner-nv committed Apr 10, 2023
    Configuration menu
    Copy the full SHA
    e628b56 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    49b8474 View commit details
    Browse the repository at this point in the history

  2. Revert unneeded changes

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    4f41628 View commit details
    Browse the repository at this point in the history

  3. Unimport modules after each test

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    faf29f4 View commit details
    Browse the repository at this point in the history

  4. Use new import_mod fixture

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    c0ab1d9 View commit details
    Browse the repository at this point in the history

  5. Update to use new import_mod fixture

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    8555f08 View commit details
    Browse the repository at this point in the history

  6. Set the configs for this subdir to NLP

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    a1c96fb View commit details
    Browse the repository at this point in the history

  7. Seed cupy's number generator as well

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    b3aecc9 View commit details
    Browse the repository at this point in the history

  8. wip

    @dagardner-nv
    dagardner-nv committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    241b564 View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2023

  1. Fix type hints

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    c62a014 View commit details
    Browse the repository at this point in the history

  2. Exclude tensors from repr

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    a5e80ba View commit details
    Browse the repository at this point in the history

  3. wip

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    f895ce3 View commit details
    Browse the repository at this point in the history

  4. Remove _infer_callback method as it was identicle to parent's _infer_… 

    …callback method
    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    2cedf1d View commit details
    Browse the repository at this point in the history

  5. wip

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    462b758 View commit details
    Browse the repository at this point in the history

  6. Use from_message

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    f31a4c5 View commit details
    Browse the repository at this point in the history

  7. move test data

    @dagardner-nv
    dagardner-nv committed Apr 12, 2023
    Configuration menu
    Copy the full SHA
    a74d529 View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2023

  1. wip

    @dagardner-nv
    dagardner-nv committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    6440ab8 View commit details
    Browse the repository at this point in the history

  2. Tests for developer guide ex2

    @dagardner-nv
    dagardner-nv committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    18ea2c6 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    3fa6d0f View commit details
    Browse the repository at this point in the history

  4. Fix license headers

    @dagardner-nv
    dagardner-nv committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    829e924 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv authored Apr 13, 2023
    Configuration menu
    Copy the full SHA
    d03bbef View commit details
    Browse the repository at this point in the history

  6. Remove unused imports

    @dagardner-nv
    dagardner-nv committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    923f452 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv authored Apr 13, 2023
    Configuration menu
    Copy the full SHA
    610b288 View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2023

  1. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    517b964 View commit details
    Browse the repository at this point in the history

  2. Optionally bypass the cache and optionally pass additional kwargs

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    a3aef0a View commit details
    Browse the repository at this point in the history

  3. Fix usage of assert_df_equal which is now a static method of DatasetM… 

    …anager
    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    d2065ad View commit details
    Browse the repository at this point in the history

  4. Use datasetmanger

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    b42d7c8 View commit details
    Browse the repository at this point in the history

  5. Use new dataset fixtures [no ci]

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    de73c01 View commit details
    Browse the repository at this point in the history

  6. Revert code updates due to removing redundant bert files, will replac… 

    …e those with symlinks for compat [no ci]
    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    3cb8b98 View commit details
    Browse the repository at this point in the history

  7. Add bert vocab files to morpheus/data to be available for testing, pr… 

    …eviously these were in LFS in stored under models/training-tuning-scripts/sid-models/resources/
    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    1215fbe View commit details
    Browse the repository at this point in the history

  8. Updates due to moving the bert vocabs [no ci]

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    ce57304 View commit details
    Browse the repository at this point in the history

  9. wip [no ci]

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    e990cbf View commit details
    Browse the repository at this point in the history

  10. Leave symlinks from the old locations for bert data to the new locati… 

    …on for compatibility
    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    2e97a48 View commit details
    Browse the repository at this point in the history

  11. Remove unused import

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    049c5d5 View commit details
    Browse the repository at this point in the history

  12. Shorted comment to fix under 100 column limit

    @dagardner-nv
    dagardner-nv committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    fc91053 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2023

  1. Move example tests data to its own subfolder

    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    225bf9f View commit details
    Browse the repository at this point in the history

  2. Update tests to reflect data move [no ci]

    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    7fa07b5 View commit details
    Browse the repository at this point in the history

  3. Copy of the first 10 lines of examples/data/email_with_addresses.json… 

    …lines since examples/data is quite large and not fetched in CI
    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    e4460d2 View commit details
    Browse the repository at this point in the history

  4. Update test to reflect data move [no ci]

    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    7b9c440 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples [no ci]
    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    fc8edeb View commit details
    Browse the repository at this point in the history

  6. Add shared libs from examples to tar

    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    39f4f3b View commit details
    Browse the repository at this point in the history

  7. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    fea441f View commit details
    Browse the repository at this point in the history

  8. Add the shared libs from examples not the build dir

    @dagardner-nv
    dagardner-nv committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    e45508f View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2023

  1. Move tests for examples

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    c53366d View commit details
    Browse the repository at this point in the history

  2. Organize test data for examples

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    5c9e338 View commit details
    Browse the repository at this point in the history

  3. Explain the overload of the config fixture

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    d86fc17 View commit details
    Browse the repository at this point in the history

  4. Fix dirs

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    49b7476 View commit details
    Browse the repository at this point in the history

  5. Update test paths

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    65af7c6 View commit details
    Browse the repository at this point in the history

  6. Replace emptry strings with nulls for the purposes of testing filter_… 

    …nulls
    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    70e0b82 View commit details
    Browse the repository at this point in the history

  7. Warn on implied no_cache

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    4c4eee1 View commit details
    Browse the repository at this point in the history

  8. Test reader_args

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    597cdfd View commit details
    Browse the repository at this point in the history

  9. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    1aa9b2f View commit details
    Browse the repository at this point in the history

  10. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv authored Apr 20, 2023
    Configuration menu
    Copy the full SHA
    232afb8 View commit details
    Browse the repository at this point in the history

  11. Fix type-hints for modules

    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    cb94631 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'david-test-examples' of github.com:dagardner-nv/Morpheu… 

    …s into david-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    367a68c View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2023

  1. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    b50782d View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2023

  1. Merge branch 'branch-23.07' of github.com:nv-morpheus/Morpheus into d… 

    …avid-test-examples
    @dagardner-nv
    dagardner-nv committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    f7d67f1 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Merge branch 'branch-23.07' into david-test-examples

    @dagardner-nv
    dagardner-nv authored Apr 28, 2023
    Configuration menu
    Copy the full SHA
    56b9393 View commit details
    Browse the repository at this point in the history