Skip to content

Commit

Permalink
Integrate NVTabular into Morpheus Core and replace existing column_in…
Browse files Browse the repository at this point in the history
…fo based workflows. (#938)

Closes #862 
Closes #863 

Integrates NVT into Morpheus core
- Appropriate conda packages added
- Existing process_dataframe calls in column_info.py are converted to nvt.Workflows
- Backwards compatibility is maintained for anyone using DataFrameInputSchema objects

Authors:
  - Devin Robison (https://github.com/drobison00)
  - Michael Demoret (https://github.com/mdemoret-nv)
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: #938
  • Loading branch information
drobison00 authored Jul 11, 2023
1 parent 0899b9e commit c3d2fc7
Show file tree
Hide file tree
Showing 29 changed files with 2,457 additions and 207 deletions.
2 changes: 1 addition & 1 deletion ci/scripts/bootstrap_local_ci.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-FileCopyrightText: Copyright (c) 2022-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand Down
2 changes: 2 additions & 0 deletions docker/conda/environments/cuda11.8_dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ dependencies:
- nodejs=18.15.0
- numba>=0.56.2
- numpydoc=1.4
- nvtabular=23.06
- pandas=1.3
- pip
- pkg-config # for mrc cmake
Expand All @@ -91,6 +92,7 @@ dependencies:
- sphinx
- sphinx_rtd_theme
- sysroot_linux-64=2.17
- tritonclient=2.26 # Required by NvTabular, force the version, so we get protobufs compatible with 4.21
- tqdm=4
- typing_utils=0.1
- watchdog=2.1
Expand Down
1 change: 0 additions & 1 deletion docker/conda/environments/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,4 @@ jupyterlab
nvidia-pyindex
# Duplicated in conda dev to ensure parity with libprotobuf
protobuf==4.21.*
tritonclient[all]==2.17.*
websockets
Loading

0 comments on commit c3d2fc7

Please sign in to comment.