Reorg docs

.clang-format

@@ -1,6 +1,6 @@
 ---
 # Refer to the following link for the explanation of each params:
-#   http://releases.llvm.org/8.0.0/tools/clang/docs/ClangFormatStyleOptions.html
+#   https://releases.llvm.org/14.0.0/tools/clang/docs/ClangFormatStyleOptions.html
 Language: Cpp
 # BasedOnStyle: Google
 AccessModifierOffset: -2

CONTRIBUTING.md

@@ -234,14 +234,14 @@ Launching a full production Kafka cluster is outside the scope of this project.
    ```
 5. Update the `kafka-docker/docker-compose.yml`, performing two changes:
    1. Update the `ports` entry to:
-      ```yml
+      ```yaml
       ports:
          - "0.0.0.0::9092"
       ```
       This will prevent the containers from attempting to map IPv6 ports.
    1. Change the value of `KAFKA_ADVERTISED_HOST_NAME` to match the value of the `KAFKA_ADVERTISED_HOST_NAME` environment variable from the previous step. For example, the line should look like:
 
-      ```yml
+      ```yaml
       environment:
          KAFKA_ADVERTISED_HOST_NAME: 172.17.0.1
       ```

docs/source/cloud_deployment_guide.md

@@ -77,7 +77,7 @@ This quick start guide provides the necessary instructions to set up the minimum
 
 Morpheus makes it easy to build and scale cybersecurity applications that harness adaptive pipelines supporting a wider range of model complexity than previously feasible. Morpheus makes it possible to analyze up to 100% of your data in real-time, for more accurate detection and faster remediation of threats as they occur. Morpheus also provides the ability to leverage AI to adjust to threats and compensate on the fly, at line rate.
 
-NVIDIA Morpheus enables organizations to attack the issue of cybersecurity head on. Rather than continuously chasing the cybersecurity problem, Morpheus provides the ability to propel you ahead of the breach and address the cybersecurity issue. With the world in a "discover and respond" state, where companies are finding breaches much too late, in a way that is way behind the curve, NVIDIA’s Morpheus cybersecurity AI framework enables any organization to warp to the present and begin to defend itself in real time.
+NVIDIA Morpheus enables organizations to attack the issue of cybersecurity head on. Rather than continuously chasing the cybersecurity problem, Morpheus provides the ability to propel you ahead of the breach and address the cybersecurity issue. With the world in a "discover and respond" state, where companies are finding breaches much too late, in a way that is way behind the curve, NVIDIA's Morpheus cybersecurity AI framework enables any organization to warp to the present and begin to defend itself in real time.
 
 The Morpheus Developer Kit allows developers to quickly and easily set up example pipelines to run inference on different sample models provided by NVIDIA and experiment with the features and capabilities available within the Morpheus framework to address their cybersecurity and information security use cases.
 
@@ -97,11 +97,11 @@ The Morpheus Developer Kit allows developers to quickly and easily set up exampl
 
 - **Real-time Telemetry**
 
-    The Morpheus native graph streaming engine can receive rich, real-time network telemetry from every NVIDIA BlueField DPU-accelerated server or NVIDIA AppShield in the data center without impacting performance. Integrating the framework into a third-party cybersecurity offering brings the world’s best AI computing to communication networks. 
+    The Morpheus native graph streaming engine can receive rich, real-time network telemetry from every NVIDIA BlueField DPU-accelerated server or NVIDIA AppShield in the data center without impacting performance. Integrating the framework into a third-party cybersecurity offering brings the world's best AI computing to communication networks. 
 
 - **AI Cybersecurity Capabilities**
 
-    Deploy your own models using common deep learning frameworks. Or get a jump-start in building applications to identify leaked sensitive information, detect malware or fraud, do network mapping, flag user behavior changes, and identify errors via logs by using one of NVIDIA’s pre-trained and tested models.
+    Deploy your own models using common deep learning frameworks. Or get a jump-start in building applications to identify leaked sensitive information, detect malware or fraud, do network mapping, flag user behavior changes, and identify errors via logs by using one of NVIDIA's pre-trained and tested models.
 
 ## Setup
 
@@ -115,7 +115,7 @@ Continue with the setup steps below once the host system is installed, configure
 
 First, you will need to set up your NGC API Key to access all the Morpheus components, using the linked instructions from the [NGC Registry CLI User Guide].
 
-Once you’ve created your API key, create an environment variable containing your API key for use by the commands used further in this document:
+Once you've created your API key, create an environment variable containing your API key for use by the commands used further in this document:
 
 ```bash
 export API_KEY="<NGC_API_KEY>"
@@ -726,7 +726,7 @@ kubectl -n $NAMESPACE exec -it deploy/broker -c broker -- kafka-console-producer
 **Note**: This should be used for development purposes only via this developer kit. Loading from the file into Kafka should not be used in production deployments of Morpheus.
 
 ### Verify Running Pipeline
-Once you’ve deployed the SDK client to run a pipeline, you can check the status of the pod using the following command:
+Once you've deployed the SDK client to run a pipeline, you can check the status of the pod using the following command:
 
 ```bash
 kubectl -n $NAMESPACE get pods sdk-cli-<RELEASE_NAME>
@@ -758,7 +758,7 @@ Inference rate: 7051messages [00:04, 4639.40messages/s]
 2.  AWS EC2 G4 instance with T4 or V100 GPU, at least 64GB RAM, 8 cores CPU, and 100 GB storage.
 
 #### Install Cloud Native Core Stack for AWS
-On your AWS EC2 G4 instance, follow the instructions in the linked document to install [NVIDIA’s Cloud Native Core Stack for AWS][NVIDIA’s Cloud Native Core Stack].
+On your AWS EC2 G4 instance, follow the instructions in the linked document to install [NVIDIA's Cloud Native Core Stack for AWS][NVIDIA's Cloud Native Core Stack].
 
 ### Prerequisites and Installation for Ubuntu
 
@@ -768,7 +768,7 @@ On your AWS EC2 G4 instance, follow the instructions in the linked document to i
 3.  Ubuntu 20.04 LTS or newer
 
 ### Installing Cloud Native Core Stack on NVIDIA Certified Systems
-On your NVIDIA-Certified System, follow the instructions in the linked document to install [NVIDIA’s Cloud Native Core Stack].
+On your NVIDIA-Certified System, follow the instructions in the linked document to install [NVIDIA's Cloud Native Core Stack].
 
 ## Appendix B
 
@@ -1089,5 +1089,5 @@ The Drop Null Attributes stage (dropna) requires the specification of a column n
 [Morpheus Contribution]: https://github.com/nv-morpheus/Morpheus/blob/main/CONTRIBUTING.md
 [Morpheus Developer Guide]: https://github.com/nv-morpheus/Morpheus/tree/main/docs/source/developer_guide/guides
 [Triton Inference Server Model Configuration]: https://github.com/triton-inference-server/server/blob/main/docs/model_configuration.md
-[NVIDIA’s Cloud Native Core Stack]: https://github.com/NVIDIA/cloud-native-core
+[NVIDIA's Cloud Native Core Stack]: https://github.com/NVIDIA/cloud-native-core
 [NGC Registry CLI User Guide]: https://docs.nvidia.com/dgx/ngc-registry-cli-user-guide/index.html#topic_4_1

docs/source/developer_guide/contributing.md

@@ -0,0 +1 @@
+../../../CONTRIBUTING.md

docs/source/developer_guide/examples/abp_nvsmi_detection/README.md

@@ -0,0 +1 @@
+../../../../../examples/abp_nvsmi_detection/README.md

docs/source/developer_guide/examples/abp_nvsmi_detection/pipeline.png

@@ -0,0 +1 @@
+../../../../../examples/abp_nvsmi_detection/pipeline.png

docs/source/developer_guide/examples/abp_pcap_detection/README.md

@@ -0,0 +1 @@
+../../../../../examples/abp_pcap_detection/README.md

docs/source/developer_guide/examples/gnn_fraud_detection_pipeline/README.md

@@ -0,0 +1 @@
+../../../../../examples/gnn_fraud_detection_pipeline/README.md

docs/source/developer_guide/examples/index.rst

@@ -5,11 +5,11 @@ Examples
 .. toctree::
    :maxdepth: 20
 
-   abp_nvsmi_detection.md
-   abp_pcap_detection.md
-   gnn_fraud_detection_pipeline.md
-   log_parsing.md
-   nlp_si_detection.md
-   ransomware_detection.md
-   root_cause_analysis.md
-   sid_visualization.md
+   abp_nvsmi_detection/README.md
+   abp_pcap_detection/README.md
+   gnn_fraud_detection_pipeline/README.md
+   log_parsing/README.md
+   nlp_si_detection/README.md
+   ransomware_detection/README.md
+   root_cause_analysis/README.md
+   sid_visualization/README.md

docs/source/developer_guide/examples/log_parsing/README.md

@@ -0,0 +1 @@
+../../../../../examples/log_parsing/README.md

docs/source/developer_guide/examples/nlp_si_detection/README.md

@@ -0,0 +1 @@
+../../../../../examples/nlp_si_detection/README.md

docs/source/developer_guide/examples/nlp_si_detection/pipeline.png

@@ -0,0 +1 @@
+../../../../../examples/nlp_si_detection/pipeline.png

docs/source/developer_guide/examples/ransomware_detection/README.md

@@ -0,0 +1 @@
+../../../../../examples/ransomware_detection/README.md

docs/source/developer_guide/examples/root_cause_analysis/README.md

@@ -0,0 +1 @@
+../../../../../examples/root_cause_analysis/README.md

docs/source/developer_guide/examples/sid_visualization/README.md

@@ -0,0 +1 @@
+../../../../../examples/sid_visualization/README.md

docs/source/developer_guide/guides.md

@@ -0,0 +1,26 @@
+<!--
+SPDX-FileCopyrightText: Copyright (c) 2022, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: Apache-2.0
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Guides
+
+Morpheus includes several stages to choose from when building a custom pipeline, which can be included and configured to suit your needs. However, there are likely going to be situations that require writing a custom stage. Morpheus stages are written in Python and optionally may include a C++ implementation. The following guides outline how to create your own stages in both Python and C++.
+
+## [Simple Python Stage](./guides/1_simple_python_stage.md)
+## [Real-World Application: Phishing Detection](./guides/2_real_world_phishing.md)
+## [Simple C++ Stage](./guides/3_simple_cpp_stage.md).
+## [Creating a C++ Source Stage](./guides/4_source_cpp_stage.md).
+## [Digital Fingerprinting (DFP)](./guides/5_digital_fingerprinting.md).

docs/source/developer_guide/guides/1_simple_python_stage.md

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-# 1. A Simple Python Stage
+# Simple Python Stage
 
 ## Background
 

docs/source/developer_guide/guides/2_real_world_phishing.md

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-# 2. A Real-World Application: Phishing Detection
+# Real-World Application: Phishing Detection
 
 ## Data Preprocessing
 

docs/source/developer_guide/guides/3_simple_cpp_stage.md

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-# 3. A Simple C++ Stage
+# Simple C++ Stage
 
 Morpheus offers the choice of writing pipeline stages in either Python or C++. For many use cases, a Python stage is perfectly fine. However, in the event that a Python stage becomes a bottleneck for the pipeline, then writing a C++ implementation for the stage becomes advantageous. The C++ implementations of Morpheus stages and messages utilize the [pybind11](https://pybind11.readthedocs.io/en/stable/index.html) library to provide Python bindings.
 

docs/source/developer_guide/guides/4_source_cpp_stage.md

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-# 4. Creating a C++ Source Stage
+# Creating a C++ Source Stage
 
 For this example, we are going to add a C++ implementation for the `RabbitMQSourceStage` we designed in the Python examples. The Python implementation of this stage emits messages of the type `MessageMeta`; as such, our C++ implementation must do the same.
 

docs/source/developer_guide/guides/5_digital_fingerprinting.md

@@ -15,7 +15,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-# 5. Digital Fingerprinting (DFP)
+# Digital Fingerprinting (DFP)
 
 ## Overview
 Every account, user, service and machine has a digital fingerprint​, which represents the typical actions performed and not performed over a given period of time.  Understanding every entity's day-to-day, moment-by-moment work helps us identify anomalous behavior and uncover potential threats in the environment​.
@@ -28,7 +28,7 @@ The data we will want to use for the training and inference will be any sensitiv
 The location of these logs could be either local to the machine running Morpheus, a shared file system like NFS or on a remote store such as [Amazon S3](https://aws.amazon.com/s3/).
 
 ### Defining a New Data Source
-Additional data sources and remote stores can easily be added using the Morpheus SDK.  The key to applying DFP to a new data source is through the process of feature selection. Any data source can be fed into DFP after some preprocessing to get a feature vector per log/data point​.  In order to build a targeted model for each entity (user/service/machine... etc.), the chosen data source needs a field that uniquely identifies the entity we’re trying to model.
+Additional data sources and remote stores can easily be added using the Morpheus SDK.  The key to applying DFP to a new data source is through the process of feature selection. Any data source can be fed into DFP after some preprocessing to get a feature vector per log/data point​.  In order to build a targeted model for each entity (user/service/machine... etc.), the chosen data source needs a field that uniquely identifies the entity we're trying to model.
 
 Adding a new source for the DFP pipeline requires defining five critical pieces:
 1. The user_id column in the Morpheus config attribute `ae.userid_column_name`. This can be any column which uniquely identifies the user, account or service being fingerprinted. Examples of possible user_ids could be:
@@ -108,8 +108,8 @@ This example is described in [`examples/digital_fingerprinting/production/README
 | Feature | Description |
 | ------- | ----------- |
 | logcount | tracks the number of logs generated by a user within that day (increments with every log)​ |
-| locincrement | increments every time we observe a new city (location.city) in a user’s logs within that day​ |
-| appincrement | increments every time we observe a new app (appDisplayName) in a user’s logs within that day​ |
+| locincrement | increments every time we observe a new city (location.city) in a user's logs within that day​ |
+| appincrement | increments every time we observe a new app (appDisplayName) in a user's logs within that day​ |
 
 #### Duo Authentication
 | Feature | Description |
@@ -125,7 +125,7 @@ This example is described in [`examples/digital_fingerprinting/production/README
 | Feature | Description |
 | ------- | ----------- |
 | logcount | tracks the number of logs generated by a user within that day (increments with every log)​ |
-| locincrement | increments every time we observe a new city (location.city) in a user’s logs within that day​ |
+| locincrement | increments every time we observe a new city (location.city) in a user's logs within that day​ |
 
 
 ## High Level Architecture

docs/source/extra_info/known_issues.md

@@ -17,4 +17,4 @@ limitations under the License.
 
 # Known Issues
 
-There are no known issues at this time.
+Refer to [open issues in the Morpheus project](https://github.com/nv-morpheus/Morpheus/issues)