feat(Integration): Truera Integration

install/mlops/truera/install.yml

@@ -0,0 +1,16 @@
+id: truera-mlops
+name: Truera Inc
+
+title: Truera diagnositcs and monitoring solution for Trustworthy AI.
+
+description: |
+  TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle.
+
+target:
+  type: integration
+  destination: cloud
+
+install:
+  mode: link
+  destination:
+    url: https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/

quickstarts/mlops/truera/config.yml

@@ -0,0 +1,50 @@
+# Name of the quickstart (required)
+name: truera
+
+# Displayed in the UI
+title: Truera
+
+# Long-form description of the quickstart (required)
+description: |
+  TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle.
+
+# Displayed in search results and recommendations. Summarizes a quickstarts functionality.
+summary: |
+  TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle.
+
+# Support level: New Relic | Verified | Community (required)
+level: New Relic
+
+# Authors of the quickstart (required)
+authors:
+  - Truera
+
+# Keywords for filtering / searching criteria in the UI
+keywords:
+  - mlops
+  - ml
+  - ai
+  - machine-learning
+  - machine learning
+  - ml monitoring
+  - machine learning monitoring
+  - xai
+  - explainable ai
+  - explanability
+  - fairness
+  - truera
+
+# Reference to install plans located under /install directory
+# Allows us to construct reusable "install plans" and just use their ID in the quickstart config
+installPlans:
+  - truera-mlops
+
+documentation:
+  - name: Truera Docs
+    url: https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/
+    description: Documentation on the workings of the Truera integration
+
+# Content / Design
+icon: icon.svg
+logo: logo.svg
+website: https://truera.com/

quickstarts/mlops/truera/dashboards/truera.json

@@ -0,0 +1,102 @@
+{
+  "name": "Truera: Initial Dashboard",
+  "pages": [
+    {
+      "name": "Initial Dashboard",
+      "description": null,
+      "widgets": [
+        {
+          "visualization": {
+            "id": "viz.markdown"
+          },
+          "layout": {
+            "column": 1,
+            "row": 1,
+            "height": 3,
+            "width": 5
+          },
+          "title": "",
+          "rawConfiguration": {
+            "text": "![Add Images](https://truera.com/wp-content/uploads/2021/04/Group-70.svg)\n\nTruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle.\n\nIf you are new to Truera and want to learn more, [contact us](https://truera.com/#elementor-action%3Aaction%3Dpopup%3Aopen%26settings%3DeyJpZCI6IjI2MDAiLCJ0b2dnbGUiOmZhbHNlfQ%3D%3D)"
+          },
+          "linkedEntityGuids": null
+        },
+        {
+          "visualization": {
+            "id": "viz.markdown"
+          },
+          "layout": {
+            "column": 6,
+            "row": 1,
+            "height": 3,
+            "width": 6
+          },
+          "title": "",
+          "rawConfiguration": {
+            "text": "\n# Startup Guide\n\n## Initial bootstrap\n\n* Create and store Truera API key and New Relic API keys.\n* Install truera2newrelic tool.\n\n## Once per model:\n\n* For models to be monitored using new relic, stream metrics from truera to new relic using \"truera2newrelic metrics\" command. The data will show up in model score instability graph below.\n* Setup one dashboard in New Relic per ML model using \"truera2newrelic dashboard\" command. This dashboard will allow you to monitor data and analyze/diagnose issues.\n\nStep by step guide is part of [Trurea - New Relic integration doc](https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/)\n"
+          },
+          "linkedEntityGuids": null
+        },
+        {
+          "visualization": {
+            "id": "viz.markdown"
+          },
+          "layout": {
+            "column": 1,
+            "row": 4,
+            "height": 1,
+            "width": 3
+          },
+          "title": "",
+          "rawConfiguration": {
+            "text": "# [Monitor models in New Relic Dashboard](launcher/dashboards.launcher?platform[filters]=IihuYW1lIExJS0UgJ1RydWVyYSBVbmlmaWVkJyBPUiBpZCA9ICdUcnVlcmEgVW5pZmllZCcgT1IgZG9tYWluSWQgPSAnVHJ1ZXJhIFVuaWZpZWQnKSI=&platform[accountId]=3206889&pane=eyJuZXJkbGV0SWQiOiJkYXNoYm9hcmRzLmhvbWUifQ==)\n"
+          },
+          "linkedEntityGuids": null
+        },
+        {
+          "visualization": {
+            "id": "viz.markdown"
+          },
+          "layout": {
+            "column": 4,
+            "row": 4,
+            "height": 1,
+            "width": 3
+          },
+          "title": "",
+          "rawConfiguration": {
+            "text": "## [Contact Truera team](https://truera.com/#elementor-action%3Aaction%3Dpopup%3Aopen%26settings%3DeyJpZCI6IjI2MDAiLCJ0b2dnbGUiOmZhbHNlfQ%3D%3D)"
+          },
+          "linkedEntityGuids": null
+        },
+        {
+          "visualization": {
+            "id": "viz.line"
+          },
+          "layout": {
+            "column": 1,
+            "row": 5,
+            "height": 3,
+            "width": 10
+          },
+          "title": "Model Instability(NUMERICAL_WASSERSTEIN)",
+          "rawConfiguration": {
+            "legend": {
+              "enabled": true
+            },
+            "nrqlQueries": [
+              {
+                "accountId": 3206889,
+                "query": "SELECT latest(model_msi_v1) FROM Metric where segment='AllData' and DistributionComparisonType='NUMERICAL_WASSERSTEIN' FACET dimensions() TIMESERIES "
+              }
+            ],
+            "yAxisLeft": {
+              "zero": true
+            }
+          },
+          "linkedEntityGuids": null
+        }
+      ]
+    }
+  ]
+}