-
Notifications
You must be signed in to change notification settings - Fork 301
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(Integration): Truera Integration
- Loading branch information
Vivek Bhalgat
committed
Nov 16, 2021
1 parent
c446685
commit 4454605
Showing
5 changed files
with
188 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
id: truera-mlops | ||
name: Truera Inc | ||
|
||
title: Truera diagnositcs and monitoring solution for Trustworthy AI. | ||
|
||
description: | | ||
TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle. | ||
target: | ||
type: integration | ||
destination: cloud | ||
|
||
install: | ||
mode: link | ||
destination: | ||
url: https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
# Name of the quickstart (required) | ||
name: truera | ||
|
||
# Displayed in the UI | ||
title: Truera | ||
|
||
# Long-form description of the quickstart (required) | ||
description: | | ||
TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle. | ||
# Displayed in search results and recommendations. Summarizes a quickstarts functionality. | ||
summary: | | ||
TruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle. | ||
# Support level: New Relic | Verified | Community (required) | ||
level: New Relic | ||
|
||
# Authors of the quickstart (required) | ||
authors: | ||
- Truera | ||
|
||
# Keywords for filtering / searching criteria in the UI | ||
keywords: | ||
- mlops | ||
- ml | ||
- ai | ||
- machine-learning | ||
- machine learning | ||
- ml monitoring | ||
- machine learning monitoring | ||
- xai | ||
- explainable ai | ||
- explanability | ||
- fairness | ||
- truera | ||
|
||
# Reference to install plans located under /install directory | ||
# Allows us to construct reusable "install plans" and just use their ID in the quickstart config | ||
installPlans: | ||
- truera-mlops | ||
|
||
documentation: | ||
- name: Truera Docs | ||
url: https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/ | ||
description: Documentation on the workings of the Truera integration | ||
|
||
# Content / Design | ||
icon: icon.svg | ||
logo: logo.svg | ||
website: https://truera.com/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
{ | ||
"name": "Truera: Initial Dashboard", | ||
"pages": [ | ||
{ | ||
"name": "Initial Dashboard", | ||
"description": null, | ||
"widgets": [ | ||
{ | ||
"visualization": { | ||
"id": "viz.markdown" | ||
}, | ||
"layout": { | ||
"column": 1, | ||
"row": 1, | ||
"height": 3, | ||
"width": 5 | ||
}, | ||
"title": "", | ||
"rawConfiguration": { | ||
"text": "![Add Images](https://truera.com/wp-content/uploads/2021/04/Group-70.svg)\n\nTruEra provides best in class AI Quality diagnostics and monitoring solution. With this integration, TruEra and NewRelic users can go beyond standard accuracy, input and output drift monitoring by identifying consequential data drift, and other AI Quality diagnostics such as fairness, conceptual soundness and segment behavior. When combined with NewRelic’s comprehensive observability capabilities, MLOps teams can correlate signals throughout the ML lifecycle.\n\nIf you are new to Truera and want to learn more, [contact us](https://truera.com/#elementor-action%3Aaction%3Dpopup%3Aopen%26settings%3DeyJpZCI6IjI2MDAiLCJ0b2dnbGUiOmZhbHNlfQ%3D%3D)" | ||
}, | ||
"linkedEntityGuids": null | ||
}, | ||
{ | ||
"visualization": { | ||
"id": "viz.markdown" | ||
}, | ||
"layout": { | ||
"column": 6, | ||
"row": 1, | ||
"height": 3, | ||
"width": 6 | ||
}, | ||
"title": "", | ||
"rawConfiguration": { | ||
"text": "\n# Startup Guide\n\n## Initial bootstrap\n\n* Create and store Truera API key and New Relic API keys.\n* Install truera2newrelic tool.\n\n## Once per model:\n\n* For models to be monitored using new relic, stream metrics from truera to new relic using \"truera2newrelic metrics\" command. The data will show up in model score instability graph below.\n* Setup one dashboard in New Relic per ML model using \"truera2newrelic dashboard\" command. This dashboard will allow you to monitor data and analyze/diagnose issues.\n\nStep by step guide is part of [Trurea - New Relic integration doc](https://docs.newrelic.com/docs/alerts-applied-intelligence/applied-intelligence/mlops-integrations/truera-integration/)\n" | ||
}, | ||
"linkedEntityGuids": null | ||
}, | ||
{ | ||
"visualization": { | ||
"id": "viz.markdown" | ||
}, | ||
"layout": { | ||
"column": 1, | ||
"row": 4, | ||
"height": 1, | ||
"width": 3 | ||
}, | ||
"title": "", | ||
"rawConfiguration": { | ||
"text": "# [Monitor models in New Relic Dashboard](launcher/dashboards.launcher?platform[filters]=IihuYW1lIExJS0UgJ1RydWVyYSBVbmlmaWVkJyBPUiBpZCA9ICdUcnVlcmEgVW5pZmllZCcgT1IgZG9tYWluSWQgPSAnVHJ1ZXJhIFVuaWZpZWQnKSI=&platform[accountId]=3206889&pane=eyJuZXJkbGV0SWQiOiJkYXNoYm9hcmRzLmhvbWUifQ==)\n" | ||
}, | ||
"linkedEntityGuids": null | ||
}, | ||
{ | ||
"visualization": { | ||
"id": "viz.markdown" | ||
}, | ||
"layout": { | ||
"column": 4, | ||
"row": 4, | ||
"height": 1, | ||
"width": 3 | ||
}, | ||
"title": "", | ||
"rawConfiguration": { | ||
"text": "## [Contact Truera team](https://truera.com/#elementor-action%3Aaction%3Dpopup%3Aopen%26settings%3DeyJpZCI6IjI2MDAiLCJ0b2dnbGUiOmZhbHNlfQ%3D%3D)" | ||
}, | ||
"linkedEntityGuids": null | ||
}, | ||
{ | ||
"visualization": { | ||
"id": "viz.line" | ||
}, | ||
"layout": { | ||
"column": 1, | ||
"row": 5, | ||
"height": 3, | ||
"width": 10 | ||
}, | ||
"title": "Model Instability(NUMERICAL_WASSERSTEIN)", | ||
"rawConfiguration": { | ||
"legend": { | ||
"enabled": true | ||
}, | ||
"nrqlQueries": [ | ||
{ | ||
"accountId": 3206889, | ||
"query": "SELECT latest(model_msi_v1) FROM Metric where segment='AllData' and DistributionComparisonType='NUMERICAL_WASSERSTEIN' FACET dimensions() TIMESERIES " | ||
} | ||
], | ||
"yAxisLeft": { | ||
"zero": true | ||
} | ||
}, | ||
"linkedEntityGuids": null | ||
} | ||
] | ||
} | ||
] | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.