diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/config.yaml b/quickstarts/audit/k8s-data-ingestion-analysis/config.yaml new file mode 100644 index 0000000000..2da63ead2b --- /dev/null +++ b/quickstarts/audit/k8s-data-ingestion-analysis/config.yaml @@ -0,0 +1,34 @@ +id: 15c23bf4-3dfa-47ba-850d-362bd8600fa0 +# Name of the quickstart (required) +name: audit-kubernetes-data-ingest-analysis + +# Displayed in the UI (required) +title: Kubernetes Data Ingest Analysis + +# Long-form description of the quickstart (required) +description: | + Paginated dashboard that analyzes overall data ingest generated by the New Relic Kubernetes integration and outlines options available to fine-tune your environment to help control ingest and cost. + +# Displayed in search results and recommendations. Summarizes a quickstarts functionality. +summary: | + Paginated dashboard that analyzes overall data ingest generated by the New Relic Kubernetes integration and outlines options available to fine-tune your environment to help control ingest and cost. + +# Support level: New Relic | Verified | Community (required) +level: New Relic + +# Authors of the quickstart (required) +authors: + - New Relic + - Zack Mutchler + +# Keywords for filtering / searching criteria in the UI +keywords: + - k8s + - kubernetes + - data ingest + - audit + +documentation: + - name: New Relic Kubernetes integration + url: https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/get-started/introduction-kubernetes-integration/ + description: Introduction to the New Relic Kubernetes integration diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis.json b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis.json new file mode 100644 index 0000000000..918ebcd347 --- /dev/null +++ b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis.json @@ -0,0 +1,847 @@ +{ + "name": "Kubernetes Data Ingest Analysis", + "description": null, + "pages": [ + { + "name": "Summary", + "description": null, + "widgets": [ + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 1, + "row": 1, + "height": 5, + "width": 6 + }, + "title": "", + "rawConfiguration": { + "text": "## Overview\nThis page analyzes the last 24 hours of data ingested by the New Relic Kubernetes integration and then estimates 1 month's usage. \n\nAll recommendations on this page are relative to the needs of the [Kubernetes Cluster Explorer](https://docs.newrelic.com/docs/integrations/kubernetes-integration/understand-use-data/kubernetes-cluster-explorer). As such, greater manipulation of both the scrape interval and `kube-state-metrics` is possible for teams who are not interested in utilizing the features and functionality of the Kubernetes Cluster Explorer.\n\nGranular details about the various event types and the amount of data they are generating can be found on the *\"Usage Breakdown -** pages.\n\nThe event types created by these integrations are described in greater detail in our docs:\n* [Kubernetes Data Types](https://docs.newrelic.com/docs/integrations/kubernetes-integration/understand-use-data/find-use-your-kubernetes-data#event-types)\n* [Kubernetes Events Integration](https://docs.newrelic.com/docs/integrations/kubernetes-integration/kubernetes-events/install-kubernetes-events-integration#use-data)\n* [Infrastructure Data Types](https://docs.newrelic.com/docs/infrastructure/manage-your-data/data-instrumentation/default-infrastructure-events#infrastructure-events)" + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 1, + "height": 5, + "width": 6 + }, + "title": "Current Total K8s Ingest", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "ONE MONTH (GB)", + "precision": 2, + "type": "humanized" + }, + { + "name": "ONE DAY (GB)", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nK8sApiServerSample,\nK8sClusterSample,\nK8sContainerSample,\nK8sControllerManagerSample,\nK8sDaemonsetSample,\nK8sDeploymentSample,\nK8sEndpointSample,\nK8sNamespaceSample,\nK8sNodeSample,\nK8sPodSample,\nK8sReplicasetSample,\nK8sSchedulerSample,\nK8sServiceSample,\nK8sStatefulsetSample,\nK8sVolumeSample,\nProcessSample,\nNetworkSample,\nStorageSample,\nSystemSample\nSELECT\nbytecountestimate()/10e8 AS 'ONE DAY (GB)',\nrate(bytecountestimate()/10e8, 1 MONTH) AS 'ONE MONTH (GB)'\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO COMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 1, + "row": 6, + "height": 5, + "width": 6 + }, + "title": "", + "rawConfiguration": { + "text": "## Required Metrics\nThis widget breaks out the data types that are required for the Kubernetes Cluster Explorer to maintain functionality. These include: \n```\n# Kubernetes Data Types\n K8sClusterSample,\n K8sContainerSample,\n K8sNodeSample,\n K8sPodSample,\n K8sVolumeSample,\n K8sApiServerSample,**\n K8sControllerManagerSample,**\n K8sEtcdSample,**\n K8sSchedulerSample**\n# Kubernetes Events Integration\n InfrastructureEvent\n# Infrastructure Data Types\n NetworkSample,\n StorageSample,\n SystemSample\n```\n***Control Plane metrics are not collected on hosted Kubernetes clusters by default*" + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 6, + "height": 5, + "width": 6 + }, + "title": "Required Metrics", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "ONE MONTH (GB)", + "precision": 2, + "type": "humanized" + }, + { + "name": "ONE DAY (GB)", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nK8sClusterSample,\nK8sContainerSample,\nK8sNodeSample,\nK8sPodSample,\nK8sVolumeSample,\nInfrastructureEvent,\nNetworkSample,\nStorageSample,\nSystemSample,\nK8sApiServerSample,\nK8sControllerManagerSample,\nK8sEtcdSample,\nK8sSchedulerSample\nSELECT bytecountestimate()/10e8 AS 'ONE DAY (GB)',\nrate(bytecountestimate()/10e8, 1 MONTH) AS 'ONE MONTH (GB)'\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO COMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + } + ] + }, + { + "name": "Potential Savings (GB)", + "description": null, + "widgets": [ + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 1, + "row": 1, + "height": 5, + "width": 6 + }, + "title": "", + "rawConfiguration": { + "text": "## Scrape Interval Reduction - Potential Savings\nThis widget shows the potential savings (estimated at 25% reduction per data type) you can achieve by changing the default scrape interval of `newrelic-infrastructure` from 15s to 20s.\n\nThis change includes adding a `ConfigMap` to provide the custom definition, and then adding associated `volumes` and `volumeMounts` entries to the `Daemonset`. Sample manifest can be found on the **Configuration Options** tab." + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 1, + "height": 5, + "width": 6 + }, + "title": "Scrape Interval Reduction - Potential Savings", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "SAVED PER MONTH (GB)", + "precision": 2, + "type": "humanized" + }, + { + "name": "SAVED PER DAY (GB)", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nK8sClusterSample,\nK8sContainerSample,\nK8sNodeSample,\nK8sPodSample,\nK8sVolumeSample,\nNetworkSample,\nStorageSample,\nSystemSample,\nProcessSample,\nK8sApiServerSample,\nK8sControllerManagerSample,\nK8sEtcdSample,\nK8sSchedulerSample,\nK8sDaemonsetSample,\nK8sDeploymentSample,\nK8sEndpointSample,\nK8sNamespaceSample,\nK8sReplicasetSample,\nK8sServiceSample,\nK8sStatefulsetSample\nSELECT (bytecountestimate()/10e8)*.25 AS 'SAVED PER DAY (GB)',\nrate((bytecountestimate()/10e8)*.25, 1 MONTH) AS 'SAVED PER MONTH (GB)'\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 1, + "row": 6, + "height": 5, + "width": 6 + }, + "title": "", + "rawConfiguration": { + "text": "## Optional Metrics - Potential Savings\nThis widget breaks out the data types that are NOT required and can be removed without impacting the Kubernetes Cluster Explorer. These include:\n```\n# Kubernetes Data Types\n K8sDaemonsetSample,\n K8sDeploymentSample,\n K8sEndpointSample,\n K8sNamespaceSample,\n K8sReplicasetSample,\n K8sServiceSample,\n K8sStatefulsetSample\n\n# Infrastructure Data Types\n ProcessSample\n```\n\nGuidance on disabling KSM collectors and `ProcessSample` events are available for both Helm and Manifest deployments on the **Configuration Options** tab." + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 6, + "height": 5, + "width": 6 + }, + "title": "Optional Metrics - Potential Savings", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "SAVED PER MONTH (GB)", + "precision": 2, + "type": "humanized" + }, + { + "name": "SAVED PER DAY (GB)", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nProcessSample,\nK8sDaemonsetSample,\nK8sDeploymentSample,\nK8sEndpointSample,\nK8sNamespaceSample,\nK8sReplicasetSample,\nK8sServiceSample,\nK8sStatefulsetSample\nSELECT bytecountestimate()/10e8 AS 'SAVED PER DAY (GB)',\nrate(bytecountestimate()/10e8, 1 MONTH) AS 'SAVED PER MONTH (GB)'\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO\nLIMIT MAX" + } + ] + } + } + ] + }, + { + "name": "Usage Breakdown - Required Metrics", + "description": null, + "widgets": [ + { + "visualization": { + "id": "viz.pie" + }, + "layout": { + "column": 1, + "row": 1, + "height": 6, + "width": 6 + }, + "title": "Data Type Breakdown", + "rawConfiguration": { + "facet": { + "showOtherSeries": false + }, + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nK8sClusterSample,\nK8sContainerSample,\nK8sNodeSample,\nK8sPodSample,\nK8sVolumeSample,\nInfrastructureEvent,\nNetworkSample,\nStorageSample,\nSystemSample,\nK8sApiServerSample,\nK8sControllerManagerSample,\nK8sEtcdSample,\nK8sSchedulerSample\nSELECT bytecountestimate()/10e8 AS 'GB'\nFACET eventType()\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 1, + "height": 3, + "width": 3 + }, + "title": "K8sApiServerSample Comparison", + "rawConfiguration": { + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sApiServerSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 10, + "row": 1, + "height": 3, + "width": 3 + }, + "title": "K8sClusterSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sClusterSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 4, + "height": 3, + "width": 3 + }, + "title": "K8sContainerSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sContainerSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 10, + "row": 4, + "height": 3, + "width": 3 + }, + "title": "K8sControllerManagerSample Comparison", + "rawConfiguration": { + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sControllerManagerSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 1, + "row": 7, + "height": 3, + "width": 4 + }, + "title": "K8sEtcdSample Comparison", + "rawConfiguration": { + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sEtcdSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 5, + "row": 7, + "height": 3, + "width": 4 + }, + "title": "K8sNodeSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sNodeSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 9, + "row": 7, + "height": 3, + "width": 4 + }, + "title": "K8sPodSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sPodSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 1, + "row": 10, + "height": 3, + "width": 4 + }, + "title": "K8sSchedulerSample Comparison", + "rawConfiguration": { + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sSchedulerSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 5, + "row": 10, + "height": 3, + "width": 4 + }, + "title": "K8sVolumeSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sVolumeSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 9, + "row": 10, + "height": 3, + "width": 4 + }, + "title": "Kubernetes Events Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM InfrastructureEvent SELECT\nbytecountestimate()/10e8 AS 'GB'\nWHERE clusterName IS NOT NULL\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 1, + "row": 13, + "height": 3, + "width": 4 + }, + "title": "NetworkSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM NetworkSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nWHERE clusterName IS NOT NULL\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 5, + "row": 13, + "height": 3, + "width": 4 + }, + "title": "StorageSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM StorageSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nWHERE clusterName IS NOT NULL\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 9, + "row": 13, + "height": 3, + "width": 4 + }, + "title": "SystemSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM SystemSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nWHERE clusterName IS NOT NULL\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + } + ] + }, + { + "name": "Usage Breakdown - Optional Metrics", + "description": null, + "widgets": [ + { + "visualization": { + "id": "viz.pie" + }, + "layout": { + "column": 1, + "row": 1, + "height": 6, + "width": 6 + }, + "title": "Data Type Breakdown", + "rawConfiguration": { + "facet": { + "showOtherSeries": false + }, + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM\nProcessSample,\nK8sDaemonsetSample,\nK8sDeploymentSample,\nK8sEndpointSample,\nK8sNamespaceSample,\nK8sReplicasetSample,\nK8sServiceSample,\nK8sStatefulsetSample\nSELECT bytecountestimate()/10e8 AS 'GB'\nFACET eventType()\nWHERE clusterName IS NOT NULL\nSINCE 1 DAY AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 1, + "height": 3, + "width": 3 + }, + "title": "K8sDaemonsetSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sDaemonsetSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 10, + "row": 1, + "height": 3, + "width": 3 + }, + "title": "K8sDeploymentSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sDeploymentSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 4, + "height": 3, + "width": 3 + }, + "title": "K8sEndpointSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sEndpointSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 10, + "row": 4, + "height": 3, + "width": 3 + }, + "title": "K8sNamespaceSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sNamespaceSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 1, + "row": 7, + "height": 3, + "width": 3 + }, + "title": "K8sReplicasetSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sReplicasetSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 4, + "row": 7, + "height": 3, + "width": 3 + }, + "title": "K8sServiceSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sServiceSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 7, + "row": 7, + "height": 3, + "width": 3 + }, + "title": "K8sStatefulsetSample Comparison", + "rawConfiguration": { + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM K8sStatefulsetSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + }, + { + "visualization": { + "id": "viz.billboard" + }, + "layout": { + "column": 10, + "row": 7, + "height": 3, + "width": 3 + }, + "title": "ProcessSample Comparison", + "rawConfiguration": { + "dataFormatters": [ + { + "name": "GB", + "precision": 2, + "type": "humanized" + } + ], + "nrqlQueries": [ + { + "accountId": 0, + "query": "FROM ProcessSample SELECT\nbytecountestimate()/10e8 AS 'GB'\nWHERE clusterName IS NOT NULL\nSINCE 24 HOURS AGO\nCOMPARE WITH 1 WEEK AGO\nLIMIT MAX" + } + ] + } + } + ] + }, + { + "name": "Configuration Options", + "description": null, + "widgets": [ + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 1, + "row": 1, + "height": 10, + "width": 4 + }, + "title": "", + "rawConfiguration": { + "text": "# Manifest: Scrape Interval\n\nThis manifest adds the ConfigMap, Volume, and VolumeMount required to provide a custom definition file and update the default scrape interval for `nri-kubernetes` daemon from 15s to 20s. You will need to update the `` and `` placeholders in the environment variables section of the Daemonset.\n\n*Note that customization of the scrape interval is not supported on Helm or Kubernetes on Windows at this time.*\n\n---\n\n```yaml\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n name: newrelic\n namespace: default\nautomountServiceAccountToken: true\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n name: newrelic\nrules:\n- apiGroups: [\"\"]\n resources:\n - \"nodes\"\n - \"nodes/metrics\"\n - \"nodes/stats\"\n - \"nodes/proxy\"\n - \"pods\"\n - \"secrets\"\n - \"services\"\n verbs: [\"get\", \"list\"]\n- nonResourceURLs: [\"/metrics\"]\n verbs: [\"get\"]\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n name: newrelic\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: newrelic\nsubjects:\n- kind: ServiceAccount\n name: newrelic\n namespace: default\n---\napiVersion: apps/v1\nkind: DaemonSet\nmetadata:\n name: newrelic-infra\n namespace: default\n labels:\n app: newrelic-infra\nspec:\n selector:\n matchLabels:\n name: newrelic-infra\n updateStrategy:\n type: RollingUpdate\n template:\n metadata:\n labels:\n name: newrelic-infra\n spec:\n serviceAccountName: newrelic\n hostNetwork: true # This option is a requirement for the Infrastructure Agent to report the proper hostname in New Relic.\n dnsPolicy: ClusterFirstWithHostNet\n containers:\n - name: newrelic-infra\n image: newrelic/infrastructure-k8s:2.0.1\n securityContext:\n privileged: true\n resources:\n limits:\n memory: 300M\n requests:\n cpu: 100m\n memory: 150M\n volumeMounts:\n - mountPath: /host\n name: host-volume\n readOnly: true\n - mountPath: /var/run/docker.sock\n name: host-docker-socket\n - mountPath: /var/db/newrelic-infra/integrations.d/\n name: nri-default-integration-cfg-volume\n - mountPath: /etc/newrelic-infra/integrations.d/\n name: nri-integration-cfg-volume\n # Volume Mount added for custom defintion file used to set scrape interval\n - mountPath: /var/db/newrelic-infra/newrelic-integrations/nri-kubernetes-definition.yml\n name: nri-kubernetes-definition-cfg-volume\n subPath: nri-kubernetes-definition.yml\n env:\n - name: \"CLUSTER_NAME\"\n value: \"\"\n - name: \"NRIA_LICENSE_KEY\"\n value: \"\"\n - name: \"NRIA_VERBOSE\"\n value: \"0\"\n # - name: \"KUBE_STATE_METRICS_POD_LABEL\" # Enables discovery of the KSM pod via a label. The value of the label needs to be \"true\".\n # value: \"\" # Remember to replace this placeholder with the label name of your choice.\n # - name: \"KUBE_STATE_METRICS_PORT\" # If the KUBE_STATE_METRICS_POD_LABEL is present, it changes the port queried in the pod.\n # value: \"8080\"\n # - name: \"KUBE_STATE_METRICS_SCHEME\" # If the KUBE_STATE_METRICS_POD_LABEL is present, it changes the scheme used to send to request to the pod.\n # value: \"http\"\n # - name: \"CADVISOR_PORT\" # Enable direct connection to cAdvisor by specifying the port. Needed for Kubernetes versions prior to 1.7.6.\n # value: \"4194\"\n # - name: \"KUBE_STATE_METRICS_URL\" # If this value is specified then discovery process for kube-state-metrics endpoint won't be triggered.\n # value: \"http://172.17.0.3:8080\" # This is example value. Only HTTP request is accepted.\n # - name: \"ETCD_TLS_SECRET_NAME\" # Name of the secret containing the cacert, cert and key used for setting the mTLS config for retrieving metrics from ETCD.\n # value: \"newrelic-infra-etcd-tls-secret\"\n # - name: \"ETCD_TLS_SECRET_NAMESPACE\" # Namespace where the the secret specified in ETCD_TLS_SECRET_NAME was created.\n # value: \"default\"\n # Note: Usage of API_SERVER_SECURE_PORT has been deprecated in favor of API_SERVER_ENDPOINT_URL.\n # - name: API_SERVER_SECURE_PORT\n # value: \"6443\"\n # - name: \"SCHEDULER_ENDPOINT_URL\"\n # value: \"https://localhost:10259\"\n # - name: \"ETCD_ENDPOINT_URL\"\n # value: \"https://localhost:9979\"\n # - name: \"CONTROLLER_MANAGER_ENDPOINT_URL\"\n # value: \"https://localhost:10257\"\n # - name: \"API_SERVER_ENDPOINT_URL\"\n # value: \"https://localhost:6443\"\n - name: \"NRIA_DISPLAY_NAME\"\n valueFrom:\n fieldRef:\n apiVersion: \"v1\"\n fieldPath: \"spec.nodeName\"\n - name: \"NRK8S_NODE_NAME\"\n valueFrom:\n fieldRef:\n apiVersion: \"v1\"\n fieldPath: \"spec.nodeName\"\n - name: \"NRIA_CUSTOM_ATTRIBUTES\"\n value: '{\"clusterName\":\"$(CLUSTER_NAME)\"}'\n - name: \"NRIA_PASSTHROUGH_ENVIRONMENT\"\n value: \"KUBERNETES_SERVICE_HOST,KUBERNETES_SERVICE_PORT,CLUSTER_NAME,CADVISOR_PORT,NRK8S_NODE_NAME,KUBE_STATE_METRICS_URL,KUBE_STATE_METRICS_POD_LABEL,ETCD_TLS_SECRET_NAME,ETCD_TLS_SECRET_NAMESPACE,API_SERVER_SECURE_PORT,KUBE_STATE_METRICS_SCHEME,KUBE_STATE_METRICS_PORT,SCHEDULER_ENDPOINT_URL,ETCD_ENDPOINT_URL,CONTROLLER_MANAGER_ENDPOINT_URL,API_SERVER_ENDPOINT_URL,DISABLE_KUBE_STATE_METRICS,NETWORK_ROUTE_FILE\"\n volumes:\n - name: host-volume\n hostPath:\n path: /\n - name: host-docker-socket\n hostPath:\n path: /var/run/docker.sock\n - name: nri-default-integration-cfg-volume\n configMap:\n name: nri-default-integration-cfg\n - name: nri-integration-cfg-volume\n configMap:\n name: nri-integration-cfg\n # Volume added for custom defintion file used to set scrape interval\n - name: nri-kubernetes-definition-cfg-volume\n configMap:\n name: nri-kubernetes-definition-cfg\n\n tolerations:\n - operator: \"Exists\"\n effect: \"NoSchedule\"\n - operator: \"Exists\"\n effect: \"NoExecute\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: nri-default-integration-cfg # integrations config provided by default\n namespace: default\ndata:\n nri-kubernetes-config.yml: |\n integration_name: com.newrelic.kubernetes\n instances:\n - name: nri-kubernetes\n command: metrics\n---\n# Custom Definition File to Set New Scrape Interval\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: nri-kubernetes-definition-cfg\n namespace: default\ndata:\n nri-kubernetes-definition.yml: |\n name: com.newrelic.kubernetes\n description: \"This NR Infrastructure definition uses a non-default 20s interval\"\n protocol_version: 2\n os: linux\n commands:\n metrics:\n command:\n - ./bin/nri-kubernetes\n - --metrics\n interval: 20\n```" + } + }, + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 5, + "row": 1, + "height": 10, + "width": 4 + }, + "title": "", + "rawConfiguration": { + "text": "# Helm Chart: Disabling Optional KSM metrics and ProcessSample\nNote that this requires that you have installed KSM via Helm. This is embedded in nri-bundle, but not in the stand-alone newrelic-infrastructure chart. \n\n## Helm command to disable \"optional\" collectors for New Relic Kubernetes integration\n[Available Collectors for KSM Helm chart](https://github.com/kubernetes/kube-state-metrics/blob/fb48293a4012c16392f370e9ea8c77a1b66d24c3/charts/kube-state-metrics/values.yaml#L112-L142)\n\n---\n\n```shell\nhelm newrelic-bundle newrelic/nri-bundle \\\n ...\n --set kube-state-metrics.collectors.daemonsets=false \\\n --set kube-state-metrics.collectors.deployments=false \\\n --set kube-state-metrics.collectors.endpoints=false \\\n --set kube-state-metrics.collectors.namespaces=false \\\n --set kube-state-metrics.collectors.replicasets=false \\\n --set kube-state-metrics.collectors.services=false \\\n --set kube-state-metrics.collectors.statefulsets=false\n```\n\n---\n## Helm command to disable \"ProcessSample\" events for New Relic Kubernetes integration\n[Disabling Process collection for New Relic](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings#enable-process-metrics)\n\n```shell\nhelm newrelic-bundle newrelic/nri-bundle \\\n ...\n --set enableProcessMetrics=false\n```" + } + }, + { + "visualization": { + "id": "viz.markdown" + }, + "layout": { + "column": 9, + "row": 1, + "height": 10, + "width": 4 + }, + "title": "", + "rawConfiguration": { + "text": "# Manifest: Disabling Optional KSM metrics and ProcessSample\n---\n## YAML Key Path: spec.template.spec.name.args\nComment out these items to update the kube-state-metrics Deployment config\n\n```yaml\n[spec]\n [template]\n [spec]\n [containers]\n [name=kube-state-metrics]\n [args]\n #- --collectors=daemonsets\n #- --collectors=deployments\n #- --collectors=endpoints\n #- --collectors=namespaces\n #- --collectors=replicasets\n #- --collectors=services\n #- --collectors=statefulsets\n```\n---\n## YAML Key Path: rules.apiGroups\nComment out these items to update the kube-state-metrics ClusterRole config\n\n```yaml\n[rules]\n# - apiGroups: [\"extensions\", \"apps\"]\n# resources:\n# - daemonsets\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"extensions\", \"apps\"]\n# resources:\n# - deployments\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"\"]\n# resources:\n# - endpoints\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"\"]\n# resources:\n# - namespaces\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"extensions\", \"apps\"]\n# resources:\n# - replicasets\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"\"]\n# resources:\n# - services\n# verbs: [\"list\", \"watch\"]\n\n# - apiGroups: [\"apps\"]\n# resources:\n# - statefulsets\n# verbs: [\"list\", \"watch\"]\n```\n---\n## YAML Key Path: spec.template.spec.containers.name.env\nAdd [environment variable](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/configuration/infrastructure-agent-configuration-settings#enable-process-metrics) to disable ProcessSample collection in the newrelic-infrastructure Daemonset\n\n```yaml\n[spec]\n [template]\n [spec]\n [containers]\n [name=newrelic-infrastructure]\n [env]\n - name: \"NRIA_ENABLE_PROCESS_METRICS\"\n value: false\n``` " + } + } + ] + } + ] +} \ No newline at end of file diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis01.png b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis01.png new file mode 100644 index 0000000000..ee2b298115 Binary files /dev/null and b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis01.png differ diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis02.png b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis02.png new file mode 100644 index 0000000000..72b73cbe5e Binary files /dev/null and b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis02.png differ diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis03.png b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis03.png new file mode 100644 index 0000000000..eec19d628d Binary files /dev/null and b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis03.png differ diff --git a/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis04.png b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis04.png new file mode 100644 index 0000000000..aa18e75ac5 Binary files /dev/null and b/quickstarts/audit/k8s-data-ingestion-analysis/dashboards/k8s-data-ingest-analysis04.png differ