Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(PE-28451) switch from the v1 to v2 metrics api #57

Merged
merged 5 commits into from
Mar 17, 2020
Merged

(PE-28451) switch from the v1 to v2 metrics api #57

merged 5 commits into from
Mar 17, 2020

Conversation

tkishel
Copy link
Contributor

@tkishel tkishel commented Mar 11, 2020

With this commit ...

We switch from the v1 to v2 Metrics API for additional metrics (for PuppetDB),
and drop metrics for unsupported versions of PE.

(This is required in response to PE-28451 / PE-28468.

Note these additional metrics generate errors in PE 2019.4 via v2:

puppetlabs.puppetdb.ha:name=failed-request-counter
puppetlabs.puppetdb.mq:name=global.generate-retry-message-time
puppetlabs.puppetdb.mq:name=global.retry-persistence-time

With this commit ...

We switch from the v1 to v2 Metrics API for additional metrics (for PuppetDB),
and drop metrics for unsupported versions of PE.

(This is required in response to PE-28451 / PE-28468.

Note these additional metrics generate errors in PE 2019.4 via v2:

  puppetlabs.puppetdb.ha:name=failed-request-counter
  puppetlabs.puppetdb.mq:name=global.generate-retry-message-time
  puppetlabs.puppetdb.mq:name=global.retry-persistence-time
@tkishel tkishel requested a review from Sharpie March 11, 2020 23:30
@tkishel tkishel requested a review from m0dular March 12, 2020 20:06
@tkishel
Copy link
Contributor Author

tkishel commented Mar 12, 2020

I've manually tested this on 2018111, 201921, and 201940; and metrics for puppetdb, puppetserver, and orchestrator are collected without error (after commenting out the three additional metrics for puppetdb as per the commit message).

@tkishel
Copy link
Contributor Author

tkishel commented Mar 13, 2020

Like Krylon: No runs, no drips, no errors ...

[root@pe-201940-master ~]# /opt/puppetlabs/puppet-metrics-collector/scripts/tk_metrics --metrics_type puppetdb
{"timestamp":"2020-03-13T15:16:04Z","servers":{"pe-201940-master-puppetdebug-vlan":{"puppetdb":{"puppetdb-status":{"service_version":"6.9.1-20200306_235136-gbfdda4a","service_status_version":1,"detail_level":"debug","state":"running","status":{"maintenance_mode?":false,"queue_depth":0,"read_db_up?":true,"write_db_up?":true,"rbac_status":"running","sync_status":{"state":"idle"}},"active_alerts":[]},"status-service":{"service_version":"1.1.0","service_status_version":1,"detail_level":"debug","state":"running","status":{"experimental":{"jvm-metrics":{"cpu-usage":0.19996001,"up-time-ms":99014249,"gc-cpu-usage":0.0,"threading":{"thread-count":65,"peak-thread-count":68},"heap-memory":{"committed":126877696,"init":67108864,"max":134217728,"used":84994264},"gc-stats":{"G1 Young Generation":{"count":1518,"total-time-ms":18582,"last-gc-info":{"duration-ms":21}},"G1 Old Generation":{"count":0,"total-time-ms":0}},"start-time-ms":1583861857816,"file-descriptors":{"used":63,"max":4096},"non-heap-memory":{"committed":355688448,"init":7667712,"max":-1,"used":193497048}}}},"active_alerts":[]},"global_command-parse-time":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"global_discarded":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"global_fatal":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"global_message-persistence-time":{"75thPercentile":142.604082,"Mean":85.78193137858966,"StdDev":40.10027311923665,"98thPercentile":142.604082,"RateUnit":"events/second","95thPercentile":142.604082,"99thPercentile":142.604082,"Max":142.604082,"Count":129,"FiveMinuteRate":5.661215681937322e-17,"50thPercentile":82.855102,"MeanRate":0.0013032409937433756,"Min":6.801457,"OneMinuteRate":2.8929432674120234e-73,"DurationUnit":"milliseconds","999thPercentile":142.604082,"FifteenMinuteRate":7.070875488359495e-08},"global_retried":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"global_retry-counts":{"SnapshotSize":7,"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":0,"999thPercentile":0.0,"Count":129,"50thPercentile":0.0},"global_seen":{"RateUnit":"events/second","OneMinuteRate":2.8929432674120234e-73,"Count":129,"FifteenMinuteRate":7.070875488359505e-08,"FiveMinuteRate":5.661215681937322e-17,"MeanRate":0.0013032409947441896},"global_processed":{"RateUnit":"events/second","OneMinuteRate":2.892943267417102e-73,"Count":129,"FifteenMinuteRate":7.077261679984099e-08,"FiveMinuteRate":5.661900115181431e-17,"MeanRate":0.0013032409946311446},"global_processing-time":{"75thPercentile":2118.722391,"Mean":1681.1932779389044,"StdDev":500.5252958272022,"98thPercentile":2118.722391,"RateUnit":"events/second","95thPercentile":2118.722391,"99thPercentile":2118.722391,"Max":3217.079069,"Count":129,"FiveMinuteRate":5.661900115181431e-17,"50thPercentile":1667.930893,"MeanRate":0.0013032409952186458,"Min":851.779902,"OneMinuteRate":2.892943267417102e-73,"DurationUnit":"milliseconds","999thPercentile":2118.722391,"FifteenMinuteRate":7.077261679984099e-08},"storage_add-edges":{"75thPercentile":239.973021,"Mean":239.973021,"StdDev":0.0,"98thPercentile":239.973021,"RateUnit":"events/second","95thPercentile":239.973021,"99thPercentile":239.973021,"Max":1420.2792,"Count":3,"FiveMinuteRate":1.3548343549795395e-19,"50thPercentile":239.973021,"MeanRate":3.0306809115747534e-05,"Min":239.973021,"OneMinuteRate":1.8492113502790446e-84,"DurationUnit":"milliseconds","999thPercentile":239.973021,"FifteenMinuteRate":3.820253585658231e-09},"storage_add-resources":{"75thPercentile":1277.816188,"Mean":1277.816188,"StdDev":0.0,"98thPercentile":1277.816188,"RateUnit":"events/second","95thPercentile":1277.816188,"99thPercentile":1277.816188,"Max":4535.027097,"Count":3,"FiveMinuteRate":1.3548343549795395e-19,"50thPercentile":1277.816188,"MeanRate":3.0306808843819922e-05,"Min":1277.816188,"OneMinuteRate":1.8492113502790446e-84,"DurationUnit":"milliseconds","999thPercentile":1277.816188,"FifteenMinuteRate":3.820253585658231e-09},"storage_catalog-hash":{"75thPercentile":1402.26812,"Mean":1402.2681199094563,"StdDev":0.005858070946624684,"98thPercentile":1402.26812,"RateUnit":"events/second","95thPercentile":1402.26812,"99thPercentile":1402.26812,"Max":1402.26812,"Count":43,"FiveMinuteRate":1.8633059854388444e-17,"50thPercentile":1402.26812,"MeanRate":0.00043439759330045764,"Min":572.9428419999999,"OneMinuteRate":8.777939108976516e-74,"DurationUnit":"milliseconds","999thPercentile":1402.26812,"FifteenMinuteRate":2.347127348284861e-08},"storage_catalog-hash-match-time":{"75thPercentile":3.406177,"Mean":3.406177,"StdDev":0.0,"98thPercentile":3.406177,"RateUnit":"events/second","95thPercentile":3.406177,"99thPercentile":3.406177,"Max":79.730867,"Count":40,"FiveMinuteRate":1.8499815763110536e-17,"50thPercentile":3.406177,"MeanRate":0.00040409078528734884,"Min":3.406177,"OneMinuteRate":8.777939108806374e-74,"DurationUnit":"milliseconds","999thPercentile":3.406177,"FifteenMinuteRate":1.967218468271482e-08},"storage_catalog-hash-miss-time":{"75thPercentile":1604.390492,"Mean":1604.390492,"StdDev":0.0,"98thPercentile":1604.390492,"RateUnit":"events/second","95thPercentile":1604.390492,"99thPercentile":1604.390492,"Max":6003.718328,"Count":3,"FiveMinuteRate":1.3548343549795395e-19,"50thPercentile":1604.390492,"MeanRate":3.030680875340956e-05,"Min":1604.390492,"OneMinuteRate":1.8492113502790446e-84,"DurationUnit":"milliseconds","999thPercentile":1604.390492,"FifteenMinuteRate":3.820253585658231e-09},"storage_gc-catalogs-time":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"storage_gc-environments-time":{"75thPercentile":0.9622219999999999,"Mean":0.9622219999999999,"StdDev":3.397830425968875e-11,"98thPercentile":0.9622219999999999,"RateUnit":"events/second","95thPercentile":0.9622219999999999,"99thPercentile":0.9622219999999999,"Max":19.040319,"Count":28,"FiveMinuteRate":9.520167738641521e-06,"50thPercentile":0.9622219999999999,"MeanRate":0.00028286355100278735,"Min":0.7092379999999999,"OneMinuteRate":3.167854361449394e-15,"DurationUnit":"milliseconds","999thPercentile":0.9622219999999999,"FifteenMinuteRate":0.0001605852116923434},"storage_gc-fact-paths":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"storage_gc-params-time":{"75thPercentile":9.111377,"Mean":9.111377,"StdDev":8.85080081210982e-11,"98thPercentile":9.111377,"RateUnit":"events/second","95thPercentile":9.111377,"99thPercentile":9.111377,"Max":56.201907,"Count":28,"FiveMinuteRate":9.520167738641521e-06,"50thPercentile":9.111377,"MeanRate":0.0002828635506478865,"Min":8.518606,"OneMinuteRate":3.167854361449394e-15,"DurationUnit":"milliseconds","999thPercentile":9.111377,"FifteenMinuteRate":0.0001605852116923434},"storage_gc-report-statuses":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"storage_gc-time":{"75thPercentile":52.090970999999996,"Mean":52.090970999999996,"StdDev":3.358283203754212e-10,"98thPercentile":52.090970999999996,"RateUnit":"events/second","95thPercentile":52.090970999999996,"99thPercentile":52.090970999999996,"Max":230.76783799999998,"Count":28,"FiveMinuteRate":9.520167738641521e-06,"50thPercentile":52.090970999999996,"MeanRate":0.00028286355038352755,"Min":46.334572,"OneMinuteRate":3.167854361449394e-15,"DurationUnit":"milliseconds","999thPercentile":52.090970999999996,"FifteenMinuteRate":0.0001605852170585244},"storage_new-catalog-time":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"storage_new-catalogs":{"Count":3},"storage_replace-catalog-time":{"75thPercentile":1407.65879,"Mean":1407.6587903530183,"StdDev":0.022499859743586884,"98thPercentile":1407.65879,"RateUnit":"events/second","95thPercentile":1407.65879,"99thPercentile":1407.65879,"Max":2841.702923,"Count":43,"FiveMinuteRate":1.863529919860848e-17,"50thPercentile":1407.65879,"MeanRate":0.00043439759255842284,"Min":678.570695,"OneMinuteRate":8.77793910899134e-74,"DurationUnit":"milliseconds","999thPercentile":1407.65879,"FifteenMinuteRate":2.3492438268373112e-08},"storage_replace-facts-time":{"75thPercentile":462.61553299999997,"Mean":462.6155331614394,"StdDev":0.010289451127793472,"98thPercentile":462.61553299999997,"RateUnit":"events/second","95thPercentile":462.61553299999997,"99thPercentile":462.61553299999997,"Max":1118.420822,"Count":43,"FiveMinuteRate":1.7724314801977277e-17,"50thPercentile":462.61553299999997,"MeanRate":0.00043439759252182354,"Min":331.770247,"OneMinuteRate":6.836265851824023e-74,"DurationUnit":"milliseconds","999thPercentile":462.61553299999997,"FifteenMinuteRate":2.3083327455222853e-08},"storage_resource-hashes":{"75thPercentile":209.304856,"Mean":209.304856,"StdDev":0.0,"98thPercentile":209.304856,"RateUnit":"events/second","95thPercentile":209.304856,"99thPercentile":209.304856,"Max":253.353234,"Count":3,"FiveMinuteRate":1.3548343549795395e-19,"50thPercentile":209.304856,"MeanRate":3.0306808938344252e-05,"Min":209.304856,"OneMinuteRate":1.8492113502790446e-84,"DurationUnit":"milliseconds","999thPercentile":209.304856,"FifteenMinuteRate":3.820253585657784e-09},"storage_store-report-time":{"75thPercentile":1481.270182,"Mean":1481.2701820212174,"StdDev":0.0013422054920390644,"98thPercentile":1481.270182,"RateUnit":"events/second","95thPercentile":1481.270182,"99thPercentile":1481.270182,"Max":1566.1773799999999,"Count":43,"FiveMinuteRate":2.0254782163007266e-17,"50thPercentile":1481.270182,"MeanRate":0.0004343975911983036,"Min":490.009728,"OneMinuteRate":1.3315227713319766e-73,"DurationUnit":"milliseconds","999thPercentile":1481.270182,"FifteenMinuteRate":2.415415394552346e-08},"PDBReadPool_pool_ActiveConnections":{"Value":0},"PDBReadPool_pool_IdleConnections":{"Value":25},"PDBReadPool_pool_PendingConnections":{"Value":0},"PDBReadPool_pool_TotalConnections":{"Value":25},"PDBReadPool_pool_Usage":{"SnapshotSize":330,"75thPercentile":0.0,"Mean":null,"StdDev":null,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":749,"999thPercentile":0.0,"Count":2360,"50thPercentile":0.0},"PDBReadPool_pool_Wait":{"75thPercentile":0.002245,"Mean":null,"StdDev":null,"98thPercentile":0.002245,"RateUnit":"events/second","95thPercentile":0.002245,"99thPercentile":0.002245,"Max":171.326294,"Count":2360,"FiveMinuteRate":0.008305072341608115,"50thPercentile":0.002245,"MeanRate":0.023845697334691947,"Min":0.002245,"OneMinuteRate":0.006441170397454053,"DurationUnit":"milliseconds","999thPercentile":0.002245,"FifteenMinuteRate":0.010315973551935574},"PDBWritePool_pool_ActiveConnections":{"Value":0},"PDBWritePool_pool_IdleConnections":{"Value":2},"PDBWritePool_pool_PendingConnections":{"Value":0},"PDBWritePool_pool_TotalConnections":{"Value":2},"PDBWritePool_pool_Usage":{"SnapshotSize":152,"75thPercentile":0.0,"Mean":null,"StdDev":null,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":2865,"999thPercentile":0.0,"Count":883,"50thPercentile":0.0},"PDBWritePool_pool_Wait":{"75thPercentile":0.00156,"Mean":null,"StdDev":null,"98thPercentile":0.00156,"RateUnit":"events/second","95thPercentile":0.00156,"99thPercentile":0.00156,"Max":131.837401,"Count":883,"FiveMinuteRate":0.0044300987446615914,"50thPercentile":0.00156,"MeanRate":0.00892192784351763,"Min":0.00156,"OneMinuteRate":0.006437443011823482,"DurationUnit":"milliseconds","999thPercentile":0.00156,"FifteenMinuteRate":0.0048117925312396274},"mq_replace_catalog_retried":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"mq_replace_catalog_retry-counts":{"SnapshotSize":3,"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":0,"999thPercentile":0.0,"Count":43,"50thPercentile":0.0},"mq_replace_facts_retried":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"mq_replace_facts_retry-counts":{"SnapshotSize":3,"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":0,"999thPercentile":0.0,"Count":43,"50thPercentile":0.0},"mq_store_report_retried":{"RateUnit":"events/second","OneMinuteRate":0.0,"Count":0,"FifteenMinuteRate":0.0,"FiveMinuteRate":0.0,"MeanRate":0.0},"mq_store_reports_retry-counts":{"SnapshotSize":3,"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"Min":0,"95thPercentile":0.0,"99thPercentile":0.0,"Max":0,"999thPercentile":0.0,"Count":43,"50thPercentile":0.0},"ha_last-sync-succeeded":{"Value":false},"ha_seconds-since-last-successful-sync":{"Value":null},"ha_sync-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"ha_catalogs-sync-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"ha_reports-sync-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"ha_factsets-sync-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"ha_nodes-sync-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"ha_record-transfer-duration":{"75thPercentile":0.0,"Mean":0.0,"StdDev":0.0,"98thPercentile":0.0,"RateUnit":"events/second","95thPercentile":0.0,"99thPercentile":0.0,"Max":0.0,"Count":0,"FiveMinuteRate":0.0,"50thPercentile":0.0,"MeanRate":0.0,"Min":0.0,"OneMinuteRate":0.0,"DurationUnit":"milliseconds","999thPercentile":0.0,"FifteenMinuteRate":0.0},"error":[],"error_count":0,"api-query-start":"2020-03-13T15:16:04Z","api-query-duration":0.098372139}}}}
[root@pe-201940-master ~]# /opt/puppetlabs/puppet-metrics-collector/scripts/tk_metrics --metrics_type puppetserver
{"timestamp":"2020-03-13T15:15:55Z","servers":{"pe-201940-master-puppetdebug-vlan":{"puppetserver":{"puppet-profiler":{"service_version":"6.9.1-20200303_131311-g601dd34","service_status_version":1,"detail_level":"debug","state":"running","status":{"experimental":{"function-metrics":[{"function":"include","count":7877,"mean":14,"aggregate":110278},{"function":"puppetdb_query","count":524,"mean":103,"aggregate":53972},{"function":"template","count":3784,"mean":11,"aggregate":41624},{"function":"map","count":1857,"mean":13,"aggregate":24141},{"function":"each","count":1505,"mean":11,"aggregate":16555},{"function":"defined","count":9890,"mean":1,"aggregate":9890},{"function":"epp","count":238,"mean":31,"aggregate":7378},{"function":"filter","count":516,"mean":11,"aggregate":5676},{"function":"pe_validate_bool","count":4042,"mean":1,"aggregate":4042},{"function":"lookup","count":1290,"mean":2,"aggregate":2580},{"function":"inline_template","count":430,"mean":6,"aggregate":2580},{"function":"versioncmp","count":1286,"mean":1,"aggregate":1286},{"function":"pe_compile_master","count":860,"mean":1,"aggregate":860},{"function":"pe_build_version","count":172,"mean":4,"aggregate":688},{"function":"any","count":172,"mean":3,"aggregate":516},{"function":"hocon_data","count":86,"mean":5,"aggregate":430},{"function":"pe_compiling_server_aio_build","count":86,"mean":3,"aggregate":258},{"function":"file","count":258,"mean":1,"aggregate":258},{"function":"return","count":180,"mean":1,"aggregate":180},{"function":"pe_compiling_server_version","count":86,"mean":1,"aggregate":86},{"function":"pe_flatten","count":3268,"mean":0,"aggregate":0},{"function":"pe_keys","count":172,"mean":0,"aggregate":0},{"function":"scope_defaults","count":86,"mean":0,"aggregate":0},{"function":"pe_format_urls","count":258,"mean":0,"aggregate":0},{"function":"pe_concat_getparam","count":4730,"mean":0,"aggregate":0},{"function":"pe_clamp","count":258,"mean":0,"aggregate":0},{"function":"pe_concat","count":516,"mean":0,"aggregate":0},{"function":"match","count":43,"mean":0,"aggregate":0},{"function":"regsubst","count":11137,"mean":0,"aggregate":0},{"function":"contain","count":86,"mean":0,"aggregate":0},{"function":"pe_upcase","count":1032,"mean":0,"aggregate":0},{"function":"pe_wrap_join_keys_to_values","count":344,"mean":0,"aggregate":0},{"function":"pe_getvar","count":1032,"mean":0,"aggregate":0},{"function":"pe_is_integer","count":10492,"mean":0,"aggregate":0},{"function":"pe_count","count":258,"mean":0,"aggregate":0},{"function":"pe_prefix","count":172,"mean":0,"aggregate":0},{"function":"staging_parse","count":172,"mean":0,"aggregate":0},{"function":"pe_validate_absolute_path","count":3698,"mean":0,"aggregate":0},{"function":"pe_min","count":172,"mean":0,"aggregate":0},{"function":"pe_is_bool","count":2021,"mean":0,"aggregate":0},{"function":"pe_suffix","count":86,"mean":0,"aggregate":0},{"function":"pe_strip","count":602,"mean":0,"aggregate":0},{"function":"pe_is_string","count":21672,"mean":0,"aggregate":0},{"function":"pe_merge","count":344,"mean":0,"aggregate":0},{"function":"pe_join","count":12728,"mean":0,"aggregate":0},{"function":"pe_unique","count":602,"mean":0,"aggregate":0},{"function":"pe_delete_undef_values","count":2236,"mean":0,"aggregate":0},{"function":"pe_puppetserver_static_content_list","count":86,"mean":0,"aggregate":0},{"function":"pe_max","count":774,"mean":0,"aggregate":0},{"function":"pe_empty","count":26058,"mean":0,"aggregate":0}],"resource-metrics":[{"resource":"Class[Puppet_enterprise::Profile::Master]","count":43,"mean":2594,"aggregate":111542},{"resource":"Class[Puppet_enterprise::Profile::Controller]","count":43,"mean":1325,"aggregate":56975},{"resource":"Class[Puppet_enterprise::Cli_config]","count":43,"mean":1126,"aggregate":48418},{"resource":"Class[Puppet_enterprise::Profile::Console]","count":43,"mean":791,"aggregate":34013},{"resource":"Class[Puppet_enterprise::Master]","count":43,"mean":785,"aggregate":33755},{"resource":"Class[Puppet_metrics_collector]","count":43,"mean":783,"aggregate":33669},{"resource":"Class[Puppet_enterprise::Profile::Database]","count":43,"mean":609,"aggregate":26187},{"resource":"Class[Puppet_enterprise::Params]","count":43,"mean":358,"aggregate":15394},{"resource":"Class[Puppet_enterprise::Master::Puppetserver]","count":43,"mean":353,"aggregate":15179},{"resource":"Class[Puppet_enterprise::Profile::Puppetdb]","count":43,"mean":332,"aggregate":14276},{"resource":"Class[Puppet_enterprise::Profile::Orchestrator]","count":43,"mean":275,"aggregate":11825},{"resource":"Class[Puppet_enterprise::Profile::Console::Proxy]","count":43,"mean":217,"aggregate":9331},{"resource":"Class[Puppet_enterprise::Packages]","count":43,"mean":214,"aggregate":9202},{"resource":"Class[Puppet_enterprise::Puppetdb]","count":43,"mean":202,"aggregate":8686},{"resource":"Class[Puppet_enterprise::Console_services]","count":43,"mean":187,"aggregate":8041},{"resource":"Class[Pe_postgresql::Server]","count":43,"mean":164,"aggregate":7052},{"resource":"Class[Puppet_enterprise::Repo]","count":43,"mean":106,"aggregate":4558},{"resource":"Class[Puppet_enterprise::Certs::Puppetdb_whitelist]","count":43,"mean":85,"aggregate":3655},{"resource":"Class[Puppet_enterprise::Profile::Master::Auth_conf]","count":43,"mean":74,"aggregate":3182},{"resource":"Class[Puppet_enterprise::Profile::Console::Proxy::Http_redirect]","count":43,"mean":72,"aggregate":3096},{"resource":"Class[Puppet_enterprise::Profile::Console::Proxy::Nginx_conf]","count":43,"mean":70,"aggregate":3010},{"resource":"Class[Puppet_enterprise::Master::Tk_authz]","count":43,"mean":61,"aggregate":2623},{"resource":"Class[Puppet_enterprise::Puppetdb::Jetty_ini]","count":43,"mean":61,"aggregate":2623},{"resource":"Class[Puppet_enterprise::Profile::Certificate_authority]","count":43,"mean":58,"aggregate":2494},{"resource":"Class[Puppet_enterprise::Master::Installer_bolt]","count":43,"mean":58,"aggregate":2494},{"resource":"Class[Puppet_metrics_collector::Puppetserver]","count":39,"mean":56,"aggregate":2184},{"resource":"Class[Puppet_enterprise::Postgresql::Client]","count":43,"mean":45,"aggregate":1935},{"resource":"Class[Puppet_enterprise::Trapperkeeper::Analytics]","count":43,"mean":41,"aggregate":1763},{"resource":"Class[Puppet_enterprise::Profile::Agent]","count":43,"mean":39,"aggregate":1677},{"resource":"Class[Puppet_enterprise::Profile::Console::Console_services_config]","count":43,"mean":38,"aggregate":1634},{"resource":"Class[Pe_repo]","count":43,"mean":36,"aggregate":1548},{"resource":"Class[Puppet_enterprise::Profile::Master::Puppetdb]","count":43,"mean":32,"aggregate":1376},{"resource":"Class[Puppet_enterprise::Puppetdb::Rbac_consumer_conf]","count":43,"mean":32,"aggregate":1376},{"resource":"Class[Puppet_enterprise]","count":43,"mean":30,"aggregate":1290},{"resource":"Class[Pe_postgresql::Server::Service]","count":43,"mean":28,"aggregate":1204},{"resource":"Class[Puppet_enterprise::Trapperkeeper::Inventory]","count":43,"mean":22,"aggregate":946},{"resource":"Class[Pe_postgresql::Server::Config]","count":43,"mean":20,"aggregate":860},{"resource":"Class[Puppet_metrics_collector::Puppetdb]","count":39,"mean":22,"aggregate":858},{"resource":"Class[Pe_postgresql::Server::Reload]","count":43,"mean":18,"aggregate":774},{"resource":"Class[Puppet_enterprise::Symlinks]","count":43,"mean":18,"aggregate":774},{"resource":"Class[Puppet_enterprise::Puppetdb::Database_ini]","count":43,"mean":18,"aggregate":774},{"resource":"Class[Puppet_enterprise::Profile::Bolt_server]","count":43,"mean":18,"aggregate":774},{"resource":"Class[Puppet_enterprise::Pxp_agent]","count":43,"mean":16,"aggregate":688},{"resource":"Class[Puppet_enterprise::Profile::Master::Classifier]","count":43,"mean":14,"aggregate":602},{"resource":"Class[Puppet_enterprise::Profile::Plan_executor]","count":43,"mean":13,"aggregate":559},{"resource":"Class[Puppet_enterprise::Profile::Console::Certs]","count":43,"mean":10,"aggregate":430},{"resource":"Class[Puppet_enterprise::Master::Recover_configuration]","count":43,"mean":9,"aggregate":387},{"resource":"Class[Pe_postgresql::Client]","count":43,"mean":8,"aggregate":344},{"resource":"Class[Pe_repo::Platform::Windows_x86_64]","count":43,"mean":8,"aggregate":344},{"resource":"Class[Pe_postgresql::Server::Contrib]","count":43,"mean":8,"aggregate":344}],"catalog-metrics":[{"metric":"compile","count":43,"mean":12959,"aggregate":557237},{"metric":"find_node","count":43,"mean":114,"aggregate":4902}],"puppetdb-metrics":[{"metric":"command_submit_replace_catalog","count":43,"mean":1597,"aggregate":68671},{"metric":"query","count":524,"mean":96,"aggregate":50304},{"metric":"catalog_munge","count":43,"mean":456,"aggregate":19608},{"metric":"command_submit_store_report","count":43,"mean":297,"aggregate":12771},{"metric":"command_submit_replace_facts","count":43,"mean":196,"aggregate":8428},{"metric":"report_convert_to_wire_format_hash","count":43,"mean":100,"aggregate":4300},{"metric":"facts_encode","count":43,"mean":0,"aggregate":0}],"inline-metrics":[]}},"active_alerts":[]},"jruby-metrics":{"service_version":"6.9.1-20200303_131311-g601dd34","service_status_version":1,"detail_level":"debug","state":"running","status":{"experimental":{"jruby-pool-lock-status":{"current-state":":not-in-use","last-change-time":"2020-03-10T17:37:16.591Z"},"metrics":{"average-lock-wait-time":0,"num-free-jrubies":1,"borrow-count":1138,"average-requested-jrubies":"NaN","borrow-timeout-count":0,"return-count":1138,"borrow-timers":{"puppet-v3-report":{"count":43,"mean":618,"max":618,"rate":2.0595190723940514e-17},"puppet-v3-environment_classes":{"count":339,"mean":0,"max":1484,"rate":0.006585896547810582},"puppet-v3-file_metadata":{"count":360,"mean":18,"max":32,"rate":2.370700423444941e-16},"puppet-v3-environments":{"count":169,"mean":0,"max":3,"rate":0.0032615831427850912},"total":{"count":1138,"mean":0,"max":8,"rate":0.009841111335489757},"puppet-v3-catalog":{"count":43,"mean":17289,"max":17290,"rate":1.926462953325816e-17},"puppet-v3-environment_modules":{"count":3,"mean":74,"max":74,"rate":3.8727005305673576e-32},"puppet-v3-file_content":{"count":9,"mean":22,"max":155,"rate":2.302287735278268e-49},"puppet-v3-file_metadatas":{"count":129,"mean":144,"max":395,"rate":5.407308426658072e-17},"puppet-v3-node":{"count":43,"mean":235,"max":235,"rate":1.8024361422193543e-17}},"borrow-retry-count":0,"borrowed-instances":[],"average-borrow-time":0,"num-jrubies":1,"requested-count":1138,"queue-limit-hit-rate":0.0,"average-lock-held-time":0,"requested-instances":[],"queue-limit-hit-count":0,"average-free-jrubies":"NaN","num-pool-locks":0,"average-wait-time":0}}},"active_alerts":[]},"ca":{"service_version":"6.9.1-20200303_131311-g601dd34","service_status_version":1,"detail_level":"debug","state":"running","status":{},"active_alerts":[]},"master":{"service_version":"6.9.1-20200303_131311-g601dd34","service_status_version":1,"detail_level":"debug","state":"running","status":{"experimental":{"http-metrics":[{"route-id":"puppet-v3-catalog-/*/","count":43,"mean":17292,"aggregate":743556},{"route-id":"puppet-v3-report-/*/","count":43,"mean":643,"aggregate":27649},{"route-id":"puppet-v3-file_metadatas-/*/","count":129,"mean":146,"aggregate":18834},{"route-id":"puppet-v3-node-/*/","count":43,"mean":238,"aggregate":10234},{"route-id":"puppet-v3-file_metadata-/*/","count":360,"mean":22,"aggregate":7920},{"route-id":"puppet-v3-environment_modules-/*/","count":3,"mean":75,"aggregate":225},{"route-id":"puppet-v3-file_content-/*/","count":9,"mean":24,"aggregate":216},{"route-id":"puppet-v3-static_file_content-/*/","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-environments","count":169,"mean":0,"aggregate":0},{"route-id":"puppet-v3-tasks-:module-name-:task-name","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-facts-/*/","count":0,"mean":0,"aggregate":0},{"route-id":"other","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-status-/*/","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-tasks","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-compile","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-plans-:module-name-:plan-name","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-file_bucket_file-/*/","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v4-catalog","count":0,"mean":0,"aggregate":0},{"route-id":"total","count":1138,"mean":0,"aggregate":0},{"route-id":"puppet-v3-environment_classes-/*/","count":339,"mean":0,"aggregate":0},{"route-id":"puppet-v3-environment-/*/","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-plans","count":0,"mean":0,"aggregate":0},{"route-id":"puppet-v3-environment_transports-/*/","count":0,"mean":0,"aggregate":0}],"http-client-metrics":[{"count":524,"mean":94,"aggregate":49256,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.puppetdb.query.full-response","metric-id":["puppetdb","query"]},{"count":43,"mean":226,"aggregate":9718,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.puppetdb.command.replace_catalog.full-response","metric-id":["puppetdb","command","replace_catalog"]},{"count":86,"mean":105,"aggregate":9030,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.classifier.nodes.full-response","metric-id":["classifier","nodes"]},{"count":43,"mean":174,"aggregate":7482,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.puppetdb.command.replace_facts.full-response","metric-id":["puppetdb","command","replace_facts"]},{"count":43,"mean":140,"aggregate":6020,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.puppetdb.command.store_report.full-response","metric-id":["puppetdb","command","store_report"]},{"count":3,"mean":100,"aggregate":300,"metric-name":"puppetlabs.pe-201940-master.http-client.experimental.with-metric-id.puppetdb.facts.find.full-response","metric-id":["puppetdb","facts","find"]}]}},"active_alerts":[]},"status-service":{"service_version":"1.1.0","service_status_version":1,"detail_level":"debug","state":"running","status":{"experimental":{"jvm-metrics":{"cpu-usage":0.19996001,"up-time-ms":99037853,"gc-cpu-usage":0.0,"threading":{"thread-count":68,"peak-thread-count":70},"heap-memory":{"committed":533725184,"init":268435456,"max":536870912,"used":401267616},"gc-stats":{"G1 Young Generation":{"count":2955,"total-time-ms":57556,"last-gc-info":{"duration-ms":19}},"G1 Old Generation":{"count":0,"total-time-ms":0}},"start-time-ms":1583861824686,"file-descriptors":{"used":55,"max":12000},"non-heap-memory":{"committed":415461376,"init":2555904,"max":-1,"used":334678312}}}},"active_alerts":[]},"error":[],"error_count":0,"api-query-start":"2020-03-13T15:15:55Z","api-query-duration":0.087908123}}}}
[root@pe-201940-master ~]# cat /var/log/cron | grep -i err
[root@pe-201940-master ~]# puppet agent -t
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for pe-201940-master.puppetdebug.vlan
Info: Applying configuration version '1584112741'
Notice: Applied catalog in 17.73 seconds

Copy link
Member

@Sharpie Sharpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

A couple notes about ActiveMQ metrics (I don't think we need to change anything there?) and a stray line of 2016.4/2016.5 logic that can be dropped.

Tested 2018.1.12 and 2018.1.13 w/ HA enabled and the only differences in output were the three metrics that are purposefully excluded:

  • puppetlabs.puppetdb.ha:name=failed-request-counter
  • puppetlabs.puppetdb.mq:name=global.generate-retry-message-time
  • puppetlabs.puppetdb.mq:name=global.retry-persistence-time

These are problematic because the counters don't exist until the event happens for the first time. However, they seem a bit off the beaten path so we can re-enable them later if the need arises.

manifests/service/activemq.pp Show resolved Hide resolved
manifests/service/puppetdb.pp Outdated Show resolved Hide resolved
manifests/service/puppetdb.pp Outdated Show resolved Hide resolved
Restore additional activemq metrics.
Delete PE 2016 conditionals.
Restore counters that don't exist until a failure occurs.
Sharpie
Sharpie previously approved these changes Mar 16, 2020
Copy link
Member

@Sharpie Sharpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

Just have to get the Travis tests green and this is ready to merge.

/^2017.(1|2)/ => {'catalogs' => 9, 'facts' => 5, 'reports' => 8},
default => {'catalogs' => 9, 'facts' => 5, 'reports' => 8},
}
$version = {'catalogs' => 9, 'facts' => 5, 'reports' => 8},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's a trailing comma at the end of this line that is failing the Travis tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit for this pushed ...

Copy link
Member

@Sharpie Sharpie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

@Sharpie Sharpie merged commit 576b549 into puppetlabs:master Mar 17, 2020
m0dular pushed a commit to m0dular/puppetlabs-puppet_metrics_collector that referenced this pull request Jul 7, 2020
…-support-script

Add double quotes to copy_drop_match
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants