Provide support for scraping metrics for Azure IoT Hubs #1011

ChristianEder · 2020-05-06T05:33:04Z

Fixes #372

Thanks to the very good documentation, the process to implement this was rather straightforward. Nevertheless, I am not sure regarding these questions:

How can I verify this will actually work?
What should I put into the "Availability Badge" link inside the iot-hub.md file?
Is there some kind of changelog that needs to be updated?

CLAassistant · 2020-05-06T05:33:12Z

All committers have signed the CLA.

promitor-bot · 2020-05-06T05:35:54Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

promitor-bot · 2020-05-06T05:41:48Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

tomkerkhove · 2020-05-06T05:47:51Z

Thanks to the very good documentation, the process to implement this was rather straightforward. Nevertheless, I am not sure regarding these questions:

Great to hear, thank you!

How can I verify this will actually work?

You can run Promitor locally by configuring it to your subscription and verifying the metrics show up. If that's too much than I can pull in your PR and run it myself

What should I put into the "Availability Badge" link inside the iot-hub.md file?

I've added it to the v1.6 release plan; feel free to add that

Is there some kind of changelog that needs to be updated?

Not a must but you may update our upcoming features list here, if not then I'm happy to take care of this!

That's a good point, I should include that in the guide! (#1012)

I'll review it today, thanks!

promitor-bot · 2020-05-06T05:49:10Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

promitor-bot · 2020-05-06T05:51:37Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-06T05:58:56Z

Thanks - I will try to run it on my own, if I fail I will come back to you.

Besides that:

The CLA signing does not seem to get recognised. I already verified that the email adress shown in the CLA form (which is readonly by now) is the same thats shown in my github account
Some CI builds are red - is this due to me missing access to some github action secrets? In other words: are they ok to fail?

promitor-bot · 2020-05-06T05:59:53Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-06T08:24:33Z

I ran it locally and it worked - tested retrieving the number of devices in one of my IoT Hubs, and also saw the value changing when I registered a new device.

tomkerkhove

LGTM, thank you very much!

I've added a few nits but the main one is to change Iot to IoT, would you be up for that? Can you also please sign the CLA?

docs/configuration/v1.x/metrics/index.md

docs/configuration/v1.x/metrics/iot-hub.md

adding-a-new-scraper.md

src/Promitor.Agents.Scraper/Validation/MetricDefinitions/ResourceTypes/IotHubMetricValidator.cs

promitor-bot · 2020-05-06T15:01:09Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-06T15:06:04Z

I resolved your comments. But I cannot for the life of me figure out how to make GitHub acknowledge that I already signed the CLA.

It just shows this message:

But I logged in using this git hub account (it is my only) and also verified that the email address shown in the CLA form is in my account. It is my primary email address

The CLA form also show everything as ok and doesnt let me change anything:

promitor-bot · 2020-05-06T15:09:54Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-06T15:20:26Z

oh dammit. Most of my commits where assigned to another email of mine on which I didnt even have a github account. Created one, signed the CLA with that, tadaaa -> done

promitor-bot · 2020-05-06T15:23:30Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

tomkerkhove · 2020-05-06T16:29:29Z

Thanks!

tomkerkhove

LGTM, thanks!

adding-a-new-scraper.md

promitor-bot · 2020-05-06T16:34:30Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

promitor-bot · 2020-05-06T16:35:26Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

....Tests.Unit/Validation/Metrics/ResourceTypes/IoTHubMetricsDeclarationValidationStepsTests.cs

promitor-bot · 2020-05-06T16:59:41Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

promitor-bot · 2020-05-06T16:59:51Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-06T17:55:32Z

What about the failing checks - are these related to this PR?

ChristianEder · 2020-05-06T17:56:29Z

I see that now at least one test went red - will fix tomorrow.

tomkerkhove · 2020-05-06T18:16:12Z

No worries, should've noticed in the review - Sorry!

promitor-bot · 2020-05-07T04:57:19Z

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr1011-linux \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr1011-linux

You can find a CI version of our Helm chart on hub.helm.sh

ChristianEder · 2020-05-07T05:46:32Z

So the tests are green again, but some checks are still failing, but I cannot figure out what the failures mean.

Unable to find image 'tomkerkhove/promitor-agent-discovery-ci:pr1011-linux' locally docker: Error response from daemon: manifest for tomkerkhove/promitor-agent-discovery-ci:pr1011-linux not found: manifest unknown: manifest unknown. See 'docker run --help'. Error: No such container: promitor-discovery-agent

tomkerkhove · 2020-05-07T06:14:29Z

/azp run Promitor CI - Resource Discovery Agent

azure-pipelines · 2020-05-07T06:14:39Z

Azure Pipelines successfully started running 1 pipeline(s).

tomkerkhove · 2020-05-07T06:15:01Z

So the tests are green again, but some checks are still failing, but I cannot figure out what the failures mean.

Unable to find image 'tomkerkhove/promitor-agent-discovery-ci:pr1011-linux' locally docker: Error response from daemon: manifest for tomkerkhove/promitor-agent-discovery-ci:pr1011-linux not found: manifest unknown: manifest unknown. See 'docker run --help'. Error: No such container: promitor-discovery-agent

Sorry for that, there was something misconfigured on the build for PRs; my bad!

Re-running now and everything should be good - Thanks so much!

ChristianEder requested a review from tomkerkhove as a code owner May 6, 2020 05:33

ChristianEder-z added 6 commits May 6, 2020 07:46

Add configuration classes

95f1a06

Added Validation classes and tests, updated contribution docs

139995d

add IoT Hub Scraper

7167b29

add docs

742b19c

casing

9c76a34

fix docs linting

0f357fb

set availability badge to v1.6

a55f584

ChristianEder-z added 2 commits May 6, 2020 07:54

add entry to changelog

a6bf8f1

linter again

5546e3c

tomkerkhove approved these changes May 6, 2020

View reviewed changes

Iot --> IoT

b7cb219

ChristianEder-z added 3 commits May 6, 2020 17:01

review findings

c0a03fa

rename 1

06ead20

rename back

ce391df

ChristianEder added 2 commits May 6, 2020 17:18

newline

e7be918

reverted

d6a90d9

ChristianEder closed this May 6, 2020

ChristianEder reopened this May 6, 2020

tomkerkhove approved these changes May 6, 2020

View reviewed changes

adding-a-new-scraper.md Outdated Show resolved Hide resolved

Cleanup

d97a97c

tomkerkhove reviewed May 6, 2020

View reviewed changes

....Tests.Unit/Validation/Metrics/ResourceTypes/IoTHubMetricsDeclarationValidationStepsTests.cs Outdated Show resolved Hide resolved

Fix bad parameter name

54b23a8

fix test after renaming to IoT

12b38f9

tomkerkhove merged commit 7c705c5 into tomkerkhove:master May 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide support for scraping metrics for Azure IoT Hubs #1011

Provide support for scraping metrics for Azure IoT Hubs #1011

ChristianEder commented May 6, 2020

CLAassistant commented May 6, 2020 •

edited

Loading

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

tomkerkhove commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

tomkerkhove left a comment

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020 •

edited

Loading

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020 •

edited

Loading

promitor-bot commented May 6, 2020

tomkerkhove commented May 6, 2020

tomkerkhove left a comment

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

ChristianEder commented May 6, 2020

tomkerkhove commented May 6, 2020

promitor-bot commented May 7, 2020

ChristianEder commented May 7, 2020

tomkerkhove commented May 7, 2020

azure-pipelines bot commented May 7, 2020

tomkerkhove commented May 7, 2020

Provide support for scraping metrics for Azure IoT Hubs #1011

Provide support for scraping metrics for Azure IoT Hubs #1011

Conversation

ChristianEder commented May 6, 2020

CLAassistant commented May 6, 2020 • edited Loading

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

tomkerkhove commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

tomkerkhove left a comment

Choose a reason for hiding this comment

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020 • edited Loading

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020 • edited Loading

promitor-bot commented May 6, 2020

tomkerkhove commented May 6, 2020

tomkerkhove left a comment

Choose a reason for hiding this comment

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

promitor-bot commented May 6, 2020

ChristianEder commented May 6, 2020

ChristianEder commented May 6, 2020

tomkerkhove commented May 6, 2020

promitor-bot commented May 7, 2020

ChristianEder commented May 7, 2020

tomkerkhove commented May 7, 2020

azure-pipelines bot commented May 7, 2020

tomkerkhove commented May 7, 2020

CLAassistant commented May 6, 2020 •

edited

Loading

ChristianEder commented May 6, 2020 •

edited

Loading

ChristianEder commented May 6, 2020 •

edited

Loading