Provide better usage of AzureMonitorClient

[tomkerkhove]

Provide better usage of AzureMonitorClient by only creating it once.

Relates to #798

Before

After

[promitor-bot]

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr844

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr844 \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr844

You can find a CI version of our Helm chart on hub.helm.sh

[promitor-bot]

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr844

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr844 \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr844

You can find a CI version of our Helm chart on hub.helm.sh

[tomkerkhove]

@maartenba Did a bit of refactoring but didn't seem to really help.

However, it looks like there's a shitload of HttpClients being created, I presume dotMemory is showing unique instance rather than (re)used ones or not?

[maartenba]

It shows unique ones, yes. Can you send me a longer run snapshot? Will have a look at it.

[tomkerkhove]

Performance analysis before & after can be downloaded from DropBox

[maartenba]

Checking "after". Looks like lots of leftover HttpClient instances are held in memory by AzureMonitorClient:

That in turn seems to not be disposed by ServiceProvider (check the right-hand side)

The AzMon libraries seem to hold a lot of these references as well, but can't really pinpoint where those come from.

Would you mind running performance profiling in Timeline mode as well? That should give some more info on # calls etc.

[tomkerkhove]

You can download it here @maartenba

[promitor-bot]

Docker image for this PR was built and is available on Docker Hub.

You can pull it locally via the CLI:

docker pull tomkerkhove/promitor-agent-scraper-ci:pr844

Want to verify the new version? Run it locally:

docker run -d -p 8999:80 --name promitor-agent-scraper-pr844 \
                         --env PROMITOR_AUTH_APPID='<azure-ad-app-id>' \
                         --env-file C:/Promitor/az-mon-auth.creds \
                         --volume C:/Promitor/metrics-declaration.yaml:/config/metrics-declaration.yaml  \
                         --volume C:/Promitor/runtime-config.yaml:/config/runtime.yaml \
                         tomkerkhove/promitor-agent-scraper-ci:pr844

You can find a CI version of our Helm chart on hub.helm.sh

[maartenba]

OK, ran through all traces, and found HttpClients are created here.

Example instance:

All instances:

Looks like there is a way to pass in an existing HttpClient, not sure whether that is exposed in the Azure Management API's? If yes, try passing a pre-created instance into the tree.

[tomkerkhove]

@maartenba Think it's safe to merge this or think it is not?

The issue lies in the SDK so can't do much about it, but this PR still improves it a bit.

Thougths?

[maartenba]

Think it's safe to do so, however if you figure out a way to pass in an existing HttpClient (which seems supported at the lower levels) you would be golden.