Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Service Bus Queues should be able to scrape all queues in namespace, not just one #529

Closed
3 tasks
tomkerkhove opened this issue Apr 29, 2019 · 4 comments · Fixed by #1205
Closed
3 tasks
Labels
agents:scraper All issues related to the scraping agent feature All issues that are new features integration:prometheus All issues related to Prometheus integration metric-labelling All issues related to metric labelling metric-source:azure-monitor All issues that relate to scraping metrics that depends on Azure Monitor as a data source scraper-source All issues related to new scraper sources from Azure Monitor scraping All issues related to scraping specs-required All issues where the specifications are still being defined and implementation should be halted

Comments

@tomkerkhove
Copy link
Owner

Service Bus Queues should be able to scrape all queues in namespace, not just one.

It should annotate all queues that it scraped with a entity_name label in Prometheus endpoint.

Relates to #513 and was suggested in this comment by @marcinbudny

Checklist

  • Make queueName optional in the configuration
  • Scrape all queues if no queue name was configured
  • Always apply entity_name label, even if queueName was configured for consistency reasons
@tomkerkhove tomkerkhove added integration:prometheus All issues related to Prometheus integration feature All issues that are new features scraper-source All issues related to new scraper sources from Azure Monitor metric-source:azure-monitor All issues that relate to scraping metrics that depends on Azure Monitor as a data source scraping All issues related to scraping labels Apr 29, 2019
@tomkerkhove tomkerkhove added this to the v1.0.0 milestone Apr 29, 2019
@tomkerkhove
Copy link
Owner Author

One thing we'll need to evaluate is if we can get it out of the box or if this means increasing required permissions for our Azure AD app.

If that is the case we'll need to see if it's worth it.

@tomkerkhove tomkerkhove added the metric-labelling All issues related to metric labelling label Apr 29, 2019
@tomkerkhove tomkerkhove modified the milestones: v1.0.0, v1.1.0 May 21, 2019
@tomkerkhove tomkerkhove modified the milestones: v1.1.0, v2.0.0, v1.2.0 Oct 21, 2019
@tomkerkhove tomkerkhove added the specs-required All issues where the specifications are still being defined and implementation should be halted label Oct 30, 2019
@tomkerkhove tomkerkhove modified the milestones: v1.2.0, v1.3.0, v1.4.0 Jan 6, 2020
@tomkerkhove tomkerkhove modified the milestones: v1.4.0, v1.3.0 Jan 16, 2020
@tomkerkhove tomkerkhove removed this from the v1.3.0 milestone Jan 18, 2020
@tomkerkhove tomkerkhove added the agents:scraper All issues related to the scraping agent label Apr 21, 2020
@ChristianEder
Copy link
Contributor

ChristianEder commented May 19, 2020

I am currently struggling with a similar issue, but I'm not sure if it is totally the same: I cannot get service bus queue metrics for more than 1 queue. I tried 2 things:

  1. Using a ServiceBusQueue scraper with multiple resources:
- name: azure_service_bus_active_messages
  description: "Count of active messages per queue / topic"
  resourceType: ServiceBusQueue
  azureMetricConfiguration:
    metricName: ActiveMessages
    aggregation:
      type: Average
  resources:
  - namespace: my-namespace
    queueName: queue1
  - namespace:  my-namespace
    queueName: queue2
  - namespace:  my-namespace
    queueName: queue3
  1. Using a Generic scraper with multiple resources, using filters:
- name: azure_service_bus_deadlettered_messages
  description: "Count of dead-lettered messages per queue / topic"
  resourceType: Generic
  azureMetricConfiguration:
    metricName: DeadletteredMessages
    aggregation:
      type: Average
  resources:
  - resourceUri: Microsoft.ServiceBus/namespaces/my-namespace
    filter: EntityName eq 'queue1'
  - resourceUri: Microsoft.ServiceBus/namespaces/my-namespace
    filter: EntityName eq 'queue2'
  - resourceUri: Microsoft.ServiceBus/namespaces/my-namespace
    filter: EntityName eq 'queue3'

The 1) approach leads to only the metrics for queue3 being reported, the 2) approach to only the unfiltered, total sum of DeadletteredMessages being reported. Is this a known issue or am I doing something wrong here?

@tomkerkhove
Copy link
Owner Author

Hm that feels like a bug, would you mind creating a new issue for this please? Sorry for this 😐

@ChristianEder
Copy link
Contributor

Done: #1050

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agents:scraper All issues related to the scraping agent feature All issues that are new features integration:prometheus All issues related to Prometheus integration metric-labelling All issues related to metric labelling metric-source:azure-monitor All issues that relate to scraping metrics that depends on Azure Monitor as a data source scraper-source All issues related to new scraper sources from Azure Monitor scraping All issues related to scraping specs-required All issues where the specifications are still being defined and implementation should be halted
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants