Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mqtt2prometheus push gateway to third party tools #1649

Merged
merged 3 commits into from
Jun 11, 2020

Conversation

hikhvar
Copy link
Contributor

@hikhvar hikhvar commented May 31, 2020

No description provided.

Signed-off-by: Christoph Petrausch <chrobbert@gmail.com>
@hikhvar hikhvar force-pushed the add-mqtt2prometheus branch from 01276f5 to c276237 Compare May 31, 2020 19:56
Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like a reasonable idea. But it would be nice if the MQTT messages were more "Prometheus Native", as in they included the counter/gauge information so the exporter could Set() or Inc() appropriately without having to configure it.

@brian-brazil
Copy link
Contributor

Is this working off an existing standard way that someone would already would have things setup, or would they have to set all of the pushing to MQTT with such and such a topic name with specifically formatted data just to use this? What happens when a sensor stops publishing or no longer exists?

I'm concerned that this might be inventing a new way of doing things rather than building on what a user is likely to already have, in which case as per above using the Prometheus format would be better.

@roidelapluie
Copy link
Member

Also how does it compare to alternatives? I know there are multiple of the.

@hikhvar
Copy link
Contributor Author

hikhvar commented Jun 2, 2020

Hello,
thank you for your feedback. It was a design goal of mine to build the MQTT side of the gateway very generic. I had in mind a use case where the user of the MQTT gateway can not change exiting IoT devices. The feedback I got via PRs and issues reflect just that. For example: hikhvar/mqtt2prometheus#9. I assume they send (at the moment JSON) payload to certain topics. I did not implement any specific sensoring protocoll based on MQTT. For that use case, there should be a dedicated exporter. @roidelapluie that distinct my gateways from alternatives like

@hikhvar
Copy link
Contributor Author

hikhvar commented Jun 2, 2020

What happens when a sensor stops publishing or no longer exists?

At the moment the exporter caches the ingested metrics for a configurable time. If there is no update during the cache time, the exporter discard the value and do not present it anymore. I work on a solution using timestamps.

@brian-brazil
Copy link
Contributor

Thanks for the explanation, this sounds like this is trying to work with what exists rather than inventing a whole new thing. The inuits one seems to require devices to be coded for it. The tg44 is more generic, however also appears to include an entire bespoke client library which is a bit odd, and has a smaller community. Accordingly I'd be tending towards this one.

The readme indicates that this is an analog of the pushgateway. This is in fact explicitly not the sort of use case that pushgateway is for. Can you correct this please? The text in this PR should also only be the exporter name.

This should probably go up in messaging systems.

In terms of general comments to help you improve your exporter it'd be good if your temperature examples included _celsius as a suffix on the metric name. Your default port number is outside of https://github.com/prometheus/prometheus/wiki/Default-port-allocations, so presuming there's not some technical consideration I'm missing I'd suggest changing it. const_labels is a bit odd, for the given example I'd expect that to be part of the metric name. I'd also suggest looking at the other two exporters, as they do have ideas that'd be good to incorporate into one really good exporter (inuit's metric for last push timestamp, tg44's topic name label extraction).

hikhvar/mqtt2prometheus#9

As an aside, if that ends up not being sophisticated enough you should take a look at how the JMX and SNMP exporters approach this. Given it's JSON hopefully you can avoid that level of complexity though.

@beorn7 beorn7 added the exporters and integrations Requests for new entries in the list of exporters and integrations label Jun 2, 2020
hikhvar pushed a commit to hikhvar/mqtt2prometheus that referenced this pull request Jun 9, 2020
hikhvar pushed a commit to hikhvar/mqtt2prometheus that referenced this pull request Jun 9, 2020
@hikhvar hikhvar force-pushed the add-mqtt2prometheus branch from ae56a11 to f379b1f Compare June 9, 2020 14:22
Signed-off-by: Christoph Petrausch <chrobbert@gmail.com>
@hikhvar hikhvar force-pushed the add-mqtt2prometheus branch from f379b1f to f2ccc06 Compare June 9, 2020 14:36
Signed-off-by: Christoph Petrausch <chrobbert@gmail.com>
@brian-brazil brian-brazil merged commit 4aaf643 into prometheus:master Jun 11, 2020
@brian-brazil
Copy link
Contributor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exporters and integrations Requests for new entries in the list of exporters and integrations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants