InfluxDB 0.9 support

[torkelo]

InfluxDB 0.9 is approaching release.

Since its API is almost completly changed it will require big updates to both the query editor and the InfluxDB data source.

In order not to force Grafana users to upgrade I think the best approach is to split the InfluxDB support into two different data source implementations (one called influxdb_08, and the one just called influxdb will be 0.9.x).

Of course the influxdb_08 implementation will be removed after a while.

[tmonk42]

If all datasources were plugins, users could include the one(s) they need. Related to issue #1276.
I do think long term versioned datasource plugins will be valuable :)

[torkelo]

@tmonk42 yes, but tricky to do right, and to support & test the different versions.

[NikolaBorisov]

Cool, how far is this. Having grafana is essential for using influxdb 0.9. Is the latest develop branch usable with influxdb 0.9? Basically what is the most stable grafana that works the best with influx 0.9?

[torkelo]

basic raw queries are working in develop branch, but needs a lot more work

[NikolaBorisov]

Probably not the best place to ask, but how do I build the develop branch, the documentation does not mention the go server component.

[torkelo]

@NikolaBorisov http://docs.grafana.org/v2.0/project/building_from_source/

[liyichao]

Influxdb has a fork for 0.9: https://github.com/influxdb/grafana/tree/influx-0.9rc4

[torkelo]

@liyichao yes, I know, I have looked at it but that fork it not functional or usable. That fork is more of a quick hack & proof of concept. But it has some good ideas that I have also adopted :)

I will work on this more when InfluxDB 0.9 matures.

[markuslang79]

Some days ago you've wrote that queries using tags always returned an empty result. I just quickly tried Grafana 2.0.0-beta1 with InfluxDB 0.9.0-rc17 with some collectd data written into InfluxDB. Grafana generated the following query after clicking on 'select metric', selecting "host" from the 'select tag' drop-down and after choosing a host from the 'select tag value' drop down:

SELECT mean(value) FROM "load_shortterm" WHERE $timeFilter AND host='localhost' GROUP BY time($interval) ORDER BY asc

This query returned an empty result - but only because collectd doesn't use 'value' for storing the 'load_shortterm' values. I only had to change "mean(value)" to "mean(load_shortterm)" to get this query up and running.

So queries using tags like "AND host='localhost'" in this case do seem to work fine with the current versions.

[torkelo]

@pret79 yea, I solved that issue yesterday. Seems like the tag filter has be after time filter for some reason.

[greglook]

Since it seems like you want all the 0.9.0 stuff tracked here, I'll add a note about security. Right now, the database user and password set for InfluxDB 0.9 data sources cause the credentials to be sent as part of the query string (u and p parameters, specifically). On the InfluxDB side, these get written to the log, leaking the data.

Instead, InfluxDB also supports basic auth to pass the credentials, which lets it log the authenticated user without the password showing up. Grafana's backend already supports basic auth, but currently requires a username and password setting, so there's no way to stop it from sending them.

[victorhooi]

Is there another branch of Grafana somewhere else that has more up-to-date support for InfluxDB 0.9? Or are these just going straight into Master? (My understanding is that they are, just wanted to confirm).

NB: There's InfluxDB's fork of Grafana here - https://github.com/influxdb/grafana/tree/influx-0.9rc4. However, that was last updated back in January 2015.

What is the current state of InfluxDB 0.9 support? (i.e. what works, and what doesn't work?)

Is there any ETA on when you think InfluxDB 0.9 support would be in a usable state? (I believe the API is meant to be fairly stable already, the actual 0.9 is meant to ship this month).

[greglook]

@victorhooi for what it's worth, I've been running Grafana 2.0.1 (now) with InfluxDB 0.9.0-rc25 and most things work okay. Specifically, you can build graphs that properly show metrics stored in influx.

Some things I've noticed that are currently rough/not working:

• InfluxDB can't be used as a source for annotations.
• When the call to the InfluxDB API fails, the error message is not visible.
• All calls to InfluxDB pass credentials in the query string (see above).
• The query builder mostly works but is very primitive, and I often just edit the query by hand.
• The $interval variable seems to default to quite small values, e.g. 1 minute for a 6 hour period.
• It's not obvious that to get per-host series (assuming your points in InfluxDB had a host tag) you need to add host to the GROUP BY clause.

[Garinder]

Annotations work for grafana 2.1-pre with influxdb 0.9 just tested it , sweet thank you Torkel!!!

[cemo]

@torkelo I have checked builds page and it seems that latest build is based on 4 days ago. Is there any particular reason why Grafana builds are not built daily?

[torkelo]

Progress today, now editor supports regex values (operator switch is handled automatically when the value changs to regex pattern, also value will no longer be put in quotes)

[timgriffiths]

Could we get the option "previous" and "none" added to the fill option in the query builder

https://influxdb.com/docs/v0.9/query_language/querying_data.html

Looking great @torkelo

[Jhors2]

This is looking great! I think the only thing that is missing for me now is the ability to add a "where" clause that also allows me to specify if the metric is above or below a certain value. Something like:

select value from "cpu_utilization" where value > 100000

[damm]

@Jhors2 do you mean

You can add additional clauses with the +

[deric]

Is there right now any workaround how to perform nested queries? Something like mentioned above:

TRANSFORM Query(A) -> scale(8) -> timeshift(1d) -> asPercentOf(Query(B))

2.1-pre1 query editor looks great! There's just a small bug, when changing AND to OR, the whole query is broken.

[torkelo]

@deric thanks for repoting the AND -> OR bug.

@deric no nested queries in InfluxDB 0.9 at the moment. But I have a few ideas for how it can be added in the future :)

Will close this issue as the InfluxDB 0.9 support is starting to come together, further improvement and fixes is better handled in new issues as this issue is getting long enough :)

[liyichao]

Just wonder when will grafana 2.1 be released. 😸

[torkelo]

Checkout this issue for enhancements to the field and function selector (can now suggest fields, and have multiple fields)
#2311 (comment)

[jshafton]

FYI, current grafana/master isn't working with influxdb9.1/master on account of Grafana sending an order by clause in the query, which isn't currently supported by InfluxDB. That clause needs to be removed completely from the query for results to be returned (and InfluxDB orders by ASC by default anyway).

[torkelo]

@jshafton thanks for the info. Strange, never had an issue with the order by clause, either in InfluxDB 0.9 or 0.9.1

[jshafton]

Yeah, it's working in the stable build of 0.9.1 -- InfluxDB just ignores the order by. But now, even through the admin console, if you specify any order by clause -- event the default of ORDER BY ASC you get an error. I'm guessing they did this to make sure callers know that the feature isn't implemented as of yet.

[torkelo]

you mean in nightly builds of InfluxDB master branch?

[jshafton]

Sorry, I tested specifically the nightly .deb build from here, which indicates it's a build from the master branch.

[jshafton]

I've tested locally that it now works without the ORDER BY (I'm not a go developer; I just hacked the js asset in my .deb-based install).

It looks like they moved the InfluxDB issue from the 0.9.2 milestone to the 0.9.3 milestone. Do you want me to submit a PR to remove this for now (or do it yourself?) or wait for 0.9.3 (or later) in InfluxDB to support it?

[torkelo]

PR please :)

[jshafton]

Done.

[jshafton]

@torkelo - looks like they fixed the ORDER BY regression in 0.9.2, so you could revert my commit if you want to. The only thing to note there is that it only supports ORDER BY ASC or ORDER BY time ASC, so it may be simpler to leave the ORDER BY out of the Grafana query builder until InfluxDB fully supports it.

[tuukkamustonen]

@torkelo Should referencing tags in alias by $tag_tagname or [[tagname]] work at the moment or am I missing something? I'm running Grafana 2.1.0-pre1, connected to InfluxDB 0.9.2 and tag placeholders don't evaluate to anything. $m and $measurement do work, but can't refer tags (e.g. $tag_host or [[project]]). The tag dropdowns get populated just fine and queries work, but using tags in aliasing does not seem to function.

[jgerschk]

@tuukkamustonen reference #2416 -- this should help you out.

[tuukkamustonen]

Thanks @jgerschk. Indeed, tag needs to be in "group by" clause (as shown in #1525 (comment)) for it to work as an alias field. And the pattern for using brackets is also [[tag_tagname]] not [[tagname]] - not sure where I think I took that originally...