panic: expected either int64 or float64, got...

[ymettier]

Hello,

I have a panic...

2015/10/27 09:27:01 InfluxDB starting, version 0.9.5-nightly-2fe5e6b, branch master, commit 2fe5e6b4f7192445deec0898ec702d1f8cbba5da, built '2015-10
-27T04:00:42+0000'
[…]
panic: expected either int64 or float64, got {1445937292000000000 3.725094769e+09 map[:3.725094769e+09] map[direction:rx fqdn:myhost iface:eth0 unit:bytes category1:if_octets collector:collectd]}

goroutine 36037 [running]:
github.com/influxdb/influxdb/tsdb.int64toFloat64(0xb9e6a0, 0xc87fa65d40, 0x0)
        /tmp/tmp.CBAd6vC3wi/src/github.com/influxdb/influxdb/tsdb/raw.go:764 +0x1f5
github.com/influxdb/influxdb/tsdb.ProcessAggregateDerivative(0xc854e2e3c0, 0xd, 0xd, 0x0, 0x1bf08eb000, 0x0, 0x0, 0x0)
        /tmp/tmp.CBAd6vC3wi/src/github.com/influxdb/influxdb/tsdb/raw.go:708 +0x741
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).processDerivative(0xc83b310740, 0xc854e2e3c0, 0xd, 0xd, 0x0, 0x0, 0x0)
        /tmp/tmp.CBAd6vC3wi/src/github.com/influxdb/influxdb/tsdb/aggregate.go:363 +0x18f
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).execute(0xc83b310740, 0xc8431de240)
        /tmp/tmp.CBAd6vC3wi/src/github.com/influxdb/influxdb/tsdb/aggregate.go:162 +0x14a3
created by github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).Execute
        /tmp/tmp.CBAd6vC3wi/src/github.com/influxdb/influxdb/tsdb/aggregate.go:48 +0x5d

I also read some code :
https://github.com/influxdb/influxdb/blob/master/tsdb/raw.go from line 678 to line 708.

678 for i := 1; i < len(results); i++ {
679     prev := results[i-1]
680     cur := results[i]
[...]
690     // Check the value's type to ensure it's an numeric, if not, return a nil result. We only check the first value
691     // because derivatives cannot be combined with other aggregates currently.
692     validType := false
693     switch cur[1].(type) {
694     case int64:
695         validType = true
696     case float64:
697         validType = true
698     }
[...]
708     diff := int64toFloat64(cur[1]) - int64toFloat64(prev[1])

For what I understand, cur[1] and prev[1] can be anything until line 692. At line 708, we can assert that they are either int64 or float64.

It seems that you check the type of cur[1]. But not prev[1]. Not a problem because prev[1] is previous cur[1].
Except in one case : you never check the type of results[0][1] (e.g. prev[1] when i == 1 at first run of the loop)
I guess that in my case result[0][1] is neither int64 nor float64, and then int64toFloat64(prev[1]) will do strange things and panic.

I have no idea if the fix is to just add a check for prev[1] when i == 1 or if this bug shows a bigger problem and needs more reengineering.

Regards,
Yves

[jwilder]

What was the query you were running? Also, do you have some sample data points?

[beckettsean]

Thanks for the support ticket, @ymettier.

@jwilder assigned you ZD ticket 1648, which is the companion piece to this issue.

[conet]

I encountered the same issue running influxdb-0.9.5-0.1.rc3.x86_64.rpm. Here is the query:

[query] 2015/11/20 10:58:00 SELECT derivative(last(value), 1m) AS "value" FROM "collectd"."default".aggregation_value WHERE type = 'cpu' AND type_instance = 'idle' AND time > now() - 6h GROUP BY time(1m) fill(0)
panic: expected either int64 or float64, got {1448035072174905000 269669 map[:269669] map[host:hostname instance:cpu-average type:cpu type_instance:idle]}

goroutine 1966 [running]:
github.com/influxdb/influxdb/tsdb.int64toFloat64(0xb173e0, 0xc2084c7080, 0x0)
    /tmp/tmp.lIB11rDEHn/src/github.com/influxdb/influxdb/tsdb/raw.go:764 +0x237
github.com/influxdb/influxdb/tsdb.ProcessAggregateDerivative(0xc20883c200, 0x169, 0x169, 0x0, 0xdf8475800, 0x0, 0x0, 0x0)
    /tmp/tmp.lIB11rDEHn/src/github.com/influxdb/influxdb/tsdb/raw.go:708 +0x72f
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).processDerivative(0xc2084d5c80, 0xc20883c200, 0x169, 0x169, 0x0, 0x0, 0x0)
    /tmp/tmp.lIB11rDEHn/src/github.com/influxdb/influxdb/tsdb/aggregate.go:363 +0x1a6
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).execute(0xc2084d5c80, 0xc208bfe960)
    /tmp/tmp.lIB11rDEHn/src/github.com/influxdb/influxdb/tsdb/aggregate.go:162 +0x187b
created by github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).Execute
    /tmp/tmp.lIB11rDEHn/src/github.com/influxdb/influxdb/tsdb/aggregate.go:48 +0x5a

[ymettier]

Hello,

Crash occurred again.
With version 0.9.5.1 (installed from rpm; on Redhat 6)

2015/12/02 12:12:06 InfluxDB starting, version 0.9.5.1, branch 0.9.5, commit 9eab56311373ee6f788ae5dfc87e2240038f0eb4, built 2015-11-25T23:01:58+0000
[...]
[query] 2015/12/02 12:45:52 SELECT derivative(last(value), 2m) FROM "metrics"."metrics_1_year"."sys.network" WHERE collector = 'collectd' AND fqdn =~ /my.fqdn/ AND iface = 'eth0' AND category1 = 'if_octets' AND time > now() - 1h GROUP BY time(5m), direction fill(0)
panic: expected either int64 or float64, got {1449060282000000000 2.87400939910969e+06 map[:2.87400939910969e+06] map[iface:eth0 unit:bytes fqdn:my.fqdn category1:if_octets collector:collectd dsname:rx]}

goroutine 120585 [running]:
github.com/influxdb/influxdb/tsdb.int64toFloat64(0xb17400, 0xc227bee8d0, 0x0)
        /root/go/src/github.com/influxdb/influxdb/tsdb/raw.go:764 +0x237
github.com/influxdb/influxdb/tsdb.ProcessAggregateDerivative(0xc30f745680, 0xd, 0xd, 0x0, 0x1bf08eb000, 0x0, 0x0, 0x0)
        /root/go/src/github.com/influxdb/influxdb/tsdb/raw.go:708 +0x72f
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).processDerivative(0xc2c2128340, 0xc30f745680, 0xd, 0xd, 0x0, 0x0, 0x0)
        /root/go/src/github.com/influxdb/influxdb/tsdb/aggregate.go:363 +0x1a6
github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).execute(0xc2c2128340, 0xc34aceb440)
        /root/go/src/github.com/influxdb/influxdb/tsdb/aggregate.go:162 +0x187b
created by github.com/influxdb/influxdb/tsdb.(*AggregateExecutor).Execute
        /root/go/src/github.com/influxdb/influxdb/tsdb/aggregate.go:48 +0x5a

Regards,
Yves