Panic with nightly build version 0.9.3-nightly-ae674af

[tsailiming]

Same as 0.9.2, thus updated to nightly build to test.

Upon startup, access the web browser and issue a query. This started happening after simple insert, drop measurement, insert and queries.

# /etc/init.d/influxdb restart
Starting the process influxdb [ OK ]
influxdb process was started [ OK ]
[root@dev tmp]# tail -f /var/log/influxdb/influxd.log 
[admin] 2015/08/15 01:08:18 Starting admin service
[admin] 2015/08/15 01:08:18 Listening on HTTP: [::]:8083
[continuous_querier] 2015/08/15 01:08:18 Starting continuous query service
[httpd] 2015/08/15 01:08:18 Starting HTTP service
[httpd] 2015/08/15 01:08:18 Authentication enabled: false
[httpd] 2015/08/15 01:08:18 Listening on HTTP: [::]:8086
[retention] 2015/08/15 01:08:18 Starting rentention policy enforcement service
2015/08/15 01:08:18 InfluxDB starting, version 0.9.3-nightly-ae674af, branch master, commit ae674aff3a59f7783bf15ae558bcf82140204dbd
2015/08/15 01:08:18 GOMAXPROCS set to 2
[run] 2015/08/15 01:08:18 Listening for signals
[metastore] 2015/08/15 01:08:20 Node at localhost:8088 [Leader]. peers=[localhost:8088]
2015/08/15 01:08:20 Sending anonymous usage statistics to m.influxdb.com
[metastore] 2015/08/15 01:08:20 Node at localhost:8088 [Follower]. peers=[localhost:8088]

[metastore] 2015/08/15 01:08:20 No leader

[metastore] 2015/08/15 01:08:22 Node at localhost:8088 [Leader]. peers=[localhost:8088]
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x5c3704]

goroutine 58 [running]:
github.com/influxdb/influxdb/tsdb.(*FieldCodec).DecodeFields(0x0, 0x7f52c69d61d7, 0x13, 0x7fffffff, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/shard.go:553 +0x104
github.com/influxdb/influxdb/tsdb.(*FieldCodec).DecodeFieldsWithNames(0x0, 0x7f52c69d61d7, 0x13, 0x7fffffff, 0x798e2e, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/shard.go:596 +0x72
github.com/influxdb/influxdb/tsdb.(*tagSetCursor).decodeRawPoint(0xc2080685c0, 0xc2080d7110, 0xc2080d7140, 0x3, 0x3, 0xee6928, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/mapper.go:694 +0x82
github.com/influxdb/influxdb/tsdb.(*tagSetCursor).Next(0xc2080685c0, 0x0, 0x13fa6334987e6945, 0xc2080d7140, 0x3, 0x3, 0xee6928, 0x0, 0x0, 0x0, ...)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/mapper.go:680 +0x2f5
github.com/influxdb/influxdb/tsdb.(*LocalMapper).nextChunkRaw(0xc2080fea20, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/mapper.go:268 +0x139
github.com/influxdb/influxdb/tsdb.(*LocalMapper).NextChunk(0xc2080fea20, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/mapper.go:251 +0x55
github.com/influxdb/influxdb/tsdb.(*StatefulMapper).NextChunk(0xc20801f7a0, 0xc2080d7560, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/executor.go:43 +0x5c
github.com/influxdb/influxdb/tsdb.(*Executor).executeRaw(0xc2080d6cf0, 0xc208152ae0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/executor.go:190 +0x124a
created by github.com/influxdb/influxdb/tsdb.(*Executor).Execute
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/executor.go:88 +0x9a

goroutine 1 [chan receive]:
main.(*Main).Run(0xc2080680c0, 0xc20800a010, 0x4, 0x4, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/main.go:91 +0x777
main.main()
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/main.go:42 +0xdc

goroutine 6 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 8 [IO wait]:
net.(*pollDesc).Wait(0xc208010450, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010450, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080103f0, 0x0, 0x7f52c6b75e30, 0xc2080cf150)
    /usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208036f80, 0x8e82a0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208036f80, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
github.com/influxdb/influxdb/tcp.(*Mux).Serve(0xc2080d66f0, 0x7f52c6b77378, 0xc208036f80, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:45 +0x57
created by github.com/influxdb/influxdb/cmd/influxd/run.func·003
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/server.go:310 +0x8f2

goroutine 9 [chan receive]:
github.com/influxdb/influxdb/tcp.(*listener).Accept(0xc208036f88, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:117 +0x80
github.com/influxdb/influxdb/meta.(*raftLayer).Accept(0xc2080d6840, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:2058 +0x64
github.com/hashicorp/raft.(*NetworkTransport).listen(0xc208010540)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/net_transport.go:346 +0x50
created by github.com/hashicorp/raft.NewNetworkTransport
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/net_transport.go:138 +0x2c4

goroutine 10 [select]:
github.com/hashicorp/raft.(*Raft).leaderLoop(0xc2080ac1c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:815 +0xc7c
github.com/hashicorp/raft.(*Raft).runLeader(0xc2080ac1c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:787 +0x74e
github.com/hashicorp/raft.(*Raft).run(0xc2080ac1c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:584 +0xc9
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.run)·fm()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:241 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 11 [select]:
github.com/hashicorp/raft.(*Raft).runFSM(0xc2080ac1c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:498 +0xf74
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runFSM)·fm()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:242 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 12 [select]:
github.com/hashicorp/raft.(*Raft).runSnapshots(0xc2080ac1c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:1602 +0x466
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runSnapshots)·fm()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/raft.go:243 +0x27
github.com/hashicorp/raft.func·011()
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:152 +0x51
created by github.com/hashicorp/raft.(*raftState).goFunc
    /tmp/tmp.ebF47h4lNX/src/github.com/hashicorp/raft/state.go:153 +0xe3

goroutine 13 [select]:
github.com/influxdb/influxdb/meta.(*localRaft).logLeaderChanges(0xc20806ef00)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/state.go:187 +0x46e
created by github.com/influxdb/influxdb/meta.(*localRaft).open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/state.go:177 +0x130c

goroutine 14 [chan receive]:
github.com/influxdb/influxdb/tcp.(*listener).Accept(0xc208036f90, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:117 +0x80
github.com/influxdb/influxdb/meta.(*Store).serveExecListener(0xc208054140)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:619 +0x8f
created by github.com/influxdb/influxdb/meta.(*Store).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:239 +0x391

goroutine 15 [chan receive]:
github.com/influxdb/influxdb/tcp.(*listener).Accept(0xc208036f98, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:117 +0x80
github.com/influxdb/influxdb/meta.(*Store).serveRPCListener(0xc208054140)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:723 +0xc9
created by github.com/influxdb/influxdb/meta.(*Store).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:242 +0x3db

goroutine 16 [IO wait]:
net.(*pollDesc).Wait(0xc2080ca450, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080ca450, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080ca3f0, 0xc2080cf110, 0x8, 0x8, 0x0, 0x7f52c6b75e30, 0xc2080cf118)
    /usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20811a150, 0xc2080cf110, 0x8, 0x8, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7f52c6b79050, 0xc20811a150, 0xc2080cf110, 0x8, 0x8, 0x8, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7f52c6b79050, 0xc20811a150, 0xc2080cf110, 0x8, 0x8, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:316 +0x6d
encoding/binary.Read(0x7f52c6b79050, 0xc20811a150, 0x7f52c6b7a348, 0xee6928, 0x8de940, 0xc2080cf108, 0x0, 0x0)
    /usr/local/go/src/encoding/binary/binary.go:148 +0x12d
github.com/influxdb/influxdb/meta.(*Store).remoteExec(0xc208054140, 0xc2080c7400, 0x17, 0x20, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:1544 +0x828
github.com/influxdb/influxdb/meta.(*Store).exec(0xc208054140, 0xc200000013, 0xed61e0, 0xa70840, 0xc208147b60, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:1500 +0x40e
github.com/influxdb/influxdb/meta.(*Store).UpdateNode(0xc208054140, 0x1, 0xc2080d4fb0, 0xe, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:816 +0x157
github.com/influxdb/influxdb/meta.func·006(0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:285 +0x3ea
github.com/influxdb/influxdb/meta.(*Store).syncNodeInfo(0xc208054140, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:290 +0x88
created by github.com/influxdb/influxdb/meta.(*Store).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/meta/store.go:253 +0x56f

goroutine 18 [select]:
github.com/influxdb/influxdb/cmd/influxd/run.(*Server).monitorErrorChan(0xc20807c0e0, 0xc20806e6c0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/server.go:439 +0x14a
created by github.com/influxdb/influxdb/cmd/influxd/run.func·003
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/server.go:316 +0xa63

goroutine 35 [select]:
github.com/influxdb/influxdb/tsdb/engine/b1.(*Engine).autoflusher(0xc20806cf30, 0xc2081142a0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/engine/b1/b1.go:468 +0x391
created by github.com/influxdb/influxdb/tsdb/engine/b1.func·002
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/engine/b1/b1.go:133 +0x4ac

goroutine 36 [select]:
github.com/influxdb/influxdb/tsdb/engine/b1.(*Engine).autoflusher(0xc20810c510, 0xc2081144e0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/engine/b1/b1.go:468 +0x391
created by github.com/influxdb/influxdb/tsdb/engine/b1.func·002
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/engine/b1/b1.go:133 +0x4ac

goroutine 37 [select]:
github.com/influxdb/influxdb/services/hh.(*Service).retryWrites(0xc20805b4a0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/hh/service.go:103 +0x2e7
created by github.com/influxdb/influxdb/services/hh.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/hh/service.go:67 +0x20e

goroutine 38 [select]:
github.com/influxdb/influxdb/services/hh.(*Service).expireWrites(0xc20805b4a0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/hh/service.go:121 +0x2a4
created by github.com/influxdb/influxdb/services/hh.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/hh/service.go:68 +0x228

goroutine 39 [chan receive]:
github.com/influxdb/influxdb/tcp.(*listener).Accept(0xc208036fa0, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:117 +0x80
github.com/influxdb/influxdb/cluster.(*Service).serve(0xc2080102a0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cluster/service.go:83 +0x102
created by github.com/influxdb/influxdb/cluster.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cluster/service.go:60 +0x138

goroutine 40 [select]:
github.com/influxdb/influxdb/services/precreator.(*Service).runPrecreation(0xc20800aeb0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/precreator/service.go:74 +0x407
created by github.com/influxdb/influxdb/services/precreator.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/precreator/service.go:52 +0x19f

goroutine 41 [chan receive]:
github.com/influxdb/influxdb/tcp.(*listener).Accept(0xc208036fa8, 0x0, 0x0, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tcp/mux.go:117 +0x80
github.com/influxdb/influxdb/services/snapshotter.(*Service).serve(0xc20800afa0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/snapshotter/service.go:75 +0xad
created by github.com/influxdb/influxdb/services/snapshotter.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/snapshotter/service.go:48 +0x133

goroutine 42 [IO wait]:
net.(*pollDesc).Wait(0xc208011330, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011330, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080112d0, 0x0, 0x7f52c6b75e30, 0xc2080e8008)
    /usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20811a078, 0xc208021180, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20811a078, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc20806e420, 0x7f52c6b77378, 0xc20811a078, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1728 +0x92
net/http.Serve(0x7f52c6b77378, 0xc20811a078, 0x7f52c6b78d78, 0xc2080e80e0, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1606 +0xa1
github.com/influxdb/influxdb/services/admin.(*Service).serve(0xc20800b040)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/admin/service.go:107 +0x147
created by github.com/influxdb/influxdb/services/admin.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/admin/service.go:70 +0x472

goroutine 43 [select]:
github.com/influxdb/influxdb/services/continuous_querier.(*Service).backgroundLoop(0xc208010310)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/continuous_querier/service.go:157 +0x3e9
created by github.com/influxdb/influxdb/services/continuous_querier.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/continuous_querier/service.go:93 +0x2df

goroutine 44 [IO wait]:
net.(*pollDesc).Wait(0xc208011480, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011480, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208011420, 0x0, 0x7f52c6b75e30, 0xc2080e8428)
    /usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20811a080, 0xc2080bddc8, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20811a080, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208114780, 0x7f52c6b77378, 0xc20811a080, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1728 +0x92
net/http.Serve(0x7f52c6b77378, 0xc20811a080, 0x7f52c6b78c60, 0xc208010380, 0x0, 0x0)
    /usr/local/go/src/net/http/server.go:1606 +0xa1
github.com/influxdb/influxdb/services/httpd.(*Service).serve(0xc20800b180)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/service.go:108 +0x79
created by github.com/influxdb/influxdb/services/httpd.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/service.go:76 +0x68b

goroutine 45 [select]:
github.com/influxdb/influxdb/services/retention.(*Service).deleteShardGroups(0xc20806eba0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/retention/service.go:68 +0x433
created by github.com/influxdb/influxdb/services/retention.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/retention/service.go:45 +0x138

goroutine 46 [select]:
github.com/influxdb/influxdb/services/retention.(*Service).deleteShards(0xc20806eba0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/retention/service.go:102 +0x821
created by github.com/influxdb/influxdb/services/retention.(*Service).Open
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/retention/service.go:46 +0x152

goroutine 47 [chan receive]:
github.com/influxdb/influxdb/cmd/influxd/run.(*Server).startServerReporting(0xc20807c0e0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/server.go:390 +0x1d7
created by github.com/influxdb/influxdb/cmd/influxd/run.func·003
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/server.go:339 +0xf0e

goroutine 48 [select]:
github.com/influxdb/influxdb/cmd/influxd/run.(*Command).monitorServerErrors(0xc20807e000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/command.go:136 +0x2da
created by github.com/influxdb/influxdb/cmd/influxd/run.(*Command).Run
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/cmd/influxd/run/command.go:118 +0xde2

goroutine 56 [chan receive]:
github.com/influxdb/influxdb/services/httpd.(*Handler).serveQuery(0xc208010380, 0x7f52c6b7a780, 0xc20801f3a0, 0xc20815c000, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:260 +0xa44
github.com/influxdb/influxdb/services/httpd.*Handler.(github.com/influxdb/influxdb/services/httpd.serveQuery)·fm(0x7f52c6b7a780, 0xc20801f3a0, 0xc20815c000, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:95 +0x4f
github.com/influxdb/influxdb/services/httpd.func·001(0x7f52c6b7a780, 0xc20801f3a0, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:608 +0x78
net/http.HandlerFunc.ServeHTTP(0xc2080d8560, 0x7f52c6b7a780, 0xc20801f3a0, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·002(0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:666 +0x288
net/http.HandlerFunc.ServeHTTP(0xc2080d4c60, 0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·003(0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:675 +0xba
net/http.HandlerFunc.ServeHTTP(0xc2080d8580, 0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·004(0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:708 +0x31a
net/http.HandlerFunc.ServeHTTP(0xc2080d4c90, 0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·005(0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:718 +0x179
net/http.HandlerFunc.ServeHTTP(0xc2080d4cb0, 0x7f52c6b7a6a8, 0xc20801f300, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·006(0x7f52c6b7a6a8, 0xc20801f2e0, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:726 +0x123
net/http.HandlerFunc.ServeHTTP(0xc2080d85a0, 0x7f52c6b7a6a8, 0xc20801f2e0, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/influxdb/influxdb/services/httpd.func·007(0x7f52c6b7a670, 0xc20805a1e0, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:736 +0x12c
net/http.HandlerFunc.ServeHTTP(0xc2080d85c0, 0x7f52c6b7a670, 0xc20805a1e0, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1265 +0x41
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xc208036e88, 0x7f52c6b7a670, 0xc20805a1e0, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/bmizerany/pat/mux.go:109 +0x21c
github.com/influxdb/influxdb/services/httpd.(*Handler).ServeHTTP(0xc208010380, 0x7f52c6b7a670, 0xc20805a1e0, 0xc20815c000)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/services/httpd/handler.go:167 +0x256
net/http.serverHandler.ServeHTTP(0xc208114780, 0x7f52c6b7a670, 0xc20805a1e0, 0xc20815c000)
    /usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc20805a140)
    /usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 57 [chan receive]:
github.com/influxdb/influxdb/tsdb.(*QueryExecutor).executeSelectStatement(0xc208069d80, 0x0, 0xc208174000, 0xc208152840, 0x2710, 0x0, 0x0)
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/query_executor.go:265 +0xe1
github.com/influxdb/influxdb/tsdb.func·008()
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/query_executor.go:146 +0x6bd
created by github.com/influxdb/influxdb/tsdb.(*QueryExecutor).ExecuteQuery
    /tmp/tmp.ebF47h4lNX/src/github.com/influxdb/influxdb/tsdb/query_executor.go:196 +0x1f5

[beckettsean]

@otoolep looks like leader election is failing immediately?

@tsailiming Please give the exact writes and queries for "Upon startup, access the web browser and issue a query. This started happening after simple insert, drop measurement, insert and queries." The panic alone is insufficient to determine what is happening.

Is this a single node cluster?

[otoolep]

Possibly @beckettsean, @jwilder

I am seeing multiple issues with this trace, I am hopeful #3646 will fix it.

[jwilder]

Leader election can take a little time to stabilize at startup. It looks like when it started up and replayed the log, it became leader, lost leadership and then regain it. The panic shows it was the leader when this happened as well. I think #3646 should fix this.

[otoolep]

Believe fixed by #3646, please let us know if you see the issue with the next nightly build or 0.9.3.

[tsailiming]

This was a straight forward single node test. Had issues from stable 0.9.2 before trying the nightly build.

Steps were:

1. Create a database
2. Create a series of data with a single measurement. Various fields
3. Drop measurement.
4. Run a simple "select * from measurement" query.
5. Repeating Steps 2-4

I was debugging my code and were inserting only about 10 records of various forms.

{ "deviceidentifier" : "00137a000001f3e6", "sensoridentifier" : "01_ILLUMINANCEMEASURE_S", "data" : "12345", "datatype" : "0x21", "protocolid" : "1", "time" : 1439567564 }
{ "deviceidentifier" : "00137a000001f3e6", "sensoridentifier" : "01_ILLUMINANCEMEASURE_S", "data" : "12345", "datatype" : "0x21", "protocolid" : "1", "timestamp" : 1439567564 }
{"FOO": 1234}
{"FOO": 4}   
{"FOO: 4, "deviceidentifier": "device", "sensoridentifier":"sensor"}
{"FOO": 4, "deviceidentifier": "device", "sensoridentifier":"sensor"}
{"FOO": 5, "deviceidentifier": "device", "sensoridentifier":"sensor", "time":1439570628}
{"FOO": 6, "deviceidentifier": "device", "sensoridentifier":"sensor", "time":1439570638}            
{"FOO": 7, "deviceidentifier": "device", "sensoridentifier":"sensor"}                       
{"FOO": 8, "deviceidentifier": "device", "sensoridentifier":"sensor", "time":1439570638}
{"FOO": 9, "deviceidentifier": "device", "sensoridentifier":"sensor", "time":"1439570638"}

I was also testing the retention policy of 1h.

The exact query was select * from ".broker.pub.1.00137a000001f3e6.01_ILLUMINANCEMEASURE_S"

Will test the next build with fixed #3646.

Thanks!

[beckettsean]

@tsailiming the JSON write protocol is deprecated and will be removed in an upcoming release. Please transition your writes to the line protocol: https://influxdb.com/docs/v0.9/write_protocols/line.html, https://influxdb.com/docs/v0.9/write_protocols/write_syntax.html.

Performance issues related to the JSON protocol are not something we will investigate since it is due for deletion soon.

[beckettsean]

@tsailiming in addition, your query looks very strange. That's more of an 0.8 measurement name than an 0.9 measurement name, and it doesn't match up with your sample writes at all.

[tsailiming]

Forgot to add that I was using a combination of MQTT and Fluentd with the InfluxDB plugin. The json structure was MQTT messages.

The InfluxDB plugin uses the ruby library and creates the measurement based on the Fluentd tag/mqtt topic name.

Here's a sample query using the python library:

>>> pprint.pprint(list(conn.query('select * from ".broker.pub.1.00137a000001f3e6.01_ILLUMINANCEMEASURE_S"')))
[[{u'FOO': None,
   u'data': u'12345',
   u'datatype': u'0x21',
   u'protocolid': u'1',
   u'time': u'2015-08-14T15:52:44Z'},
  {u'FOO': None,
   u'data': u'999',
   u'datatype': u'0x21',
   u'protocolid': u'1',
   u'time': u'2015-08-14T16:43:58Z'}],
 [{u'FOO': 5,
   u'data': None,
   u'datatype': None,
   u'protocolid': None,
   u'time': u'2015-08-14T16:43:48Z'},
  {u'FOO': 9,
   u'data': None,
   u'datatype': None,
   u'protocolid': None,
   u'time': u'2015-08-14T16:43:58Z'},
  {u'FOO': 6,
   u'data': None,
   u'datatype': None,
   u'protocolid': None,
   u'time': u'2015-08-14T16:43:58Z'},
  {u'FOO': 7,
   u'data': None,
   u'datatype': None,
   u'protocolid': None,
   u'time': u'2015-08-14T16:50:11Z'}]]
>>> pprint.pprint(list(conn.query('select * from ".broker.pub.1.00137a000001f3e6.01_ILLUMINANCEMEASURE_S"')))
[[{u'data': u'11',
   u'datatype': u'0x21',
   u'protocolid': u'1',
   u'time': u'2015-08-14T16:43:58Z'},
  {u'data': u'12',
   u'datatype': u'0x21',
   u'protocolid': u'1',
   u'time': u'2015-08-14T16:44:08Z'},
  {u'data': u'13',
   u'datatype': u'0x21',
   u'protocolid': u'1',
   u'time': u'2015-08-14T16:44:18Z'}]]

[tsailiming]

I installed the latest nightly build version 0.9.3-nightly-c2dbf16 and now I can perform the queries.

[otoolep]

Great, thanks for the update.

On Saturday, August 15, 2015, Tsai Li Ming notifications@github.com wrote:

I installed the latest nightly build version 0.9.3-nightly-c2dbf16 and
now I can perform the queries.

—
Reply to this email directly or view it on GitHub
#3664 (comment).