Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"free(): invalid pointer" in RHEL 5.9 #44

Closed
dreampuf opened this issue Nov 13, 2013 · 13 comments
Closed

"free(): invalid pointer" in RHEL 5.9 #44

dreampuf opened this issue Nov 13, 2013 · 13 comments

Comments

@dreampuf
Copy link 

$ sudo /etc/init.d/influxdb start
Starting the process influxdb [ OK ]
Anomalous agent started [ OK ]

$ *** glibc detected *** /usr/bin/influxdb: free(): invalid pointer: 0x0000000000e30fa0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3a4be7164f]
/lib64/libc.so.6(cfree+0x4b)[0x3a4be7587b]
/usr/lib64/libstdc++.so.6(_ZNSs7reserveEm+0x9e)[0x3a5e89cb9e]
/usr/lib64/libstdc++.so.6(_ZNSs6appendEmc+0x60)[0x3a5e89cc90]
/usr/bin/influxdb(_ZN7leveldb10WriteBatchC1Ev+0x1d)[0x42f20d]
/usr/bin/influxdb(_ZN7leveldb6DBImplC2ERKNS_7OptionsERKSs+0x1bf)[0x41b8bf]
/usr/bin/influxdb(_ZN7leveldb2DB4OpenERKNS_7OptionsERKSsPPS0_+0x3f)[0x41fa6f]
/usr/bin/influxdb(leveldb_open+0x32)[0x416fb2]
/usr/bin/influxdb(_cgo_d9589830f294_Cfunc_leveldb_open+0x14)[0x4169e4]
/usr/bin/influxdb[0x46b9c4]
======= Memory map: ========
00400000-00c11000 r-xp 00000000 68:03 7144311                            /opt/influxdb/versions/0.1.0/influxdb
00e10000-00e11000 r--p 00810000 68:03 7144311                            /opt/influxdb/versions/0.1.0/influxdb
00e11000-00e31000 rw-p 00811000 68:03 7144311                            /opt/influxdb/versions/0.1.0/influxdb
00e31000-00e52000 rw-p 00e31000 00:00 0
0ac13000-0ac34000 rw-p 0ac13000 00:00 0                                  [heap]
3a4ba00000-3a4ba1c000 r-xp 00000000 68:03 13664258                       /lib64/ld-2.5.so
3a4bc1c000-3a4bc1d000 r--p 0001c000 68:03 13664258                       /lib64/ld-2.5.so
3a4bc1d000-3a4bc1e000 rw-p 0001d000 68:03 13664258                       /lib64/ld-2.5.so
3a4be00000-3a4bf4f000 r-xp 00000000 68:03 13664260                       /lib64/libc-2.5.so
3a4bf4f000-3a4c14e000 ---p 0014f000 68:03 13664260                       /lib64/libc-2.5.so
3a4c152000-3a4c153000 rw-p 00152000 68:03 13664260                       /lib64/libc-2.5.so
3a4c153000-3a4c158000 rw-p 3a4c153000 00:00 0
3a4c600000-3a4c616000 r-xp 00000000 68:03 13664273                       /lib64/libpthread-2.5.so
3a4c616000-3a4c816000 ---p 00016000 68:03 13664273                       /lib64/libpthread-2.5.so
3a4c816000-3a4c817000 r--p 00016000 68:03 13664273                       /lib64/libpthread-2.5.so
3a4c817000-3a4c818000 rw-p 00017000 68:03 13664273                       /lib64/libpthread-2.5.so
3a4c818000-3a4c81c000 rw-p 3a4c818000 00:00 0
3a4ca00000-3a4ca82000 r-xp 00000000 68:03 13664291                       /lib64/libm-2.5.so
3a4ca82000-3a4cc81000 ---p 00082000 68:03 13664291                       /lib64/libm-2.5.so
3a4cc81000-3a4cc82000 r--p 00081000 68:03 13664291                       /lib64/libm-2.5.so
3a4cc82000-3a4cc83000 rw-p 00082000 68:03 13664291                       /lib64/libm-2.5.so
3a5b400000-3a5b40d000 r-xp 00000000 68:03 13664610                       /lib64/libgcc_s-4.1.2-20080825.so.1
3a5b40d000-3a5b60d000 ---p 0000d000 68:03 13664610                       /lib64/libgcc_s-4.1.2-20080825.so.1
3a5b60d000-3a5b60e000 rw-p 0000d000 68:03 13664610                       /lib64/libgcc_s-4.1.2-20080825.so.1
3a5e800000-3a5e8e6000 r-xp 00000000 68:03 3272237                        /usr/lib64/libstdc++.so.6.0.8
3a5e8e6000-3a5eae5000 ---p 000e6000 68:03 3272237                        /usr/lib64/libstdc++.so.6.0.8
3a5eae5000-3a5eaeb000 r--p 000e5000 68:03 3272237                        /usr/lib64/libstdc++.so.6.0.8
3a5eaeb000-3a5eaee000 rw-p 000eb000 68:03 3272237                        /usr/lib64/libstdc++.so.6.0.8
3a5eaee000-3a5eb00000 rw-p 3a5eaee000 00:00 0
c000000000-c000001000 rw-p c000000000 00:00 0
c20fff0000-c210100000 rw-p c20fff0000 00:00 0
2b85d0efa000-2b85d0efc000 rw-p 2b85d0efa000 00:00 0
2b85d0f15000-2b85d10e8000 rw-p 2b85d0f15000 00:00 0
2b85d10e8000-2b85d10e9000 ---p 2b85d10e8000 00:00 0
2b85d10e9000-2b85d1bea000 rw-p 2b85d10e9000 00:00 0
2b85d1bea000-2b85d1beb000 ---p 2b85d1bea000 00:00 0
2b85d1beb000-2b85d25eb000 rw-p 2b85d1beb000 00:00 0
2b85d25eb000-2b85d25ec000 ---p 2b85d25eb000 00:00 0
2b85d25ec000-2b85d2fec000 rw-p 2b85d25ec000 00:00 0
2b85d2fec000-2b85d2fed000 ---p 2b85d2fec000 00:00 0
2b85d2fed000-2b85d39ed000 rw-p 2b85d2fed000 00:00 0
2b85d39ed000-2b85d39ee000 ---p 2b85d39ed000 00:00 0
2b85d39ee000-2b85d43ee000 rw-p 2b85d39ee000 00:00 0
2b85d43ee000-2b85d43ef000 ---p 2b85d43ee000 00:00 0
2b85d43ef000-2b85d4e0f000 rw-p 2b85d43ef000 00:00 0
7fff294ec000-7fff29501000 rw-p 7ffffffe9000 00:00 0                      [stack]
7fff2955f000-7fff29562000 r-xp 7fff2955f000 00:00 0                      [vdso]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0                  [vsyscall]

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.9 (Tikanga)
@jvshahid
Copy link
Contributor

What version of influxdb are you running (influxdb -v) or is that master ?
On Nov 13, 2013 3:05 AM, "Dreampuf" notifications@github.com wrote:

$ sudo /etc/init.d/influxdb start
Starting the process influxdb [ OK ]
Anomalous agent started [ OK ]

$ *** glibc detected *** /usr/bin/influxdb: free(): invalid pointer: 0x0000000000e30fa0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3a4be7164f]
/lib64/libc.so.6(cfree+0x4b)[0x3a4be7587b]
/usr/lib64/libstdc++.so.6(_ZNSs7reserveEm+0x9e)[0x3a5e89cb9e]
/usr/lib64/libstdc++.so.6(_ZNSs6appendEmc+0x60)[0x3a5e89cc90]
/usr/bin/influxdb(_ZN7leveldb10WriteBatchC1Ev+0x1d)[0x42f20d]
/usr/bin/influxdb(_ZN7leveldb6DBImplC2ERKNS_7OptionsERKSs+0x1bf)[0x41b8bf]
/usr/bin/influxdb(ZN7leveldb2DB4OpenERKNS_7OptionsERKSsPPS0+0x3f)[0x41fa6f]
/usr/bin/influxdb(leveldb_open+0x32)[0x416fb2]
/usr/bin/influxdb(_cgo_d9589830f294_Cfunc_leveldb_open+0x14)[0x4169e4]
/usr/bin/influxdb[0x46b9c4]
======= Memory map: ========
00400000-00c11000 r-xp 00000000 68:03 7144311 /opt/influxdb/versions/0.1.0/influxdb
00e10000-00e11000 r--p 00810000 68:03 7144311 /opt/influxdb/versions/0.1.0/influxdb
00e11000-00e31000 rw-p 00811000 68:03 7144311 /opt/influxdb/versions/0.1.0/influxdb
00e31000-00e52000 rw-p 00e31000 00:00 0
0ac13000-0ac34000 rw-p 0ac13000 00:00 0 [heap]
3a4ba00000-3a4ba1c000 r-xp 00000000 68:03 13664258 /lib64/ld-2.5.so
3a4bc1c000-3a4bc1d000 r--p 0001c000 68:03 13664258 /lib64/ld-2.5.so
3a4bc1d000-3a4bc1e000 rw-p 0001d000 68:03 13664258 /lib64/ld-2.5.so
3a4be00000-3a4bf4f000 r-xp 00000000 68:03 13664260 /lib64/libc-2.5.so
3a4bf4f000-3a4c14e000 ---p 0014f000 68:03 13664260 /lib64/libc-2.5.so
3a4c152000-3a4c153000 rw-p 00152000 68:03 13664260 /lib64/libc-2.5.so
3a4c153000-3a4c158000 rw-p 3a4c153000 00:00 0
3a4c600000-3a4c616000 r-xp 00000000 68:03 13664273 /lib64/libpthread-2.5.so
3a4c616000-3a4c816000 ---p 00016000 68:03 13664273 /lib64/libpthread-2.5.so
3a4c816000-3a4c817000 r--p 00016000 68:03 13664273 /lib64/libpthread-2.5.so
3a4c817000-3a4c818000 rw-p 00017000 68:03 13664273 /lib64/libpthread-2.5.so
3a4c818000-3a4c81c000 rw-p 3a4c818000 00:00 0
3a4ca00000-3a4ca82000 r-xp 00000000 68:03 13664291 /lib64/libm-2.5.so
3a4ca82000-3a4cc81000 ---p 00082000 68:03 13664291 /lib64/libm-2.5.so
3a4cc81000-3a4cc82000 r--p 00081000 68:03 13664291 /lib64/libm-2.5.so
3a4cc82000-3a4cc83000 rw-p 00082000 68:03 13664291 /lib64/libm-2.5.so
3a5b400000-3a5b40d000 r-xp 00000000 68:03 13664610 /lib64/libgcc_s-4.1.2-20080825.so.1
3a5b40d000-3a5b60d000 ---p 0000d000 68:03 13664610 /lib64/libgcc_s-4.1.2-20080825.so.1
3a5b60d000-3a5b60e000 rw-p 0000d000 68:03 13664610 /lib64/libgcc_s-4.1.2-20080825.so.1
3a5e800000-3a5e8e6000 r-xp 00000000 68:03 3272237 /usr/lib64/libstdc++.so.6.0.8
3a5e8e6000-3a5eae5000 ---p 000e6000 68:03 3272237 /usr/lib64/libstdc++.so.6.0.8
3a5eae5000-3a5eaeb000 r--p 000e5000 68:03 3272237 /usr/lib64/libstdc++.so.6.0.8
3a5eaeb000-3a5eaee000 rw-p 000eb000 68:03 3272237 /usr/lib64/libstdc++.so.6.0.8
3a5eaee000-3a5eb00000 rw-p 3a5eaee000 00:00 0
c000000000-c000001000 rw-p c000000000 00:00 0
c20fff0000-c210100000 rw-p c20fff0000 00:00 0
2b85d0efa000-2b85d0efc000 rw-p 2b85d0efa000 00:00 0
2b85d0f15000-2b85d10e8000 rw-p 2b85d0f15000 00:00 0
2b85d10e8000-2b85d10e9000 ---p 2b85d10e8000 00:00 0
2b85d10e9000-2b85d1bea000 rw-p 2b85d10e9000 00:00 0
2b85d1bea000-2b85d1beb000 ---p 2b85d1bea000 00:00 0
2b85d1beb000-2b85d25eb000 rw-p 2b85d1beb000 00:00 0
2b85d25eb000-2b85d25ec000 ---p 2b85d25eb000 00:00 0
2b85d25ec000-2b85d2fec000 rw-p 2b85d25ec000 00:00 0
2b85d2fec000-2b85d2fed000 ---p 2b85d2fec000 00:00 0
2b85d2fed000-2b85d39ed000 rw-p 2b85d2fed000 00:00 0
2b85d39ed000-2b85d39ee000 ---p 2b85d39ed000 00:00 0
2b85d39ee000-2b85d43ee000 rw-p 2b85d39ee000 00:00 0
2b85d43ee000-2b85d43ef000 ---p 2b85d43ee000 00:00 0
2b85d43ef000-2b85d4e0f000 rw-p 2b85d43ef000 00:00 0
7fff294ec000-7fff29501000 rw-p 7ffffffe9000 00:00 0 [stack]
7fff2955f000-7fff29562000 r-xp 7fff2955f000 00:00 0 [vdso]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vsyscall]

$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.9 (Tikanga)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/44
.

@dreampuf
Copy link
Author

Install file: http://s3.amazonaws.com/influxdb/influxdb-latest-1.x86_64.rpm

$ influxdb -v
InfluxDB v0.1.0 (git: )

@jvshahid
Copy link
Contributor

Can you attach the logs, you can find them here /opt/influxdb/shared/log.txt

@jvshahid
Copy link
Contributor

Just curious, is that reproducible or intermittent crash ?

@dreampuf
Copy link
Author 

$ cat /opt/influxdb/shared/log.txt
2013/11/13 03:00:08 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 03:00:08 Starting Influx Server...
2013/11/13 03:00:08 Opening database at  /opt/influxdb/shared/data/db
2013/11/13 03:00:08 Initializing Raft HTTP server
2013/11/13 03:00:08 Listening at: http://localhost:8090
2013/11/13 03:00:08 Initializing Raft Server: /opt/influxdb/shared/data/raft 8090
2013/11/13 03:00:08 Couldn't contact a leader so initializing new cluster for server on port:  8090
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2ad30a728bc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2ad30a728ba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0x18148690, 0x18149780, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2ad30a728bc0
github.com/jmhodges/levigo.Open(0xc21004ef40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2ad30a728c28
datastore.NewLevelDbDatastore(0xc21004ef40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2ad30a728cd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2ad30a728f48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2ad30a728fa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2ad30a728fa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc2100522c0, 0x2ad30a587fe8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:27 +0x60
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [IO wait]:
net.runtime_pollWait(0x2ad30a5918b0, 0x72, 0x0)
    /tmp/bindist902150335/go/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc21000b220, 0x72, 0x2ad30a5881e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc21000b220, 0xb, 0x2ad30a5881e8)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc21000b1c0, 0x8e0f48, 0x0, 0x2ad30a5881e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000040, 0x18, 0xc2100a6010, 0x4f4973)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000040, 0xc21000aaf0, 0xc21005d9c0, 0x49, 0xeca152888)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc21000a230, 0x2ad30a590910, 0xc210000040, 0x0, 0x0)
    /home/jvshahid/bin/go/src/pkg/net/http/server.go:1622 +0x91
coordinator.(*RaftServer).Serve(0xc210089280, 0x2ad30a590910, 0xc210000040, 0xe3bc78, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:267 +0x59f
coordinator.(*RaftServer).ListenAndServe(0xc210089280, 0xe3bc78, 0x0, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:245 +0x17d
main.func·002()
    /home/jvshahid/codez/influxdb/src/server/server.go:104 +0x4d
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

goroutine 7 [runnable]:
code.google.com/p/go.crypto/blowfish.ExpandKey(0xc2100c63b8, 0x5, 0x8, 0xc2100e8000)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/go.crypto/blowfish/block.go:12
code.google.com/p/go.crypto/bcrypt.expensiveBlowfishSetup(0xc2100c63a8, 0x4, 0x4, 0xc20000000a, 0xc2100c4be0, ...)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/go.crypto/bcrypt/bcrypt.go:226 +0x227
code.google.com/p/go.crypto/bcrypt.bcrypt(0xc2100c63a8, 0x4, 0x4, 0xa, 0xc2100c4be0, ...)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/go.crypto/bcrypt/bcrypt.go:190 +0x119
code.google.com/p/go.crypto/bcrypt.newFromPassword(0xc2100c63a8, 0x4, 0x4, 0xa, 0xc2100c63a8, ...)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/go.crypto/bcrypt/bcrypt.go:150 +0x1e8
code.google.com/p/go.crypto/bcrypt.GenerateFromPassword(0xc2100c63a8, 0x4, 0x4, 0xa, 0x4, ...)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/go.crypto/bcrypt/bcrypt.go:89 +0x4a
coordinator.hashPassword(0x82a500, 0x4, 0xc2100c93c0, 0x466662, 0x30, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/user.go:133 +0x59
coordinator.(*CommonUser).changePassword(0xc2100c9db0, 0x82a500, 0x4, 0x8, 0x79ed00)
    /home/jvshahid/codez/influxdb/src/coordinator/user.go:43 +0x31
coordinator.(*RaftServer).CreateRootUser(0xc210089280, 0x2ad30a590798, 0xc2100c6100)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:151 +0x90
coordinator.(*RaftServer).startRaft(0xc210089280, 0xe3bc78, 0x0, 0x0, 0x0)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:227 +0x9db
created by coordinator.(*RaftServer).Serve
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:252 +0xd8

goroutine 8 [chan receive]:
github.com/influxdb/raft.(*server).leaderLoop(0xc2100c8000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:698 +0x2b3
github.com/influxdb/raft.(*server).loop(0xc2100c8000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:501 +0x31a
created by github.com/influxdb/raft.(*server).Start
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:421 +0x7a4

rax     0x0
rbx     0x7fff25be57e0
rcx     0xffffffffffffffff
rdx     0x6
rdi     0x1aa8
rsi     0x1aa8
rbp     0x2ad30a575e20
rsp     0x7fff25be5738
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x202
r12     0x9
r13     0x7fff25be5a60
r14     0x7fff25be5900
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x202
cs      0x33
fs      0x0
gs      0x0
2013/11/13 03:02:03 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 03:02:03 Starting Influx Server...
2013/11/13 03:02:03 Opening database at  /opt/influxdb/shared/data/db
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2b14fc8f8bc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2b14fc8f8ba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0x1e745690, 0x1e746760, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2b14fc8f8bc0
github.com/jmhodges/levigo.Open(0xc210047f40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2b14fc8f8c28
datastore.NewLevelDbDatastore(0xc210047f40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2b14fc8f8cd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2b14fc8f8f48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2b14fc8f8fa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2b14fc8f8fa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [runnable]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc21004b2c0, 0x2b14fc7580a8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:23
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [runnable]:
main.func·002()
    /home/jvshahid/codez/influxdb/src/server/server.go:103
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

rax     0x0
rbx     0x7fffc1901120
rcx     0xffffffffffffffff
rdx     0x6
rdi     0x49a4
rsi     0x49a4
rbp     0x2b14fc745e20
rsp     0x7fffc1901078
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x202
r12     0x5
r13     0x7fffc19013a0
r14     0x7fffc1901240
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x202
cs      0x33
fs      0x0
gs      0x0
2013/11/13 03:04:43 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 03:04:43 Starting Influx Server...
2013/11/13 03:04:43 Opening database at  /opt/influxdb/shared/data/db
2013/11/13 03:04:43 Initializing Raft HTTP server
2013/11/13 03:04:43 Initializing Raft Server: /opt/influxdb/shared/data/raft 8090
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2b85d10c9bc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2b85d10c9ba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0xac137d0, 0xac148e0, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2b85d10c9bc0
github.com/jmhodges/levigo.Open(0xc210047f40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2b85d10c9c28
datastore.NewLevelDbDatastore(0xc210047f40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2b85d10c9cd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2b85d10c9f48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2b85d10c9fa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2b85d10c9fa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc21004b2c0, 0x2b85d0f28fe8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:27 +0x60
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [runnable]:
unicode.SimpleFold(0x67, 0xc2100c2000)
    /home/jvshahid/bin/go/src/pkg/unicode/letter.go:331
regexp/syntax.(*compiler).rune(0xc210000060, 0xc210051478, 0x1, 0x2, 0x0, ...)
    /home/jvshahid/bin/go/src/pkg/regexp/syntax/compile.go:271 +0x1ba
regexp/syntax.(*compiler).compile(0xc210000060, 0xc2100463f0, 0x200000001)
    /home/jvshahid/bin/go/src/pkg/regexp/syntax/compile.go:110 +0x1be
regexp/syntax.(*compiler).compile(0xc210000060, 0xc2100464d0, 0x4621ed)
    /home/jvshahid/bin/go/src/pkg/regexp/syntax/compile.go:156 +0x8b0
regexp/syntax.Compile(0xc2100464d0, 0xc2100464d0, 0x1, 0x1)
    /home/jvshahid/bin/go/src/pkg/regexp/syntax/compile.go:83 +0x5c
regexp.compile(0xc210047620, 0x11, 0xc2100000d4, 0x11, 0x832cd0, ...)
    /home/jvshahid/bin/go/src/pkg/regexp/regexp.go:152 +0xd8
regexp.Compile(0xc210047620, 0x11, 0x11, 0xc210047620, 0x11)
    /home/jvshahid/bin/go/src/pkg/regexp/regexp.go:109 +0x3d
github.com/gorilla/mux.newRouteRegexp(0x832cd0, 0xf, 0x0, 0x8, 0x808201, ...)
    /home/jvshahid/codez/influxdb/src/github.com/gorilla/mux/regexp.go:95 +0xb1c
github.com/gorilla/mux.(*Route).addRegexpMatcher(0xc210046230, 0x832cd0, 0xf, 0x2b85d10e0000, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/gorilla/mux/route.go:151 +0x178
github.com/gorilla/mux.(*Route).Path(0xc210046230, 0x832cd0, 0xf, 0x0)
    /home/jvshahid/codez/influxdb/src/github.com/gorilla/mux/route.go:280 +0x45
github.com/gorilla/mux.(*Router).HandleFunc(0xc21000ac80, 0x832cd0, 0xf, 0xc210036280, 0xc210036260)
    /home/jvshahid/codez/influxdb/src/github.com/gorilla/mux/mux.go:162 +0x13a
coordinator.(*RaftServer).Serve(0xc210082640, 0x2b85d0f318e8, 0xc210000040, 0xe3bc78, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:261 +0x235
coordinator.(*RaftServer).ListenAndServe(0xc210082640, 0xe3bc78, 0x0, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:245 +0x17d
main.func·002()
    /home/jvshahid/codez/influxdb/src/server/server.go:104 +0x4d
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

goroutine 7 [runnable]:
strings.ContainsRune(0x86dcd0, 0x1b, 0x6e, 0xa)
    /home/jvshahid/bin/go/src/pkg/strings/strings.go:122
encoding/json.isValidTag(0x865cd6, 0x10, 0x865cd6)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:745 +0xa9
encoding/json.typeFields(0x2b85d0f28ed8, 0x7ee080, 0x2b85d0f28ed8, 0x7ee080, 0x0)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:1026 +0x4ef
encoding/json.cachedTypeFields(0x2b85d0f28ed8, 0x7ee080, 0x80e820, 0xf026f67f4f7d6af9, 0x448fcd)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:1156 +0xcd
encoding/json.newStructEncoder(0x2b85d0f28ed8, 0x7ee080, 0x770b80)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:596 +0x42
encoding/json.newTypeEncoder(0x2b85d0f28ed8, 0x7ee080, 0x2b85d0f28e01, 0x7ee080)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:398 +0x405
encoding/json.typeEncoder(0x2b85d0f28ed8, 0x7ee080, 0x7ee080)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:348 +0x1a5
encoding/json.newPtrEncoder(0x2b85d0f28ed8, 0x7fb900, 0x770b80)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:716 +0x45
encoding/json.newTypeEncoder(0x2b85d0f28ed8, 0x7fb900, 0x2b85d0f28e01, 0x7fb900)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:406 +0x3a6
encoding/json.typeEncoder(0x2b85d0f28ed8, 0x7fb900, 0x7fb900)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:348 +0x1a5
encoding/json.newArrayEncoder(0x2b85d0f28ed8, 0x701ee0, 0x7fb900)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:699 +0x45
encoding/json.newSliceEncoder(0x2b85d0f28ed8, 0x701ee0, 0x770b80)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:678 +0x76
encoding/json.newTypeEncoder(0x2b85d0f28ed8, 0x701ee0, 0x2b85d0f28e01, 0x701ee0)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:402 +0x3c9
encoding/json.typeEncoder(0x2b85d0f28ed8, 0x701ee0, 0xc2100ba060)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:348 +0x1a5
encoding/json.newStructEncoder(0x2b85d0f28ed8, 0x7ba9e0, 0x770b80)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:602 +0x19a
encoding/json.newTypeEncoder(0x2b85d0f28ed8, 0x7ba9e0, 0x2b85d0f28e01, 0x7ba9e0)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:398 +0x405
encoding/json.typeEncoder(0x2b85d0f28ed8, 0x7ba9e0, 0x7ba9e0)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:348 +0x1a5
encoding/json.newPtrEncoder(0x2b85d0f28ed8, 0x6f8120, 0x770b80)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:716 +0x45
encoding/json.newTypeEncoder(0x2b85d0f28ed8, 0x6f8120, 0x2b85d0f28e01, 0x6f8120)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:406 +0x3a6
encoding/json.typeEncoder(0x2b85d0f28ed8, 0x6f8120, 0x160)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:348 +0x1a5
encoding/json.valueEncoder(0x6f8120, 0xc2100b93e0, 0x160, 0x51ce6b)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:319 +0x70
encoding/json.(*encodeState).reflectValue(0xc210048000, 0x6f8120, 0xc2100b93e0, 0x160)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:305 +0x3b
encoding/json.(*encodeState).marshal(0xc210048000, 0x6f8120, 0xc2100b93e0, 0x0, 0x0)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:276 +0xa6
encoding/json.Marshal(0x6f8120, 0xc2100b93e0, 0x2b85d4df08c8, 0xe3bc78, 0x0, ...)
    /home/jvshahid/bin/go/src/pkg/encoding/json/encode.go:132 +0x6c
github.com/influxdb/raft.(*server).writeConf(0xc2100bc000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:1286 +0x184
github.com/influxdb/raft.(*server).AddPeer(0xc2100bc000, 0xc2100ba110, 0x7, 0xc2100b93a0, 0x15, ...)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:1003 +0x201
github.com/influxdb/raft.(*DefaultJoinCommand).Apply(0xc2100b9320, 0x2b85d0f329c8, 0xc2100bc000, 0x3e, 0x1, ...)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/join_command.go:22 +0x69
github.com/influxdb/raft.func·003(0x2b85d0f293e8, 0xc2100b9320, 0xc2100b8180, 0x3e, 0x3e, ...)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:169 +0x58
github.com/influxdb/raft.(*Log).open(0xc21004e000, 0xc2100bd3c0, 0x22, 0x2b85d0f32b80, 0xc2100bd1e0)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/log.go:191 +0x9cb
github.com/influxdb/raft.(*server).Start(0xc2100bc000, 0x2b85d0f329c8, 0xc2100bc000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:396 +0x396
coordinator.(*RaftServer).startRaft(0xc210082640, 0xe3bc78, 0x0, 0x0, 0x0)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:195 +0x348
created by coordinator.(*RaftServer).Serve
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:252 +0xd8

rax     0x0
rbx     0x7fff294fdf20
rcx     0xffffffffffffffff
rdx     0x6
rdi     0x645
rsi     0x645
rbp     0x2b85d0f16e20
rsp     0x7fff294fde78
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x202
r12     0x7
r13     0x7fff294fe1a0
r14     0x7fff294fe040
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x202
cs      0x33
fs      0x0
gs      0x0
2013/11/13 03:24:25 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 03:24:25 Starting Influx Server...
2013/11/13 03:24:25 Opening database at  /opt/influxdb/shared/data/db
2013/11/13 03:24:25 Initializing Raft HTTP server
2013/11/13 03:24:25 Listening at: http://localhost:8090
2013/11/13 03:24:25 Initializing Raft Server: /opt/influxdb/shared/data/raft 8090
2013/11/13 03:24:25 Recovered from log
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2ad84ef2fbc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2ad84ef2fba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0x11b937d0, 0x11b948e0, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2ad84ef2fbc0
github.com/jmhodges/levigo.Open(0xc210047f40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2ad84ef2fc28
datastore.NewLevelDbDatastore(0xc210047f40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2ad84ef2fcd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2ad84ef2ff48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2ad84ef2ffa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2ad84ef2ffa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc21004b2c0, 0x2ad84ed8efe8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:27 +0x60
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [IO wait]:
net.runtime_pollWait(0x2ad84ed98888, 0x72, 0x0)
    /tmp/bindist902150335/go/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc2100461b0, 0x72, 0x2ad84ed8f1e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc2100461b0, 0xb, 0x2ad84ed8f1e8)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc210046150, 0x8e0f48, 0x0, 0x2ad84ed8f1e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000040, 0x18, 0xc2100b3010, 0x4f4973)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000040, 0xc21000ab90, 0xc210051820, 0x49, 0xeca152e39)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc21000a230, 0x2ad84ed978e8, 0xc210000040, 0x0, 0x0)
    /home/jvshahid/bin/go/src/pkg/net/http/server.go:1622 +0x91
coordinator.(*RaftServer).Serve(0xc2100835a0, 0x2ad84ed978e8, 0xc210000040, 0xe3bc78, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:267 +0x59f
coordinator.(*RaftServer).ListenAndServe(0xc2100835a0, 0xe3bc78, 0x0, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:245 +0x17d
main.func·002()
    /home/jvshahid/codez/influxdb/src/server/server.go:104 +0x4d
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

goroutine 8 [chan receive]:
github.com/influxdb/raft.(*server).leaderLoop(0xc2100bf000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:698 +0x2b3
github.com/influxdb/raft.(*server).loop(0xc2100bf000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:501 +0x31a
created by github.com/influxdb/raft.(*server).Start
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:421 +0x7a4

rax     0x0
rbx     0x7fff8a8296b0
rcx     0xffffffffffffffff
rdx     0x6
rdi     0x2791
rsi     0x2791
rbp     0x2ad84ed7ce20
rsp     0x7fff8a829608
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x202
r12     0x8
r13     0x7fff8a829930
r14     0x7fff8a8297d0
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x202
cs      0x33
fs      0x0
gs      0x0
2013/11/13 03:29:08 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 03:29:08 Starting Influx Server...
2013/11/13 03:29:08 Opening database at  /opt/influxdb/shared/data/db
2013/11/13 03:29:08 Initializing Raft HTTP server
2013/11/13 03:29:08 Listening at: http://localhost:8090
2013/11/13 03:29:08 Initializing Raft Server: /opt/influxdb/shared/data/raft 8090
2013/11/13 03:29:08 Recovered from log
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2b4603ff5bc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2b4603ff5ba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0x4873690, 0x4874780, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2b4603ff5bc0
github.com/jmhodges/levigo.Open(0xc21004ef40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2b4603ff5c28
datastore.NewLevelDbDatastore(0xc21004ef40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2b4603ff5cd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2b4603ff5f48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2b4603ff5fa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2b4603ff5fa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc2100522c0, 0x2b4603e54fe8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:27 +0x60
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [IO wait]:
net.runtime_pollWait(0x2b4603e5e888, 0x72, 0x0)
    /tmp/bindist902150335/go/src/pkg/runtime/netpoll.goc:116 +0x6a
net.(*pollDesc).Wait(0xc21000b220, 0x72, 0x2b4603e551e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:81 +0x34
net.(*pollDesc).WaitRead(0xc21000b220, 0xb, 0x2b4603e551e8)
    /home/jvshahid/bin/go/src/pkg/net/fd_poll_runtime.go:86 +0x30
net.(*netFD).accept(0xc21000b1c0, 0x8e0f48, 0x0, 0x2b4603e551e8, 0xb)
    /home/jvshahid/bin/go/src/pkg/net/fd_unix.go:382 +0x2c2
net.(*TCPListener).AcceptTCP(0xc210000040, 0x18, 0xc2100a6010, 0x4f4973)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:233 +0x47
net.(*TCPListener).Accept(0xc210000040, 0xc21000aaa0, 0xc2100568f0, 0x49, 0xeca152f54)
    /home/jvshahid/bin/go/src/pkg/net/tcpsock_posix.go:243 +0x27
net/http.(*Server).Serve(0xc21000a230, 0x2b4603e5d8e8, 0xc210000040, 0x0, 0x0)
    /home/jvshahid/bin/go/src/pkg/net/http/server.go:1622 +0x91
coordinator.(*RaftServer).Serve(0xc2100883c0, 0x2b4603e5d8e8, 0xc210000040, 0xe3bc78, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:267 +0x59f
coordinator.(*RaftServer).ListenAndServe(0xc2100883c0, 0xe3bc78, 0x0, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/coordinator/raft_server.go:245 +0x17d
main.func·002()
    /home/jvshahid/codez/influxdb/src/server/server.go:104 +0x4d
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

goroutine 8 [select]:
github.com/influxdb/raft.(*server).followerLoop(0xc2100b5000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:542 +0x6ed
github.com/influxdb/raft.(*server).loop(0xc2100b5000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:495 +0x2c4
created by github.com/influxdb/raft.(*server).Start
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:421 +0x7a4

rax     0x0
rbx     0x7fff21e949e0
rcx     0xffffffffffffffff
rdx     0x6
rdi     0xb24
rsi     0xb24
rbp     0x2b4603e42e20
rsp     0x7fff21e94938
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x206
r12     0x8
r13     0x7fff21e94c60
r14     0x7fff21e94b00
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x206
cs      0x33
fs      0x0
gs      0x0
2013/11/13 10:00:41 Loading Config from /opt/influxdb/shared/config.json
2013/11/13 10:00:41 Starting Influx Server...
2013/11/13 10:00:41 Opening database at  /opt/influxdb/shared/data/db
2013/11/13 10:00:41 Initializing Raft HTTP server
2013/11/13 10:00:41 Initializing Raft Server: /opt/influxdb/shared/data/raft 8090
2013/11/13 10:00:41 Recovered from log
SIGABRT: abort
PC=0x3a4be30265
signal arrived during cgo execution

runtime.cgocall(0x4169d0, 0x2b7e8fe6dbc0)
    /home/jvshahid/bin/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2b7e8fe6dba8
github.com/jmhodges/levigo._Cfunc_leveldb_open(0x162aa690, 0x162ab760, 0xc210000438, 0xe11ff8)
    github.com/jmhodges/levigo/_obj/_cgo_defun.c:278 +0x31 fp=0x2b7e8fe6dbc0
github.com/jmhodges/levigo.Open(0xc210046f40, 0x1c, 0xc210000420, 0x0, 0x0, ...)
    /home/jvshahid/codez/influxdb/src/github.com/jmhodges/levigo/db.go:79 +0xb6 fp=0x2b7e8fe6dc28
datastore.NewLevelDbDatastore(0xc210046f40, 0x1c, 0x2, 0x7180c0, 0xc210036ad0, ...)
    /home/jvshahid/codez/influxdb/src/datastore/leveldb_datastore.go:66 +0xcb fp=0x2b7e8fe6dcd0
main.main()
    /home/jvshahid/codez/influxdb/src/server/server.go:117 +0x905 fp=0x2b7e8fe6df48
runtime.main()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:220 +0x11f fp=0x2b7e8fe6dfa0
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394 fp=0x2b7e8fe6dfa8

goroutine 3 [syscall]:
os/signal.loop()
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/jvshahid/bin/go/src/pkg/os/signal/signal_unix.go:27 +0x31

goroutine 4 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc21004a2c0, 0x2b7e8fcccfe8, 0xc210000008)
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:27 +0x60
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/jvshahid/codez/influxdb/src/code.google.com/p/log4go/termlog.go:19 +0x67

goroutine 5 [running]:
    goroutine running on other thread; stack unavailable
created by main.main
    /home/jvshahid/codez/influxdb/src/server/server.go:105 +0x683

goroutine 6 [syscall]:
runtime.goexit()
    /home/jvshahid/bin/go/src/pkg/runtime/proc.c:1394

goroutine 8 [runnable]:
github.com/influxdb/raft.(*server).loop(0xc2100bc000)
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:486
created by github.com/influxdb/raft.(*server).Start
    /home/jvshahid/codez/influxdb/src/github.com/influxdb/raft/server.go:421 +0x7a4

rax     0x0
rbx     0x7fffdf5722b0
rcx     0xffffffffffffffff
rdx     0x6
rdi     0x6349
rsi     0x6349
rbp     0x2b7e8fcbae20
rsp     0x7fffdf572208
r8      0x80
r9      0x101010101010101
r10     0x8
r11     0x202
r12     0x8
r13     0x7fffdf572530
r14     0x7fffdf5723d0
r15     0x3a4bf237c8
rip     0x3a4be30265
rflags  0x202
cs      0x33
fs      0x0
gs      0x0

@jvshahid
Copy link
Contributor

Great, thanks.

@dreampuf
Copy link
Author

@jvshahid
BTW, I saw many 3.* MB leveldb files after a lot points insert into the database.
May I set the leveldb file custom size ? I have an approximately estimate.

@jvshahid
Copy link
Contributor

so wait, was the server working before and suddenly started crashing ?

Can you check the free space on your disk.

@pauldix
Copy link
Member

pauldix commented Nov 13, 2013

You can't set the size of the leveldb files. They'll keep stacking up.
We're currently running systems with > 60k files in there and it's ok. Are
you seeing problems with it?

On Wed, Nov 13, 2013 at 11:50 AM, Dreampuf notifications@github.com wrote:

@jvshahid https://github.com/jvshahid
BTW, I saw many 3.* MB leveldb files after a lot points insert into the
database.
May I set the leveldb file custom size ? I have an approximately estimate.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/44#issuecomment-28411363
.

@dreampuf
Copy link
Author

the disk space of the server is free enough.

/dev/cciss/c0d0p3       63G   4.7G    55G   8% /
/dev/cciss/c0d0p1      104M    17M    82M  17% /boot
/dev/cciss/c0d1p1      1.5T   194G   1.3T  14% /export
tmpfs                   26G   431k    26G   1% /dev/shm

This issue still exists in the server.
I have test Influxdb on my mac osx. The 3.* MB leveldb files were in it. I didn't test it fully ( only about several hundred million points ).

@jvshahid
Copy link
Contributor

I finally managed to get a vm with centos 5.9 running on my machine and can repro the issue. Updates on this issue soon.

@jvshahid
Copy link
Contributor

Problem is fixed with the latest build. There were incompatibilities between the glibc on the machine that we're using to build the packages and centos5.9, we're now using centos to build the packages which seems to be working fine on debian and centos with now issues.

Please feel free to reopen this issue if the problem isn't resolved with the latest package.

Thanks,

@dreampuf
Copy link
Author

It's work!
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pauldix @jvshahid @dreampuf