Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error compacting TSM files: tsm file closed #6990

Closed
freeseacher opened this issue Jul 9, 2016 · 1 comment
Closed

error compacting TSM files: tsm file closed #6990

freeseacher opened this issue Jul 9, 2016 · 1 comment
Labels
Milestone

Comments

@freeseacher
Copy link

version 0.13.0, branch 0.13, commit e57fb88
while loading got error

[tsm1] 2016/07/09 19:45:03 beginning level 2 compaction of group 0, 2 TSM files
[tsm1] 2016/07/09 19:45:03 compacting level 2 group (0) /var/lib/influxdb/data/noc/d90/163/000000082-000000002.tsm (#0)
[tsm1] 2016/07/09 19:45:03 compacting level 2 group (0) /var/lib/influxdb/data/noc/d90/163/000000084-000000002.tsm (#1)
[tsm1] 2016/07/09 19:45:03 beginning level 3 compaction of group 0, 4 TSM files
[tsm1] 2016/07/09 19:45:03 compacting level 3 group (0) /var/lib/influxdb/data/noc/d90/163/000000004-000000003.tsm (#0)
[tsm1] 2016/07/09 19:45:03 compacting level 3 group (0) /var/lib/influxdb/data/noc/d90/163/000000008-000000003.tsm (#1)
[tsm1] 2016/07/09 19:45:03 compacting level 3 group (0) /var/lib/influxdb/data/noc/d90/163/000000012-000000003.tsm (#2)
[tsm1] 2016/07/09 19:45:03 compacting level 3 group (0) /var/lib/influxdb/data/noc/d90/163/000000016-000000003.tsm (#3)
[tsm1] 2016/07/09 19:45:16 error compacting TSM files: tsm file closed
[tsm1] 2016/07/09 19:47:30 error compacting TSM files: tsm file closed

it looks like database load had stopped.
no further activity in log for at least 15 minutes and top shows huge cpu load on influxd process

@freeseacher
Copy link
Author

so after 20 minutes in cpu load and no activity in log i got

panic: runtime error: index out of range

goroutine 27 [running]:
panic(0xb5b240, 0xc82000e070)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/influxdata/influxdb/models.scanMeasurement(0xc82029b6c8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/models/points.go:381 +0x19f
github.com/influxdata/influxdb/models.ParseKey(0x0, 0x0, 0x0, 0x0, 0x1, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/models/points.go:144 +0xae
github.com/influxdata/influxdb/tsdb.MeasurementFromSeriesKey(0x0, 0x0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/meta.go:1834 +0x37
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).addToIndexFromKey(0xc820292000, 0xa3, 0x0, 0x0, 0x2, 0xc8201f0f00, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:314 +0x8e
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).LoadMetadataIndex.func1(0x0, 0x0, 0x1, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:219 +0xc3
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*FileStore).WalkKeys(0xc820230100, 0xc82029ba48, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/file_store.go:233 +0x186
github.com/influxdata/influxdb/tsdb/engine/tsm1.(*Engine).LoadMetadataIndex(0xc820292000, 0xa3, 0xc8201f0f00, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/engine/tsm1/engine.go:213 +0xd7
github.com/influxdata/influxdb/tsdb.(*Shard).Open.func1(0xc82022c7e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/shard.go:171 +0x2d5
github.com/influxdata/influxdb/tsdb.(*Shard).Open(0xc82022c7e0, 0x0, 0x0)
    /root/go/src/github.com/influxdata/influxdb/tsdb/shard.go:145 +0x3d
github.com/influxdata/influxdb/tsdb.(*Store).loadShards.func1(0xc820154578, 0xc8201e01e0, 0xc8201590e0, 0xc8201f0f00, 0xc8201f5f57, 0x3, 0xc8201fe29b, 0x3, 0xc8201db90f, 0x3)
    /root/go/src/github.com/influxdata/influxdb/tsdb/store.go:177 +0x6ec
created by github.com/influxdata/influxdb/tsdb.(*Store).loadShards
    /root/go/src/github.com/influxdata/influxdb/tsdb/store.go:185 +0xa2e

thats looks familiar.

jwilder added a commit that referenced this issue Jul 28, 2016
jwilder added a commit that referenced this issue Jul 29, 2016
@timhallinflux timhallinflux added this to the 1.0.0 milestone Dec 20, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants