Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot boot after a crash - data is effectively lost now #904

Closed
mattheworiordan opened this issue Sep 3, 2014 · 6 comments
Closed

Cannot boot after a crash - data is effectively lost now #904

mattheworiordan opened this issue Sep 3, 2014 · 6 comments

Comments

@mattheworiordan
Copy link

After running a join query that caused InfluxDB to crash, I am now unable to start up my InfluxDB database. This is pretty worrying as I have recently started using InfluxDB to store data that we cannot discard, and I am no longer able to recover this data. This happened once before, but I suspect it was a one off, but clearly it's not. I am running InfluxDB 0.8 on Ubuntu 14.04 in a Docker LXC container.

Any advice on what steps I could take to recover my data and InfluxDB would be appreciated.

As you can see from my InfluxDB log below, the server starts up, detects shards, and then never makes the admin or REST interface available.

In the console of the InfluxDB process

[09/03/14 22:45:46] [INFO] Loading configuration file /config.toml

+---------------------------------------------+
|  _____        __ _            _____  ____   |
| |_   _|      / _| |          |  __ \|  _ \  |
|   | |  _ __ | |_| |_   ___  _| |  | | |_) | |
|   | | | '_ \|  _| | | | \ \/ / |  | |  _ <  |
|  _| |_| | | | | | | |_| |>  <| |__| | |_) | |
| |_____|_| |_|_| |_|\__,_/_/\_\_____/|____/  |
+---------------------------------------------+

In my log file

[2014/09/03 22:45:46 UTC] [INFO] (main.setupLogging:52) Redirectoring logging to /var/log/influxdb/influxdb.log
[2014/09/03 22:45:46 UTC] [INFO] (main.main:124) Starting Influx Server 0.8.0 bound to 0.0.0.0...
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/server.NewServer:40) Opening database at /influxdb/data/db
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/wal.NewWAL:40) Opening wal in /influxdb/data/wal
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/api/http.(*HttpServer).EnableSsl:77) Ssl will be disabled since the ssl port or certificate path weren't set
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).Serve:550) Initializing Raft HTTP server
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).Serve:561) Raft Server Listening at 0.0.0.0:8090
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).startRaft:376) Initializing Raft Server: http://4d8d34c42484:8090
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).Recovery:676) Recovering the cluster configuration
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/cluster.(*ClusterConfiguration).Recovery:695) Checking whether 13aa72fb168d1542 is the local server 13aa72fb168d1542
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00087
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00181
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00236
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00225
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00072
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00129
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00131
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00145
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00157
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00158
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00174
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00195
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00167
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00218
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00106
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00125
[2014/09/03 22:45:46 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00139
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00143
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00179
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00212
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00081
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00094
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00122
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00126
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00135
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00194
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00206
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00104
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00138
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00029
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00230
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00064
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00182
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00021
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00046
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00070
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00193
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00198
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00237
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00062
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00073
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00082
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00110
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00136
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00165
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00115
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00121
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00137
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00151
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00009
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00032
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00091
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00074
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00057
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00209
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00242
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00086
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00159
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00161
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00166
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00219
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00040
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00127
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00154
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00190
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00208
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00010
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00017
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00053
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00079
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00243
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00043
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00168
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00231
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00045
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00180
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00049
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00088
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00112
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00119
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00133
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00170
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00204
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00034
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00100
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00176
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00200
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00093
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00105
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00111
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00201
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00221
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00067
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00177
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00229
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00038
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00103
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00188
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00061
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00083
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00186
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00246
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00012
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00024
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00160
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00191
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00117
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00223
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00234
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00240
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00140
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00146
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00156
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00018
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00071
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00090
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00124
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00148
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00037
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00211
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00220
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00097
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00116
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00130
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00153
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00162
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00011
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00023
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00207
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00235
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00042
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00055
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00068
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00078
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00095
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00098
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00226
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00019
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00030
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00113
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00134
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00171
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00185
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00232
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00008
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00060
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00092
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00147
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00196
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00217
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00015
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00020
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00144
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00245
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00013
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00016
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00063
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00066
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00213
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00216
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00244
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00187
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00026
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00033
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00059
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00099
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00150
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00172
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00173
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00224
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00005
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00164
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00107
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00175
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00202
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00004
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00007
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00096
[2014/09/03 22:45:47 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00114
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00128
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00152
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00155
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00132
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00189
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00001
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00003
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00044
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00163
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00227
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00118
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00120
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00035
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00056
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00109
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00022
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00025
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00041
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00238
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00051
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00052
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00089
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00142
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00149
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00169
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00228
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00002
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00027
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00076
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00183
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00199
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00205
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00222
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00039
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00215
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00102
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00123
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00141
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00178
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00031
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00050
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00065
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00184
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00210
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00203
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00241
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00036
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00058
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00077
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00006
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00014
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00054
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00075
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00197
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00047
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00048
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00069
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00084
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00108
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00233
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00080
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00085
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00214
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00239
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00028
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00101
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/datastore.(*ShardDatastore).GetOrCreateShard:162) DATASTORE: opening or creating shard /influxdb/data/db/shard_db_v2/00192
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).startRaft:397) Recovered from log
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:92) Waiting for local server to be added
[2014/09/03 22:45:48 UTC] [INFO] (github.com/influxdb/influxdb/wal.(*WAL).SetServerId:109) Setting server id to 1 and recovering
[2014/09/03 22:45:50 UTC] [INFO] (github.com/influxdb/influxdb/coordinator.(*RaftServer).raftEventHandler:436) (raft:13aa72fb168d1542) Selected as leader. Starting leader loop.
[2014/09/03 22:45:53 UTC] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:108) Sending change connection string command (ed2e15c229d6:8099,4d8d34c42484:8099) (http://ed2e15c229d6:8090,http://4d8d34c42484:8090)
@toddboom
Copy link
Contributor

toddboom commented Sep 4, 2014

@mattheworiordan Just out of curiosity, how much memory is available to the process? I ran into an instance recently where insufficient memory (1GB) caused repeated crashes on startup. Moving it to a larger VM fixed the issue.

@mattheworiordan
Copy link
Author

@toddboom 3.7GB on the machine, and no limits as far as I can tell. The problem is not that it was crashing, the process did not exit, it just never proceeded and never presented any errors.

@pauldix
Copy link
Member

pauldix commented Sep 4, 2014

What happens if you start Influx in a console interactively? Wondering if
it's outputting anything there.

On Wed, Sep 3, 2014 at 11:00 PM, mattheworiordan notifications@github.com
wrote:

@toddboom https://github.com/toddboom 3.7GB on the machine, and no
limits as far as I can tell. The problem is not that it was crashing, the
process did not exit, it just never proceeded and never presented any
errors.

—
Reply to this email directly or view it on GitHub
#904 (comment).

@mattheworiordan
Copy link
Author

@pauldix in my ticket you will see I included the output from a console session as well as the output from the log file

@jvshahid
Copy link
Contributor

jvshahid commented Sep 4, 2014

Hi @mattheworiordan, let me rephrase the issue to make sure I understand it correctly. Whenever you bring up the node, the daemon just hangs with the given output in the log. Assuming that the service doesn't die and just hangs, I think this issue is due to the raft node trying to change its address. Specifically the lilne below is what's causing the issue:

[2014/09/03 22:45:53 UTC] [INFO] (github.com/influxdb/influxdb/server.(*Server).ListenAndServe:108) Sending change connection string command (ed2e15c229d6:8099,4d8d34c42484:8099) (http://ed2e15c229d6:8090,http://4d8d34c42484:8090)

The node is trying to change its hotsname from ed2e15c229d6 to 4d8d34c42484. Changing the connection string of a node is tricky and has many edge cases. In your case, you can fix this issue by setting the hostname in the config file to ed2e15c229d6 to force the node to use the old hostname. The hostname config should be on line 6 in the toml config file.

@jvshahid
Copy link
Contributor

duplicate of #914

@mattheworiordan mattheworiordan mentioned this issue Sep 25, 2014
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pauldix @mattheworiordan @toddboom @jvshahid