Enforce retention policy minimum (currently 1 hour)

[corylanou]

This is a follow up PR for ##1897

[otoolep]

I feel this could be defined in a better place -- server.go, with the other constants there. It makes more sense to group it with those, as opposed to here. influxdb.go is in the same package as server.go so you can still reference the constant in this file.

[corylanou]

Hmm, I think I had a "duh" moment there. I thought I looked for constants but I clearly missed them. Moved.

[otoolep]

Looks good though I think the constant is defined in the wrong file.

[otoolep]

Nit: typo in comment.

[otoolep]

+1

[toddboom]

👍

[jnutzmann]

After this was added, I get this response

{"results":[{"error":"retention policy duration needs to be at least 1h0m0s"}]}

to this query:

CREATE RETENTION POLICY rp_inf ON properties DURATION INF REPLICATION 1

This query succeeds in 0.9.0-rc10, but fails in the current master.

[otoolep]

Thanks @jnutzmann -- you have spotted a regression. I have opened #1918 and this will be fixed soon.

[jogo]

I was testing things out from influxql/INFLUXQL.md and tried CREATE RETENTION POLICY "10m.events" ON somedb DURATION 10m REPLICATION 2; which failed due to the 1 hour retention policy minimum.

• It would be nice to document the why behind this limit so it doesn't appear so arbitrary.
• Looks like some of the in tree documentation needs updating.

Why is there a 1 hour minimum, and what about testing. I wanted to create a very short retention policy (say a minute or two) to test out how the retention policy works without having to wait for ever.

[otoolep]

This minimum is in place to prevent an excessive number of shards from being created. If the retention period was 1 minute long, the system would create new shards every minute.

Since retention enforcement is based on the timestamp within the event, not reception time, simply write data older than 1 hour (assuming you've selected a retention period of 1 hour) and that data will be evicted within minutes.

[jogo]

I see shards are pre-created now and I assume creating a shard is relatively expensive.

Documenting this trick somewhere in testing related documentation would be nice. Along with documenting why this new limit was added.