Cardinality Limits

[jwilder]

Feature Request

Proposal:

The server should have configuration limits for operators to prevent high cardinality tags and number of series.

Current behavior:

A writing process can easily write tag value that creates a high cardinality series causing increased memory usage and more risk of OOMing the process. Many times the operator of the system is not the same as the user writing to the system. This means that the operator cannot prevent an errant process from overloading the system easily.

Desired behavior:

One (or two) config options to limit tag and series cardinality at write time.

• Max Tag Cardinality - Maximum number of distinct values a given tag can have. This would apply per-shard.
• Max Series Cardinality - Maximum number of distinct series. This would apply per-shard.

When these limits are hit, writes that would increase this number would be rejected and an error logged in the system indicating the problem.

[dgnorton]

@jwilder suggestions for sane default values on the config settings? Seems like it would vary based on system. Would it make more sense to fail writes based on memory usage?

[jwilder]

Fixed via #7095