Why etcd only keeps the responses of the most recent 1000 events across all etcd keys?

[zxhdaniel]

Why it is 1000? but not 10000?

Why not expose it as a parameter of etcd?

[xiang90]

@zxhdaniel When we designed the watch event history, we think it only need to be large enough to hold the gap between two continuous watch.

Watch A
hand over result to someone else to process
Watch A again from prev Index

This latency between to watch should not exceed 1second in normal case.
The throughput of etcd was about 1K req/second at that time. So 1000 is enough to hold the gap.

Today, people are using the event history in various ways. Some want to audit the system via event history. Some want to use the event history as a consistent queue. Some might also want to not have the event history. However, today's backend store of etcd is not designed for holding a large amount of data by any mean. We cannot simply change that to 10K or as the user wants. It will have a huge impact on the stability and reliability of etcd.

Trying to solve this problem, we have been designing a new API and store. It keeps all the event and user can compact whenever they want. You can leave some feedback at #2675.

Thanks for the question.

/cc @yichengq @barakmich @philips @kelseyhightower