Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

spread cassandra write load over time #99

Closed
Dieterbe opened this issue Jan 8, 2016 · 3 comments
Closed

spread cassandra write load over time #99

Dieterbe opened this issue Jan 8, 2016 · 3 comments

Comments

@Dieterbe
Copy link
Contributor

Dieterbe commented Jan 8, 2016

currently write load spikes every chunkSpan because we try to save all chunks whenever they are ready to be saved.

we should investigate whether this has any negative impact on cassandra read performance and/or if we can solve this with cassandra tuning.

if not, we can change MT to spread the write load over time.

@woodsaj
Copy link
Member

woodsaj commented Jan 8, 2016

With #95 writes are already spread based on the speed at which cassandra will process writes. The write load being sent to cassandra can be tuned by adjusting cassandra-write-concurrency option. If set to 1, then a single goroutine will sequentially read from the writeQueue and send to cassandra.

The default configuration of Cassandra should already prevent a high rate of writes from preventing reads and vice-versa. Though we probably should verify this ourselves.

@Dieterbe
Copy link
Contributor Author

Dieterbe commented Feb 1, 2016

the mere act of adding all chunks to the write queue at once instead of spread out in time is problematic though:
it makes tuning the write queue hard (#125) and also triggers cpu spikes on the NMT machines (#105)

@stale
Copy link

stale bot commented Apr 4, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 4, 2020
@stale stale bot closed this as completed Apr 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants