Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compact() should not update queue lengths if a compaction has locked the files to be compacted #22138

Closed
davidby-influx opened this issue Aug 9, 2021 · 0 comments · Fixed by #22195 or #22235

Comments

@davidby-influx
Copy link
Contributor

When the compaction planner runs, if it cannot acquire a lock on the files it plans to compact, it returns a nil list of compaction groups. This, in turn, sets the engine statistics for compactions queues to zero, which is incorrect. This behavior results in misleading output like the following graph which alternates between ~1500 compactions in the queue and zero.

image

@davidby-influx davidby-influx self-assigned this Aug 9, 2021
davidby-influx added a commit that referenced this issue Aug 9, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect.

closes #22138
davidby-influx added a commit that referenced this issue Aug 9, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect.

closes #22138
@davidby-influx davidby-influx linked a pull request Aug 9, 2021 that will close this issue
4 tasks
davidby-influx added a commit that referenced this issue Aug 12, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138
@davidby-influx davidby-influx linked a pull request Aug 12, 2021 that will close this issue
4 tasks
davidby-influx added a commit that referenced this issue Aug 16, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138
davidby-influx added a commit that referenced this issue Aug 16, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22139
davidby-influx added a commit that referenced this issue Aug 16, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22139
davidby-influx added a commit that referenced this issue Aug 17, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141
davidby-influx added a commit that referenced this issue Aug 17, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141
davidby-influx added a commit that referenced this issue Aug 17, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141
davidby-influx added a commit that referenced this issue Aug 17, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141
davidby-influx added a commit that referenced this issue Aug 17, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141

(cherry picked from commit 9923d2e)

closes #22142
davidby-influx added a commit that referenced this issue Aug 18, 2021
When the compaction planner runs, if it cannot acquire
a lock on the files it plans to compact, it returns a
nil list of compaction groups. This, in turn, sets the
engine statistics for compactions queues to zero,
which is incorrect. Instead, use the length of pending
files which would have been returned.

closes #22138

(cherry picked from commit 7d3efe1)

closes #22141

(cherry picked from commit 9923d2e)

closes #22142
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant