[leo_storage][auto-compaction] Each storage node's data-compaction happened at almost same time

[yosukehara]

I've recognized an issue by a benchmark of LeoFS. The data-compaction of Leo's storage does not affect the front processing.

• Test Result: https://github.com/leo-project/notes/tree/master/leofs/benchmark/leofs/recover_with_front/20150519_recover_node_1.2.9

[yosukehara]

I've fixed this issue that each storage's data-compaction is executed step by step.
A number of nodes to be executed at the same time is one in case of a small storage-cluster. On the other hand, in case of a large storage-cluster, LeoStorage depends on the rule of execution of the auto-compaction as below:

<a number of data-compaction nodes at the same time> = 
        round(<total number of active nodes> / <a number of replicas>) - 1
ex)
* Total number of active nodes = 15
* A number of replicas = 3
* A number of data-compaction nodes at the same time = 4

(note: minimum value: 1)

I've been considering calculation of a number of data-compaction nodes at the same time.

[yosukehara]

I've reached the most suitable rule is as below:

<a number of data-compaction nodes at the same time> = 
        round(<total number of active nodes> x <coefficient>)

* coefficient:
    * High: 0.1
    * Middle: 0.075
    * Low: 0.05

I've considered that an auto-compaction processing needs to not affect front-end operations as much as possible.

• coefficient = 0.075

[yosukehara]

According to a benchmark report, I/we have recognized we need to more improve the auto-compaction mechanism. I've been considering I implement a negotiation(consensus) mechanism w/candidate nodes of execution of a data-compaction.

[yosukehara]

We've finally taken a good result - https://github.com/leo-project/notes/blob/master/leofs/benchmark/leofs/recover_with_front/20150629_recover_node_1.2.11-dev11/README.md

[mocchira]

@yosukehara
This result seems that compaction only happened at leofs17 and leofs18 ( not happened at leofs1[4-6] ).
Is it OK?

[yosukehara]

@mocchira I've recognized that. But leo_storage was little bit old develop version. The latest version is works fine. Tomorrow, I'm planning to take a benchmark before LeoFS v1.2.11 release.

[yosukehara]

Today, we've taken a benchmark - recover-node + front-operations w/watchdog and auto-compaction, which is good result for us.