Use zeroes instead of whitespaces as padding bytes

[jpountz]

When generating ids for conflicts, rally uses spaces as padding bytes. I rarely see spaces in ids, so I think this is less realistic than using zeroes? Furthermore Elasticsearch has some optimizations for ids that look like base64 or numeric ids, which get defeated by the usage of spaces.

[jpountz]

The following patch should do the trick:

diff --git a/esrally/track/params.py b/esrally/track/params.py
index c7c079a..5f8bb3f 100644
--- a/esrally/track/params.py
+++ b/esrally/track/params.py
@@ -622,7 +622,7 @@ def build_conflicting_ids(conflicts, docs_to_index, offset, shuffle=random.shuff
     all_ids = [0] * docs_to_index
     for i in range(docs_to_index):
         # always consider the offset as each client will index its own range and we don't want uncontrolled conflicts across clients
-        all_ids[i] = "%10d" % (offset + i)
+        all_ids[i] = "%010d" % (offset + i)
     if conflicts == IndexIdConflict.RandomConflicts:
         shuffle(all_ids)
     return all_ids

[jpountz]

Changing this makes ids compressed at the Elasticsearch level rather than Lucene. It tends to help because Elaticsearch's level compression does things a bit more efficiently since it doesn't need to preserve ordering (while Lucene needs to for range queries). This means you won't see Lucene's LowercaseAsciiCompression class in the hot classes reported by telemetry.

[jpountz]

As a data point, I can index geonames 5% faster (230k docs/s -> 242k docs/s quite consistently) with this change if no fields are indexed in the mapping (enabled=false on the root object mapper), which is likely a best-case scenario for this change.

[danielmitterdorfer]

Thanks for the analysis. I think the id structure is a leftover from the pre-Rally days even. :) This definitely makes sense to change. @drawlerr can you please look into this?