Add note about mustache template #826

hub-cap · 2019-11-21T22:48:54Z

closes #823

hub-cap · 2019-11-21T22:49:03Z

The links are the same ones from the GH issue.

dliappis

Thanks for the PR! Left a few suggestions, the main one about giving a concrete example in the put-pipeline operation docs.

dliappis · 2019-11-22T09:15:27Z

docs/adding_tracks.rst

@@ -626,6 +626,9 @@ Template Language

 Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

+.. note::


Reading the ticket (and thinking about the maximum visibility and benefit) I think we want to have a small example besides an operation.

What if we place a second example inside the put pipeline operation doc page showing a similar example to what we have in the Elasticsearch docs for handling failure in pipelines.
We could raise the attention with a note that "if you need to use Elasticsearch Mustache {{ }} syntax you must escape it properly" (and link the note you added in this PR), for example:

{ "name": "define-ip-geocoder", "operation-type": "put-pipeline", "id": "ip-geocoder", "body": { "description": "Extracts location information from the client IP.", "processors": [ { "geoip": { "field": "clientip", "properties": [ "city_name", "country_iso_code", "country_name", "location" ] } } ], "on_failure": [ { "set" : { "field" : "_index", {% raw %} "value" : "failed-{{ _index }}" {% endraw %} } } ] }

^^ For the sake of not repeating things we could even just show "processors": [ ... ], as used in the Elasticsearch docs page.

@dliappis I decided to just add to the existing example, and added a link from the adding tracks page.

dliappis · 2019-11-22T09:18:56Z

docs/adding_tracks.rst

@@ -626,6 +626,9 @@ Template Language

 Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

+.. note::
+
+    Elasticsearch utilizes Mustache formatting in a few places, notably in `Search templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-template.html>`_ and `Watcher templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/actions-email.html>`_. If you are using Mustache in your rally templates you must `escape them properly <https://jinja.palletsprojects.com/en/2.10.x/templates/#escaping>`_.


nit: we use Rally (capitalized) consistently everywhere.

danielmitterdorfer

I left a few comments.

danielmitterdorfer · 2019-12-16T07:13:20Z

docs/adding_tracks.rst

@@ -626,6 +626,7 @@ Template Language

 Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

+Elasticsearch utilizes Mustache formatting in a few places, notably in `Search templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-template.html>`_ and `Watcher templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/actions-email.html>`_. If you are using Mustache in your Rally templates you must `escape them properly <https://jinja.palletsprojects.com/en/2.10.x/templates/#escaping>`_. See :ref:`put_pipeline` for an example.


Any reason you've removed the .. note:: that you've added in the previous commit?

Nit: "Search templates" -> "search templates"

ya i removed it cuz it got big enough for a full paragraph and just looked like a huge note w/ a bunch of links, rather than some concise text. I can change it back tho if youd like

danielmitterdorfer · 2019-12-16T07:25:03Z

docs/track.rst

@@ -427,6 +429,16 @@ Example::
              ]
            }
          }
+        ],
+        "on-failure": [


There is a syntax error in the pipeline. The element is called on_failureinstead of on-failure (see docs).

danielmitterdorfer · 2019-12-16T07:26:27Z

docs/track.rst

+            "set": {
+              "field": "_index",
+              {% raw %}
+              "value": "failed-{{ _index }}"


I think this is pretty complex to grasp. We should add a note about the raw / endraw tags below the example and we should also link to the Elasticsearch docs so users can lookup what this is actually doing.

~~so are you saying to revert it, or to just add links to show what its doing better?~~

disregard: reread your comment and made a commit out of it.

danielmitterdorfer

Thanks for iterating. I left a couple more comments.

danielmitterdorfer · 2019-12-17T06:39:31Z

docs/track.rst

@@ -447,6 +447,8 @@ This example requires that the ``ingest-geoip`` Elasticsearch plugin is installe

 This is an administrative operation. Metrics are not reported by default. Reporting can be forced by setting ``include-in-reporting`` to ``true``.

+Please see `the pipeline documentation <https://www.elastic.co/guide/en/elasticsearch/reference/current/handling-failure-in-pipelines.html>`_ for details on handling failures in pipelines.


Maybe move this paragraph so it is directly below the example? Also, some introductory text would be nice; something along the lines of:

In the following example we setup a pipeline that ... . Note that we need to use raw / endraw texts to ensure that Rally passes the Mustache template as is and does not attempt to resolve it itself ...

(and then link back to your newly added note about templating Rally?)

danielmitterdorfer · 2019-12-17T06:40:54Z

docs/adding_tracks.rst

@@ -626,6 +626,7 @@ Template Language

 Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

+Elasticsearch utilizes Mustache formatting in a few places, notably in `search templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-template.html>`_ and `Watcher templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/actions-email.html>`_. If you are using Mustache in your Rally templates you must `escape them properly <https://jinja.palletsprojects.com/en/2.10.x/templates/#escaping>`_. See :ref:`put_pipeline` for an example.


Nit: If you are using Mustache in your Rally tracks?

danielmitterdorfer

Looks fine but I left one nit. Please merge without another review cycle after addressing it.

danielmitterdorfer · 2019-12-18T06:26:42Z

docs/track.rst

@@ -409,6 +409,8 @@ With the operation-type ``put-pipeline`` you can execute the `put pipeline API <
 * `id` (mandatory): Pipeline id
 * `body` (mandatory): Pipeline definition

+In this example we setup a pipeline that adds location information to a ingested document as well as a pipeline failure block to change the index in which the document was supposed to be written. Note that we need to use the ``raw`` and ``endraw`` blocks to ensure that Rally does not attempt to resolve the Mustache template. See :ref:`template_language` for more information.


nit:

:ref:`template_language`

is rendered as:

Template Language

(i.e. upper-case)

How about we spell it out explicitly?

See the documentation about Rally's :ref:`template language <template_language>` for more information.

ohhh i did not know this was possible. TIL!

Add note about mustache template

328cf5a

hub-cap added enhancement Improves the status quo :Docs Changes to the documentation labels Nov 21, 2019

hub-cap added this to the 1.4.0 milestone Nov 21, 2019

hub-cap requested a review from danielmitterdorfer November 21, 2019 22:48

dliappis reviewed Nov 22, 2019

View reviewed changes

hub-cap added 2 commits December 12, 2019 15:10

Add to put pipeline example and set a link in the adding track reference

7ca3659

upcase Rally

fa24421

danielmitterdorfer requested changes Dec 16, 2019

View reviewed changes

hub-cap added 5 commits December 16, 2019 14:28

on_failure

a138e7e

search templates

326951f

Add note about complex example

79367ed

Merge remote-tracking branch 'upstream/master' into note_for_escape

c68ef72

Fixing merge error

47de305

danielmitterdorfer reviewed Dec 17, 2019

View reviewed changes

updates from review

7bcd57b

danielmitterdorfer approved these changes Dec 18, 2019

View reviewed changes

Fix case on ref

f039d9a

hub-cap merged commit 146a623 into elastic:master Dec 18, 2019

hub-cap deleted the note_for_escape branch December 18, 2019 21:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add note about mustache template #826

Add note about mustache template #826

hub-cap commented Nov 21, 2019

hub-cap commented Nov 21, 2019 •

edited

Loading

dliappis left a comment

dliappis Nov 22, 2019

hub-cap Dec 12, 2019

dliappis Nov 22, 2019

danielmitterdorfer left a comment

danielmitterdorfer Dec 16, 2019

danielmitterdorfer Dec 16, 2019

hub-cap Dec 16, 2019

danielmitterdorfer Dec 16, 2019

danielmitterdorfer Dec 16, 2019

hub-cap Dec 16, 2019 •

edited

Loading

danielmitterdorfer left a comment

danielmitterdorfer Dec 17, 2019

hub-cap Dec 17, 2019

danielmitterdorfer Dec 17, 2019

danielmitterdorfer left a comment

danielmitterdorfer Dec 18, 2019

hub-cap Dec 18, 2019

		@@ -626,6 +626,9 @@ Template Language

		Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

		.. note::

		@@ -626,6 +626,7 @@ Template Language

		Rally uses `Jinja2 <http://jinja.pocoo.org/docs/dev/>`_ as a template language so you can use Jinja2 expressions in track files.

		Elasticsearch utilizes Mustache formatting in a few places, notably in `Search templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-template.html>`_ and `Watcher templates <https://www.elastic.co/guide/en/elasticsearch/reference/7.4/actions-email.html>`_. If you are using Mustache in your Rally templates you must `escape them properly <https://jinja.palletsprojects.com/en/2.10.x/templates/#escaping>`_. See :ref:`put_pipeline` for an example.

		@@ -447,6 +447,8 @@ This example requires that the ``ingest-geoip`` Elasticsearch plugin is installe

		This is an administrative operation. Metrics are not reported by default. Reporting can be forced by setting ``include-in-reporting`` to ``true``.

		Please see `the pipeline documentation <https://www.elastic.co/guide/en/elasticsearch/reference/current/handling-failure-in-pipelines.html>`_ for details on handling failures in pipelines.

Add note about mustache template #826

Add note about mustache template #826

Conversation

hub-cap commented Nov 21, 2019

hub-cap commented Nov 21, 2019 • edited Loading

dliappis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielmitterdorfer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hub-cap Dec 16, 2019 • edited Loading

Choose a reason for hiding this comment

danielmitterdorfer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielmitterdorfer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hub-cap commented Nov 21, 2019 •

edited

Loading

hub-cap Dec 16, 2019 •

edited

Loading