jinja includes in templates do not work as expected

[pille]

probably because templates get parsed on the minion, when using the jinja {% include "file.inc" %}
i get an template error, because the include is not found.

as a workaround you need to deploy that file.inc as a requirement.

i'm not sure if this is expected behaviour, or an issue. but currently i'm tending to treat it as a bug, since it completely fails when using test=True.

[basepi]

Hrm, I thought we had already solved this problem.

@whiteinge How does this work when you're doing an import from a JSON file or similar?

[whiteinge]

I believe Jinja includes are processed via Salt's file server same as
.sls files are. I don't know the exact mechanism for that offhand (I
think we have a custom Jinja template subclass that handles that
behind the scenes but I can't readily look it up in the code until
next week (sorry my schedule is awful)). Perhaps this is a recent
regression or perhaps something specific to test mode? @pille what
version of Salt (master & minion) and can you provide a minimal
example to reproduce the issue?

[pille]

master: 2014.1.7
minions: 2014.1.x |x=[0,1,10]

/srv/salt/state/test/15863/another.txt:

this is from another.txt

/srv/salt/state/test/15863/file.txt:

this is file.txt, that should include another.txt when rendered.
{% include "another.txt" %}

/srv/salt/state/test/15863/init.sls:

# https://github.com/saltstack/salt/issues/15863#issuecomment-56568345

/tmp/file.txt:
  file.managed:
    - source: salt://test/15863/file.txt
    - template: jinja

pille@salt-master ~ % sudo salt -v 'tmp4*' state.sls test.15863                                                                                                                                                                                                     
Executing job with jid 20140926093656911151
-------------------------------------------
tmp4:
----------
          ID: /tmp/file.txt
    Function: file.managed
      Result: False
     Comment: Unable to manage file: Jinja error: another.txt
              Traceback (most recent call last):
                File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 261, in render_jinja_tmpl
                  output = jinja_env.from_string(tmplstr).render(**unicode_context)
                File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 894, in render
                  return self.environment.handle_exception(exc_info, True)
                File "", line 2, in top-level template code
                File "/usr/lib/pymodules/python2.7/salt/utils/jinja.py", line 122, in get_source
                  raise TemplateNotFound(template)
              TemplateNotFound: another.txt
              
              ; line 2
              
              ---
              this is file.txt, that should include another.txt when rendered.
              {% include "another.txt" %}    <======================
              
              Traceback (most recent call last):
                File "/usr/lib/pymodules/python2.7/salt/utils/templates.py", line 261, in render_jinja_tmpl
                  output = jinja_env.from_string(tmplstr).render(**unicode_context)
                File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 894, in render
              [...]
              ---
     Changes:   
Summary
------------
Succeeded: 0
Failed:    1
------------
Total:     1

[basepi]

I bet this is related to #16106

[cachedout]

If it is related to that issue, it has been resolved. @pille care to test?

[pille]

could reproduce the issue using minion from git develop (2014.7.0rc2-2427).

[basepi]

Hmm, that should include the fix. I guess maybe this one is not related. Thanks for the update.

[whiteinge]

@pille Sorry for the slow reply. The {% include %} is not a relative path which is why it can't find the other file. It needs an absolute path from the file_roots setting. Try {% include "state/test/15863/another.txt" %} instead.

[steverweber]

this might work if you want to move the file around.

{% include slspath + "/another.txt" %}

[whiteinge]

Good suggestion. That addition is perfect for this.

[pille]

@whiteinge: i already tried that. it just throws the same error.
@steverweber: Unable to manage file: Jinja variable 'slspath' is undefined

[steverweber]

your running a version of salt without the fix.

salt \* cmd.run 'salt-minion --version'

you will need to use the full path or update.

[whiteinge]

@pille the paths in your base-case above are incorrect. After editing them this works correctly for me. Also, as @steverweber suggested, double-check the Salt version on your minion as well as your master.

# tree -FC --charset=ascii /srv/salt/state/
/srv/salt/state/
`-- test/
    `-- 15863/
        |-- another.txt
        |-- file.txt
        `-- init.sls

# cat /srv/salt/state/test/15863/init.sls
/tmp/file.txt:
  file.managed:
    - source: salt://state/test/15863/file.txt
    - template: jinja

# cat /srv/salt/state/test/15863/file.txt
this is file.txt, that should include another.txt when rendered.
{% include "state/test/15863/another.txt" %}

# salt 'sethtest*' state.sls state.test.15863
sethtest01:
----------
          ID: /tmp/file.txt
    Function: file.managed
      Result: True
     Comment: File /tmp/file.txt updated
     Started: 18:41:35.555352
     Duration: 525.422 ms
     Changes:
              ----------
              diff:
                  New file
              mode:
                  0644

Summary
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1

# salt 'sethtest*' cmd.run 'cat /tmp/file.txt'
sethtest01:
    this is file.txt, that should include another.txt when rendered.
    this is from another.txt

[pille]

my minion is git develop, but my master is running an old release. since i can't upgrade that for testing on-the-fly and @whiteinge provided info that it works, i'm closing this. thanks!