[question] Include jinja profile template into another

[donlk]

What is your question?

Hi!
I think I had a related question a few years back, but maybe not this one in particular.

I have a jinja2 profile template:

gcc-template.jinja:

[buildenv]
CC={{toolchain_path}}/bin/{{cc_compiler}}
CXX={{toolchain_path}}/bin/{{cxx_compiler}}
LD={{toolchain_path}}/bin/ld
AR={{toolchain_path}}/bin/ar
...

And a concrete one, in a completely different folder only defining toolchain_path:

gcc-concrete.jinja:

{% set toolchain_path = "<absolute_toolchain_path>" %}

I want to include gcc-template.jinja into gcc-concrete.jinja and have toolchain_path expanded. I've read about the include() directive here, but that doesn't work, toolchain_path does not get expanded, no matter where the include() is.
It seems I cannot use {% include <path>/gcc-template.jinja %} (or extends) either, as I get an error:

ERROR: Error while rendering the profile template file '<path>/gcc-concrete.jinja'. Check your Jinja2 syntax: ...

Jinja2 include and extends are completely valid in standard jinja2, I don't know why they give me errors here.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[donlk]

Okay, after a bit of investigation, it is not a limitation of Conan, but of Jinja. It does not natively support absolute paths given to {% include %}.
Can we work around this? Extract the directory, then load it via jinja2.FileSystemLoader?

[memsharded]

Hi @donlk

Have you tried the import approach?

Something like:

 global_conf = textwrap.dedent("""\
        {% include "user_global.conf" %}
        {% import "user_global.conf" as vars %}
        user.mycompany:dist = {{vars.myvar*2}}
        """)
    user_global_conf = textwrap.dedent("""\
        {% set myvar = 42 %}
        user.mycompany:parallel = {{myvar}}
        """)

The absolute path might still be an issue, but it allows to read and use vars from other file.

[donlk]

This would make it much more complex than it needs to be.
My profiles are made to be used together, with a templated and a concrete one. The template's variables should be expanded with the concrete's ones. Isn't this how it was intended to work? I mean it would make sense to support such cases.

[memsharded]

My profiles are made to be used together, with a templated and a concrete one.

It depends. In general I try to avoid inheritance and prefer composition when possible. Not only in code, but inheritance in data-oriented files can also have its challenges.

By the way, the include(otherprofile) syntax is still there, have you considered it?

[donlk]

Theoretically this is still a composition if I use the include directive. I'm compositing the template into my specified one.

By the way, the include(otherprofile) syntax is still there, have you considered it?

I did. It did not expand toolchain_path as I've stated.

[memsharded]

Theoretically this is still a composition if I use the include directive. I'm compositing the template into my specified one.

I am talking about "prefer composition over inheritance" pattern, the include() is mostly inheritance, the included profile being base, and having its values transparently overriden by the current profile being the derived one. Composition is more the approach above using explicit members of the inner contained object like user.mycompany:dist = {{vars.myvar*2}}, otherwise, whatever is imported is not used at all in the final result.

So far I don't see other alternative than using relative paths, but is this also an issue? Why isn't it possible to put the profiles in the same folder, or at least in known relative locations? conan config install will always put things in the right relative locations when installing profiles.

[donlk]

Profile usage gets complicated when every developer has it's own profile containing not just the binary and flag definitions but the hard-coded filesystem paths too. It gets chaotic pretty fast.
Having base profiles commited upstream and the exact filesystem paths for the toolchains downstream solves this problem. Everyone uses the same base profiles, they just define the locations for them separately. This makes profiles trackable, reusable, and maintainable.
However, I realized that profile inclusion here is not the ideal approach, when the question is just one or two additional paths, which can be rendered together pretty easily in python.
Still, I do think that jinja should support absolute paths, it makes sense and would be pretty easy to do in theory.

Thanks! Closing.