jinjyaml

An application-specific tag for Jinja2 templates within PyYAML.

This can be useful if you want to render only specially tagged nodes in the document, rather than treating the entire YAML string as a template.

Usage

Basic Example 1

1. Add Jinja2 template constructor for tag "!j2"

   import yaml
   import jinjyaml as jy
   
   ctor = jy.Constructor()
   yaml.add_constructor("!j2", ctor, yaml.SafeLoader)

2. create YAML file 1.yml, with such contents:

   array: !j2 |
     {% for i in range(n) %}
     - sub{{i}}: {{loop.index}}
     {% endfor %}

3. load and render the YAML file

   with open("1.yml") as fp:
       data = yaml.load(fp, Loader=yaml.SafeLoader)
       # or for the short:
       # data = yaml.safe_load(fp)
   
   jy.extract(data, context={"n": 3}, inplace=True)
   
   print(data)

We'll get:

{"array": [{"sub0": 1}, {"sub1": 2}, {"sub2": 3}]}

Advanced Usage

Include files

Jinja2's include filter function

We have such YAML files:

• sub-1.yml:

  "1.1": one
  "1.2": two

• sub-2.yml:

  "2.1":
    "2.1.1": three
    "2.1.2": four

• main.yml:

  foo: !j2 |
  
    {% filter indent %}
    {% include "sub-1.yml" %}
    {% endfilter %}
  
    {% filter indent %}
    {% include "sub-2.yml" %}
    {% endfilter %}

execute python code:

from pprint import pprint

import jinja2
import jinjyaml as jy
import yaml

env = jinja2.Environment(loader=jinja2.FileSystemLoader("."))

ctor = jy.Constructor()
yaml.add_constructor("!j2", ctor, yaml.SafeLoader)

with open("main.yml") as fp:
    doc = yaml.safe_load(fp)

obj = jy.extract(doc, env)
pprint(obj)

We'll get:

{"foo": {"1.1": "one",
         "1.2": "two",
         "2.1": {"2.1.1": "three", "2.1.2": "four"}}}

pyyaml-include

ℹ️ Note:
Since jinja2's include and indent do not work well with indentation-sensitive languages like Python or YAML, it is not recommended to use these features in complex cases.

For such scenarios, consider using pyyaml-include[]. It provides a PyYAML extension for including other YAML files. We can use this extension instead of Jinja2's include and indent to better maintain the indentation of YAML files.

1. install pyyaml-include:

   pip install pyyaml-include

2. add both pyyaml-include and jinjyaml's constructor:

   import yaml
   import jinjyaml as jy
   import pyyaml_include
   
   yaml.add_constructor("!j2", jy.Constructor)
   yaml.add_constructor("!inc", pyyaml_include.Constructor(base_dir="path_to_you_dir"))

3. Assume that we have YAML files same to previous example, the main.yml can be modified as below:

   foo: !j2 |
     {% for i in range(n) %}
     - !inc sub-{{loop.index}}.yml
     {% endfor %}

4. include and load other YAML files:

   Assume that we have YAML files same to previous example:

   with open("main.yml") as fp:
       doc = yaml.safe_load(fp)
   
   obj = jy.extract(doc, env)
   pprint(obj)

Then we'll get:

{
  "foo": [
    {"1.1": "one", "1.2": "two" },
    {"2.1": {"2.1.1": "three", "2.1.2": "four"}}
  ]
}

In this situation, it is not necessary to use jinja2.Environment and jinja2.FileSystemLoader to render the template, nor is it necessary to use the indent filter in the template. This is because pyyaml-include has already parsed the file into an object.

❇️ Conclusions:
We can use jinja2's include and indent to include other YAML files literally, or to use pyyaml-include to include other YAML files as already-parsed objects.