Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jinja: provide and document jinja-safe key aliases in instance-data (SC-622) #1123

Merged
merged 7 commits into from
Dec 3, 2021
Merged

jinja: provide and document jinja-safe key aliases in instance-data (SC-622) #1123

merged 7 commits into from
Dec 3, 2021

Conversation

blackboxsw
Copy link
Collaborator

@blackboxsw blackboxsw commented Nov 29, 2021 •
edited
Loading

Proposed Commit Message

Allow #cloud-config and cloud-init query to use underscore-delimited
"jinja-safe" key aliases for any instance-data.json keys
containing jinja operator characters.

This provides a means to use Jinja's dot-notation instead of square brackets
and quoting to reference "unsafe" obtain attribute names.

Support for these aliased keys is available to both #cloud-config user-data and
`cloud-init query`.

For example #cloud-config alias access can look like:
  {{ ds.config.user_network_config }}

  - instead of -
  - 
  {{ ds.config["user.network-config"] }}

Additional Context

Test Steps

 tox -e integration-tests tests/integration_tests/datasources/test_lxd_discovery.py

Checklist:

  • My code follows the process laid out in the documentation
  • I have updated or added any unit tests accordingly
  • I have updated or added any documentation accordingly

@blackboxsw blackboxsw changed the title jinja: provide and document jinja-safe key aliases in instance-data jinja: provide and document jinja-safe key aliases in instance-data (SC-622) Nov 29, 2021
@blackboxsw blackboxsw force-pushed the lxd-ds branch 3 times, most recently from fbd3394 to f6cf805 Compare December 1, 2021 19:45
TheRealFalcon
TheRealFalcon requested changes Dec 2, 2021
View reviewed changes
Copy link
Member

@TheRealFalcon TheRealFalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! I left mostly minor comments inline with the only major thing being removing the unused parameter.

cloudinit/handlers/jinja_template.py Show resolved Hide resolved
cloudinit/cmd/query.py Outdated
break
if walked_key_path:
walked_key_path += "."
walked_key_path += key_path_part
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section is hard to follow in a function that's already fairly long. It might help to break it down some or maybe even just add some more comments, but I don't think it needs to block merging.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

118 line long function... I've done worse, but could do a lot better. breaking this down into a couple of separate functions for readability.

tests/unittests/test_builtin_handlers.py Outdated Show resolved Hide resolved
tests/integration_tests/modules/test_jinja_templating.py Outdated Show resolved Hide resolved
TheRealFalcon
TheRealFalcon approved these changes Dec 2, 2021
View reviewed changes
Copy link
Member

@TheRealFalcon TheRealFalcon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave the existing comments up to you, but I think this is good as-is.

@blackboxsw
lxd: provide v1.0 prefix for API version to aid in jinja alias
bb6ec42 
Without a leading non-integer character jinja's lexer gets confused trying to
parse underscore-delimited variable names such as 1_0 as the lexer starts to
interpret an integer and chokes on the underscore. This prevents cloud-init
from providing a simple jinja dot-notation variable alias such as ds.1_0
because the lexer barfs claiminig invalid variable end "_0".

Providing the leading character allows the lexer to interpret the variable
as a string type and properly interpret v1_0 as the whole variable text.
@blackboxsw
jinja: provide and document jinja-safe key aliases in instance-data
d96acf9 
Provide additional aliased keys in instance-data.json which have
replaced any jinja operator characters in key names with underscores.

This provides a simple mechanism for "unsafe" key names to be used
in #cloud-config user-data or cloud-init query commands using
jinja's dot-notation for dict attributes versus the more verbose
square-backet accessor and quoting the unsafe key name.

For example:
  ds.1_0.config.user_data instead of ds["1.0"].config["user-data"]
@blackboxsw
tests: integration updates for aliased keys
871280a
@blackboxsw
lxd: encode API version as _metadata_api_version key
5059629
@blackboxsw
jinja: avoid reporting key aliases in commandline
3892269 
Accept a include_key_aliases param to convert_jinja_instance_data.
When true, add key aliases to the instance-data dict returned.

Hide jinja variable aliases from simple cloud-init query --all
but Use aliases when rendering jinja template files from query --format
command or #cloud-config userdata.
@blackboxsw
flake me
9805a5a
@blackboxsw
query: create smaller functions to aid in readability
14a2af3
@blackboxsw blackboxsw merged commit 0fe96a4 into canonical:main Dec 3, 2021
@ubuntu-server-builder ubuntu-server-builder mentioned this pull request May 12, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TheRealFalcon TheRealFalcon TheRealFalcon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@blackboxsw @TheRealFalcon