Deck support placeholder replace

[cello86]

Hi,
is it possible to apply a sort of placeholder substitution into deck sync operation to support the multi environment configuration via single branch of SCM like git?

Regards,
Marcello

[antodr]

Hello @cello86 ,
I usually do these substitutions during the build step

[cello86]

HI @Zaragon-AQ,
Actually we have implemented a pipeline that downloads the YAML file config from our SCM and we replace the placeholders with replace ansible module.

This work fine but we would understand if we have to adjust the files before the release via deck sync or there are some other quick steps like env variables to perform the same operation.

Thanks,
Marcello

[dowster]

Seconding the approach of doing this in the build step. We use YTT https://get-ytt.io/ to do string replacement along with replacement of entire YAML structures based off environment specific properties files that get fed in for each of our environments.

[hbagdi]

Up until this point, I've been against adding such a feature into decK because templating is a whole another problem that decK will never be able to solve in a way that makes even 80% of users happy. And hence, I've been advising all users to use solutions like YTT or j2, bring-your-own-templating-engine.

Having said that, there are more and more users who are requesting this feature and I'm now reconsider my position on this. Adding some basic templating into decK could help many users. But before we do so, we need some examples from the community on how people are templating their YAMLs today.

Would all users in this thread mind sharing their solutions with some code and samples? That should help us understand the expectations and come up with a design that works well for everyone.

Cheers!

[cello86]

Hi @hbagdi,
in our case we don't use an advanced templating, but we are only replace some placeholders as reported below:

_format_version: "1.1"
  | _workspace: $WORKSPACE
  | services:
  | - connect_timeout: 60000
  | host: $KONG_LOCALHOST_BACKEND
  | name: kong-localinstance
  | path: /
  | port: 80
  | protocol: http
  | read_timeout: 60000
  | retries: 5
  | write_timeout: 60000
  | routes:
  | - hosts:
  | - $KONG_LOCALHOST_BACKEND
  | name: kong-internal-probe
  | methods:
  | - GET
  | paths:
  | - /kong-probe.html
  | preserve_host: false
  | protocols:
  | - http
  | regex_priority: 0
  | strip_path: false
  | plugins:
  | - name: request-termination
  | config:
  | body: null
  | content_type: null
  | message: Kong localhost probe
  | status_code: 200
  | enabled: true
  run_on: first

Marcello

[cwurm]

To me, the most basic requirement would to be to avoid putting secrets and passwords into deck files. Unfortunately, I have seen it happen several times that AWS secrets were exposed that were used to configure the Lambda plugin.

Maybe a straightforward implementation would be to use bash-style variable replacement, e.g.:

plugins:
- name: aws-lambda
  config:
    aws_secret: ${AWS_SECRET}

decK already uses Viper which can read environment variables, config files, and even values from etcd and Consul - so it would mostly be about inserting those in the right places when reading the state.

[hbagdi]

Thanks for the input here. I've something concrete in my mind to get to the feature state. Here is the summary:

• The templating language will look like ${{ env "AWS_SECRET" }}. We will expand the template functions in future to include injection of values from other sources.
• deck render or a similar named CLI command which can render a state file based on the inputs (env vars, reading from vault etc) you give it

[hbagdi]

Please take a look at this PR for this feature: #286

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.