Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🪲 Bug Report - loadTextContent called multiple times for the same file causes bloat #254

Closed
KevinRabun opened this issue Jun 6, 2022 · 2 comments · Fixed by #288
Closed
Assignees

Comments

@KevinRabun
Copy link
Contributor

KevinRabun commented Jun 6, 2022

Describe the bug

custom-policy-definitions.bicep calls loadTextContent multiple times for the same file. This causes the content of the file to be put into the output file multiple times which bloats the file significantly

To Reproduce

Steps to reproduce the behaviour:

  1. Build a bicep file that uses custom-policy-definitions.bicep and take not of the size of the output file
  2. Manually replace the redundant calls to loadTextContent in custom-policy-definitions.bicep by using a var instead
  3. Build the new bicep file and take note of the size of the output file

Expected behaviour

custom-policy-definitions.bicep should only load each file it needs one time and store the content in a var to prevent file bloat

Screenshots 📷

Good example

Note the definitionParameters in each of the examples

var publicPaasEndpointsParams = json(loadTextContent('lib/policy_set_definitions/policy_set_definition_es_deny_publicpaasendpoints.parameters.json'))

... other stuff removed for clarity

libSetChildDefinitions: [
      {
        definitionReferenceID: 'ACRDenyPaasPublicIP'
        definitionID: '/providers/Microsoft.Authorization/policyDefinitions/0fdf0491-d080-4575-b627-ad0e843cba0f'
        definitionParameters: publicPaasEndpointsParams.ACRDenyPaasPublicIP.parameters
      }
      {
        definitionReferenceID: 'AFSDenyPaasPublicIP'
        definitionID: '/providers/Microsoft.Authorization/policyDefinitions/21a8cd35-125e-4d13-b82d-2e19b7208bb7'
        definitionParameters: publicPaasEndpointsParams.AFSDenyPaasPublicIP.parameters
      }

Bloated Example (code today)

libSetChildDefinitions: [
      {
        definitionReferenceID: 'ACRDenyPaasPublicIP'
        definitionID: '/providers/Microsoft.Authorization/policyDefinitions/0fdf0491-d080-4575-b627-ad0e843cba0f'
        definitionParameters: json(loadTextContent('lib/policy_set_definitions/policy_set_definition_es_deny_publicpaasendpoints.parameters.json')).ACRDenyPaasPublicIP.parameters
      }
      {
        definitionReferenceID: 'AFSDenyPaasPublicIP'
        definitionID: '/providers/Microsoft.Authorization/policyDefinitions/21a8cd35-125e-4d13-b82d-2e19b7208bb7'
        definitionParameters: json(loadTextContent('lib/policy_set_definitions/policy_set_definition_es_deny_publicpaasendpoints.parameters.json')).AFSDenyPaasPublicIP.parameters
      }

Correlation ID

A correlation ID really helps us investigate your issue further. Please provide one if possible. Details on how to find a correlation ID can be found here: Correlation ID and support

Additional context

Anything else we should know to help us troubleshoot this bug? I have created a separate but highly related issue on the bicep side here Azure/bicep#6970

@KevinRabun KevinRabun added the bug label Jun 6, 2022
@ghost ghost added the Needs: Triage 🔍 Needs triaging by the team label Jun 6, 2022
@jtracey93 jtracey93 self-assigned this Jun 6, 2022
@jtracey93
Copy link
Collaborator

Thanks @KevinRabun for this, once #227 is merged we will start looking at this.

Will also consider converting to using loadJsonContent() that will be available in v0.8 of Bicep (unless it makes it to v0.7 @alex-frankel????)

@alex-frankel
Copy link

0.7 is going to release on 6/10, so it still might make it in. We'll see :)

@jtracey93 jtracey93 removed the Needs: Triage 🔍 Needs triaging by the team label Jun 21, 2022
jtracey93 added a commit that referenced this issue Jul 19, 2022
…es) & `'` apostrophes + Fix #254 (#288)

* update policy to bicep script to handle spaces and hyphens for txt file var outputs

* add support for apostrophe escaping

* adding parameters

* update path and name vars

* add more params

* remove comment

* add param vars for sets

* fixing empty paramters file bug

* ps1 updates

* updates

* linter fixes

* add should process

* surpress false positives

* update azure public policies

* update mc policies

* wiki update

* update codetour

* more docs

* add hyphen support

* updates to indents

* docs

* Update .github/scripts/Invoke-PolicyToBicep-China.ps1

Co-authored-by: Jan Faurskov <22591930+jfaurskov@users.noreply.github.com>

* Update .github/scripts/Invoke-PolicyToBicep.ps1

Co-authored-by: Jan Faurskov <22591930+jfaurskov@users.noreply.github.com>

Co-authored-by: SeSeicht <Sebastian.Seichter@microsoft.com>
Co-authored-by: Jan Faurskov <22591930+jfaurskov@users.noreply.github.com>
@ghost ghost added the Status: Fixed label Jul 19, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants