You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The instance type in dev/preprod was too slow to generate embeddings in any reasonable time for testing. Hopefully this helps, but if not we can drop it back down and test in production initially.
…/preprod
The instance type in dev/preprod was too slow to generate embeddings in any reasonable time for testing. Hopefully this helps, but if not we can drop it back down and test in production initially.
Previously:
* ml.t2.medium: 2 vCPU x 4GB - $0.062/hr
Now:
* ml.m6g.large: 2 vCPU x 8GB - $0.10656/hr
Trivy will check the following folders:
terraform/environments/analytical-platform-compute
Running Trivy in terraform/environments/analytical-platform-compute
2025-01-17T09:53:09Z INFO [vulndb] Need to update DB
2025-01-17T09:53:09Z INFO [vulndb] Downloading vulnerability DB...
2025-01-17T09:53:09Z INFO [vulndb] Downloading artifact... repo="public.ecr.aws/aquasecurity/trivy-db:2"
2025-01-17T09:53:11Z INFO [vulndb] Artifact successfully downloaded repo="public.ecr.aws/aquasecurity/trivy-db:2"
2025-01-17T09:53:11Z INFO [vuln] Vulnerability scanning is enabled
2025-01-17T09:53:11Z INFO [misconfig] Misconfiguration scanning is enabled
2025-01-17T09:53:11Z INFO [misconfig] Need to update the built-in checks
2025-01-17T09:53:11Z INFO [misconfig] Downloading the built-in checks...
160.80 KiB / 160.80 KiB [---------------------------------------------------------] 100.00% ? p/s 0s2025-01-17T09:53:12Z INFO [secret] Secret scanning is enabled
2025-01-17T09:53:12Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2025-01-17T09:53:12Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.57/docs/scanner/secret#recommendation for faster secret detection
2025-01-17T09:53:13Z INFO [terraform scanner] Scanning root module file_path="."
2025-01-17T09:53:13Z WARN [terraform parser] Variable values was not found in the environment or variable files. Evaluating may not work correctly. module="root" variables="networking"
2025-01-17T09:53:13Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.transit_gateway_routes" value="cty.NilVal"
2025-01-17T09:53:24Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.eks.aws_ec2_tag.cluster_primary_security_group" value="cty.NilVal"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.kms.data.aws_iam_policy_document.this[0]" err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.kms.data.aws_iam_policy_document.this[0]" err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_iam_role_policy_attachment.this" value="cty.NilVal"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.eks.module.eks_managed_node_group["general"].aws_iam_role_policy_attachment.this" value="cty.NilVal"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["general"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["general"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks_cluster_logs_kms.data.aws_iam_policy_document.this[0]" err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement.content.dynamic.condition block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks_cluster_logs_kms.data.aws_iam_policy_document.this[0]" err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement.content.dynamic.condition block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_iam_role_policy_attachment.this" value="cty.NilVal"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["airflow-high-memory"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="module.eks.module.eks_managed_node_group["general"].aws_iam_role_policy_attachment.this" value="cty.NilVal"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["general"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:25Z ERROR [terraform evaluator] Failed to expand dynamic block. block="module.eks.module.eks_managed_node_group["general"].aws_launch_template.this[0]" err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"
2025-01-17T09:53:26Z INFO [terraform executor] Ignore finding rule="aws-ec2-no-public-egress-sgr" range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/node_groups.tf:247"
2025-01-17T09:53:26Z INFO [terraform executor] Ignore finding rule="aws-eks-no-public-cluster-access-to-cidr" range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/main.tf:70"
2025-01-17T09:53:26Z INFO [terraform executor] Ignore finding rule="aws-eks-no-public-cluster-access" range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/main.tf:69"
2025-01-17T09:53:28Z INFO Number of language-specific files num=0
2025-01-17T09:53:28Z INFO Detected config files num=14
trivy_exitcode=0
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
terraform/environments/analytical-platform-compute
*****************************
Running Checkov in terraform/environments/analytical-platform-compute
Excluding the following checks: CKV_GIT_1,CKV_AWS_126,CKV2_AWS_38,CKV2_AWS_39
2025-01-17 09:53:31,018 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/iam/aws//modules/iam-policy:5.52.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,018 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks:5.52.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,018 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/iam/aws//modules/iam-github-oidc-role:5.52.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,018 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/iam/aws//modules/iam-assumable-role:5.52.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,018 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/cloudwatch/aws//modules/log-group:5.7.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,019 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/kms/aws:3.1.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,019 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/eks-pod-identity/aws:1.9.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,020 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/s3-bucket/aws:4.3.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,020 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/rds/aws:6.10.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,020 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/security-group/aws:5.2.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module ministryofjustice/observability-platform-tenant/aws:1.2.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/vpc/aws//modules/vpc-endpoints:5.17.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/secrets-manager/aws:1.3.1 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/eks/aws:20.31.6 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/eks/aws//modules/karpenter:20.31.6 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/route53/aws//modules/zones:4.1.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,021 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/vpc/aws:5.17.0 (for external modules, the --download-external-modules flag is required)
2025-01-17 09:53:31,022 [MainThread ] [WARNI] Failed to download module terraform-aws-modules/managed-service-prometheus/aws:3.0.0 (for external modules, the --download-external-modules flag is required)
terraform scan results:
Passed checks: 169, Failed checks: 0, Skipped checks: 155
checkov_exitcode=0
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version:0.9.1)
tflint will check the following folders:
terraform/environments/analytical-platform-compute
*****************************
Running tflint in terraform/environments/analytical-platform-compute
Excluding the following checks: terraform_unused_declarations
tflint_exitcode=0
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
terraform/environments/analytical-platform-compute
*****************************
Running Trivy in terraform/environments/analytical-platform-compute
2025-01-17T09:53:09Z INFO [vulndb] Need to update DB
2025-01-17T09:53:09Z INFO [vulndb] Downloading vulnerability DB...2025-01-17T09:53:09Z INFO [vulndb] Downloading artifact...repo="public.ecr.aws/aquasecurity/trivy-db:2"2025-01-17T09:53:11Z INFO [vulndb] Artifact successfully downloaded repo="public.ecr.aws/aquasecurity/trivy-db:2"2025-01-17T09:53:11Z INFO [vuln] Vulnerability scanning is enabled
2025-01-17T09:53:11Z INFO [misconfig] Misconfiguration scanning is enabled
2025-01-17T09:53:11Z INFO [misconfig] Need to update the built-in checks
2025-01-17T09:53:11Z INFO [misconfig] Downloading the built-in checks...160.80 KiB /160.80 KiB [---------------------------------------------------------] 100.00%? p/s 0s2025-01-17T09:53:12Z INFO [secret] Secret scanning is enabled
2025-01-17T09:53:12Z INFO [secret] If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2025-01-17T09:53:12Z INFO [secret] Please see also https://aquasecurity.github.io/trivy/v0.57/docs/scanner/secret#recommendation for faster secret detection2025-01-17T09:53:13Z INFO [terraformscanner] Scanning root module file_path="."2025-01-17T09:53:13Z WARN [terraformparser] Variable values was not found in the environment or variable files. Evaluating may not work correctly.module="root"variables="networking"2025-01-17T09:53:13Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.transit_gateway_routes"value="cty.NilVal"2025-01-17T09:53:24Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.eks.aws_ec2_tag.cluster_primary_security_group"value="cty.NilVal"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.kms.data.aws_iam_policy_document.this[0]"err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.kms.data.aws_iam_policy_document.this[0]"err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_iam_role_policy_attachment.this"value="cty.NilVal"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.eks.module.eks_managed_node_group[\"general\"].aws_iam_role_policy_attachment.this"value="cty.NilVal"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"general\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"general\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks_cluster_logs_kms.data.aws_iam_policy_document.this[0]"err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement.content.dynamic.condition block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks_cluster_logs_kms.data.aws_iam_policy_document.this[0]"err="1 error occurred:\n\t* invalid for-each in data.aws_iam_policy_document.this[0].dynamic.statement.content.dynamic.condition block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_iam_role_policy_attachment.this"value="cty.NilVal"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"airflow-high-memory\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.block="module.eks.module.eks_managed_node_group[\"general\"].aws_iam_role_policy_attachment.this"value="cty.NilVal"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"general\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:25Z ERROR [terraformevaluator] Failed to expand dynamic block.block="module.eks.module.eks_managed_node_group[\"general\"].aws_launch_template.this[0]"err="1 error occurred:\n\t* invalid for-each in aws_launch_template.this[0].dynamic.block_device_mappings block: cannot use a cty.NilVal value in for_each. An iterable collection is required\n\n"2025-01-17T09:53:26Z INFO [terraformexecutor] Ignore finding rule="aws-ec2-no-public-egress-sgr"range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/node_groups.tf:247"2025-01-17T09:53:26Z INFO [terraformexecutor] Ignore finding rule="aws-eks-no-public-cluster-access-to-cidr"range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/main.tf:70"2025-01-17T09:53:26Z INFO [terraformexecutor] Ignore finding rule="aws-eks-no-public-cluster-access"range="git::https:/github.com/terraform-aws-modules/terraform-aws-eks?ref=a713f6f464eb579a39918f60f130a5fbb77a6b30/main.tf:69"2025-01-17T09:53:28Z INFO Number of language-specific files num=02025-01-17T09:53:28Z INFO Detected config files num=14trivy_exitcode=0
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
bot
added
the
environments-repository
marcus-bcl
temporarily deployed
to
analytical-platform-compute-development
…/preprod
The instance type in dev/preprod was too slow to generate embeddings in any reasonable time for testing. Hopefully this helps, but if not we can drop it back down and test in production initially.
Previously:
Now: