Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Planetfm/cost reporting #4507

Merged
merged 27 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
6fab090
looking at what this outputs
robertsweetman Jan 11, 2024
ee83a2b
wrong module name
robertsweetman Jan 11, 2024
448ee53
add athena resources
robertsweetman Jan 11, 2024
bb86fed
add s3 prefix (required but not documented)
robertsweetman Jan 11, 2024
04a9413
change input output format values
robertsweetman Jan 11, 2024
bc42b1e
update report location
robertsweetman Jan 11, 2024
5cdc5c9
change parameters
robertsweetman Jan 11, 2024
6f73a80
deploy some more changes
robertsweetman Jan 11, 2024
f85d283
add column
robertsweetman Jan 11, 2024
cae49c0
resource table definition is incomplete
robertsweetman Jan 11, 2024
1fc6c75
add remaining cur report definitions
robertsweetman Jan 15, 2024
136636a
set force destroy to true for athena resources also
robertsweetman Jan 15, 2024
020c071
add explicit dependency so bucket is created first
robertsweetman Jan 15, 2024
806afaf
make naming generic
robertsweetman Jan 15, 2024
a0332b8
modularise cost usage report
robertsweetman Jan 16, 2024
d505312
update to module but currently deploys to all environments
robertsweetman Jan 17, 2024
8563dcd
fix issue where cur report deploy is all or nothing
robertsweetman Jan 17, 2024
f14a007
test deploy to dev
robertsweetman Jan 17, 2024
c8d3007
change partitions
robertsweetman Jan 18, 2024
c5e7bec
update report range
robertsweetman Jan 18, 2024
c8779df
forgot to put parameters in quotes
robertsweetman Jan 18, 2024
84fd343
update README
robertsweetman Jan 18, 2024
8a9fabe
add comments
robertsweetman Jan 18, 2024
2a3dc9a
address pr comments
robertsweetman Jan 18, 2024
031f0b0
fix variable structure
robertsweetman Jan 18, 2024
c5b3cd4
change variable structure and remove un-needed merges
robertsweetman Jan 18, 2024
a6e23ae
address checkov results
robertsweetman Jan 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion terraform/environments/planetfm/ec2-common.tf
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ resource "aws_ssm_document" "remove_local_users_windows" {
},
)
}

resource "aws_ssm_document" "network-testing-tools" {
name = "network-testing-tools"
document_type = "Command"
Expand Down
5 changes: 5 additions & 0 deletions terraform/environments/planetfm/locals_development.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,10 @@ locals {
# baseline config
development_config = {

# example code for creating a cost usage report in the development environment
#
# baseline_cost_usage_report = {
# create = true
# }
}
}
4 changes: 4 additions & 0 deletions terraform/environments/planetfm/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -132,4 +132,8 @@ module "baseline" {
local.baseline_ssm_parameters,
lookup(local.baseline_environment_config, "baseline_ssm_parameters", {}),
)

# example code for creating a cost usage report - locals_development.tf
cost_usage_report = lookup(local.baseline_environment_config, "baseline_cost_usage_report", { create = false })

}
17 changes: 17 additions & 0 deletions terraform/modules/baseline/cost_usage_report.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
module "cost_usage_report" {

count = lookup(var.cost_usage_report, "create", false) ? 1 : 0

source = "../../modules/cost_usage_report"

providers = {
aws.us-east-1 = aws.us-east-1
aws.bucket-replication = aws
}

application_name = var.environment.application_name
account_number = var.environment.account_id
environment = var.environment.environment
tags = local.tags

}
2 changes: 1 addition & 1 deletion terraform/modules/baseline/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,4 @@ output "ssm_parameters" {
aws_ssm_parameter.fixed,
aws_ssm_parameter.placeholder
)
}
}
10 changes: 10 additions & 0 deletions terraform/modules/baseline/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -1021,3 +1021,13 @@ variable "resource_explorer" {
type = bool
default = false
}

variable "cost_usage_report" {
description = "Enables AWS Cost Usage Report"
type = object({
create = bool
})
default = {
create = false
}
}
45 changes: 45 additions & 0 deletions terraform/modules/cost_usage_report/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
Enable Cost Usage Reports on a 'per-environment' basis.

## Usage

For example:

```
module "cost_usage_report" {

count = lookup(var.cost_usage_report, "create", false) ? 1 : 0

source = "../../modules/cost_usage_report"

providers = {
aws.us-east-1 = aws.us-east-1
aws.bucket-replication = aws
}

application_name = var.environment.application_name
account_number = var.environment.account_id
environment = var.environment.environment
tags = merge(local.tags)

}
```

triggered by the following in the environment configuration file:

```
cost_usage_report = {
create = true
}
```

This needs to be in each environment configuration file that you want a cost usage report for.

Since each environment is in a different account there's no way to create a single cost usage report for all environments.

## Maintenance

The main challenge with this module is whether/when AWS decide to change their report schema and/or the report format.

The module translates the planetfm-cost-usage-report-create-table.sql file into a terraform 'aws_glue_catalog_table' resource. This has been done manually and is not automated so if the schema changes then the terraform resource will need to be updated. This sql table file is created in the S3 bucket when the `aws_cur_report_definition` resource is created.

This also performs a check to make sure that the s3 bucket is writable by the report definition. Any permission failures will be reported in the terraform plan.
Loading
Loading