Terraform Transform Data Sources

Overview

This provider defines a set of data sources providing data transformation primitives missing from core Terraform.

Data Sources

This plugin defines following data sources:

• transform_group_by_value - group map keys by map values

Reference

transform_group_by_value

Arguments

The following arguments are supported:

• input - (Required) A map with both keys and values as strings
• extract - (Required) A key (one of values in the input) which value should be returned in output after the grouping. Because of Terraform limitation it's not possible to return the full grouped map (which would be a map of strings to lists).

Attributes

The following attribute is exported:

• output - A list of keys from input that contain extract value. Sorted in lexicographical order.

Installation

Terraform automatically discovers the Providers when it parses configuration files. This only occurs when the init command is executed.

Currently Terraform is able to automatically download only official plugins distributed by HashiCorp.

All other plugins should be installed manually.

Terraform will search for matching Providers via a Discovery process, including the current local directory.

This means that the plugin should either be placed into current working directory where Terraform will be executed from or it can be installed system-wide.

Usage

main.tf

locals {
  input = {
    "aaa/bbb/111" = "val1"
    "aaa/ccc/111" = "val1"
    "aaa/ddd/222" = "val2"
  }
}

data "transform_group_by_value" "data" {
  input = local.input
  extract = "val1"
}

data "transform_glob_map" "include" {
  input = local.input
  pattern = "aaa/*/111"
}

data "transform_glob_map" "exclude" {
  input = local.input
  pattern = "aaa/*/111"
  exclude = true
}

data "transform_glob_map" "include_w_sep" {
  input = local.input
  pattern = "aaa/*"
  separator = "/"
}

data "transform_glob_map" "include_wo_sep" {
  input = local.input
  pattern = "aaa/*"
}

output "result" {
  value = {
    grouped = data.transform_group_by_value.data.items

    glob_include        = data.transform_glob_map.include.output
    glob_exclude        = data.transform_glob_map.exclude.output

    glob_include_w_sep  = data.transform_glob_map.include_w_sep.output
    glob_include_wo_sep = data.transform_glob_map.include_wo_sep.output
  }
}

Download

wget "https://github.com/ashald/terraform-provider-transform/releases/download/v1.2.0/terraform-provider-transform_v1.2.0-$(uname -s | tr '[:upper:]' '[:lower:]')-amd64"
chmod +x ./terraform-provider-transform*

Init

$ ls -1
  main.tf
  terraform-provider-transform_v1.2.0-linux-amd64

$ terraform init
  
  Initializing provider plugins...
  
  The following providers do not have any version constraints in configuration,
  so the latest version was installed.
  
  To prevent automatic upgrades to new major versions that may contain breaking
  changes, it is recommended to add version = "..." constraints to the
  corresponding provider blocks in configuration, with the constraint strings
  suggested below.
  
  * provider.transform: version = "~> 1.1"
  
  Terraform has been successfully initialized!
  
  You may now begin working with Terraform. Try running "terraform plan" to see
  any changes that are required for your infrastructure. All Terraform commands
  should now work.
  
  If you ever set or change modules or backend configuration for Terraform,
  rerun this command to reinitialize your working directory. If you forget, other
  commands will detect it and remind you to do so if necessary.

Apply

$ terraform apply
  data.transform_glob_map.include: Refreshing state...
  data.transform_glob_map.include_wo_sep: Refreshing state...
  data.transform_glob_map.include_w_sep: Refreshing state...
  data.transform_glob_map.exclude: Refreshing state...
  data.transform_group_by_value.data: Refreshing state...
  
  Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
  
  Outputs:
  
  result = {
    glob_exclude = map[aaa/ddd/222:val2]
    glob_include = map[aaa/ccc/111:val1 aaa/bbb/111:val1]
    glob_include_w_sep = map[]
    glob_include_wo_sep = map[aaa/bbb/111:val1 aaa/ddd/222:val2 aaa/ccc/111:val1]
    grouped = [aaa/ccc/111 aaa/bbb/111]
  }

Development

Go

In order to work on the provider, Go should be installed first (version 1.11+ is required). goenv and gvm are great utilities that can help a lot with that and simplify setup tremendously. GOPATH should be setup correctly and $GOPATH/bin should be added $PATH.

This plugin uses Go modules available starting from Go 1.11 and therefore it should not be checked out within $GOPATH tree.

Source Code

Source code can be retrieved with git

$ git clone git@github.com:ashald/terraform-provider-transform.git .

Dependencies

This project uses go mod to manage its dependencies and it's expected that all dependencies are vendored so that it's buildable without internet access. When adding/removing a dependency run following commands:

$ go mod vendor
$ go mod tidy

Test

$ make test
  GOPROXY="off" GOFLAGS="-mod=vendor" go test -v ./...
  ?   	github.com/ashald/terraform-provider-transform	[no test files]
  === RUN   TestGlobMapDataSource
  --- PASS: TestGlobMapDataSource (0.04s)
  === RUN   TestGroupByValueDataSource
  --- PASS: TestGroupByValueDataSource (0.02s)
  === RUN   TestProvider
  --- PASS: TestProvider (0.00s)
  PASS
  ok  	github.com/ashald/terraform-provider-transform/transform	0.075s
  GOPROXY="off" GOFLAGS="-mod=vendor" go vet ./...

Build

In order to build plugin for the current platform use [GNU]make:

$ make build
  GOPROXY="off" GOFLAGS="-mod=vendor" go build -o terraform-provider-transform_v1.2.0

it will build provider from sources and put it into current working directory.

If Terraform was installed (as a binary) or via go get -u github.com/hashicorp/terraform it'll pick up the plugin if executed against a configuration in the same directory.

Release

In order to prepare provider binaries for all platforms:

$ make release
  GOPROXY="off" GOFLAGS="-mod=vendor" GOOS=darwin GOARCH=amd64 go build -o './release/terraform-provider-transform_v1.2.0-darwin-amd64'
  GOPROXY="off" GOFLAGS="-mod=vendor" GOOS=linux GOARCH=amd64 go build -o './release/terraform-provider-transform_v1.2.0-linux-amd64'

Versioning

This project follow Semantic Versioning

Changelog

This project follows keep a changelog guidelines for changelog.

Contributors

Please see CONTRIBUTORS.md

License

This is free and unencumbered software released into the public domain. See LICENSE