provider/aws: API Gateway resources

[nicolai86]

Hello,

I implemented many new resources for the AWS API Gateway.

The new resources are:

• aws_api_gateway_rest_api - manage REST APIs
• aws_api_gateway_resource - manage API resources
• aws_api_gateway_method - manage resource methods
• aws_api_gateway_method_response - manage resource method responses
• aws_api_gateway_integration - manage resource method integration
• aws_api_gateway_integration_response - manage resource integration responses
• aws_api_gateway_model - manage resource models
• aws_api_gateway_deployment - manage REST API deployments
• aws_api_gateway_api_key - manage REST API keys

Example usage as of now:

provider "aws" {
  region = "eu-west-1"
}

resource "aws_api_gateway" "MyDemoAPI" {
  name = "MyDemoAPI"
  description = "This is my API for demonstration purposes"
}

resource "aws_api_gateway_resource" "MyDemoResource" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  parent_resource_id = "${aws_api_gateway.MyDemoAPI.root_resource_id}"
  path_part = "mydemoresource"
}

resource "aws_api_gateway_model" "user" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  name = "user"
  description = "a user schema"
  content_type = "application/xml"
  schema = <<EOF
{
  "type": "object"
}
EOF
}

resource "aws_api_gateway_method" "users-get" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
  http_method = "GET"
  authorization = "NONE"

  request_models = {
    "application/json" = "Error"
  }
}

resource "aws_api_gateway_integration" "users-get" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
  http_method = "${aws_api_gateway_method.users-get.http_method}"

  type = "MOCK"
}

resource "aws_api_gateway_method_response" "400" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
  http_method = "${aws_api_gateway_method.users-get.http_method}"
  status_code = "400"

  response_models = {
    "application/json" = "Error"
  }
}

resource "aws_api_gateway_method_response" "200" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
  http_method = "${aws_api_gateway_method.users-get.http_method}"
  status_code = "200"

  response_models = {
    "application/xml" = "${aws_api_gateway_model.user.name}"
  }
}

resource "aws_api_gateway_integration_response" "users-get" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  resource_id = "${aws_api_gateway_resource.MyDemoResource.id}"
  http_method = "${aws_api_gateway_method.users-get.http_method}"
  status_code = "${aws_api_gateway_method_response.400.status_code}"

  response_templates = {
    "application/json" = ""
    "application/xml" = "#set($inputRoot = $input.path('$'))\n{ }"
  }
}

resource "aws_api_gateway_api_key" "access" {
  name = "foo"
  description = "foo"

  stage_key {
    api_id = "${aws_api_gateway.MyDemoAPI.id}"
    stage_name = "${aws_api_gateway_deployment.test.stage_name}"
  }
}

resource "aws_api_gateway_deployment" "test" {
  api_id = "${aws_api_gateway.MyDemoAPI.id}"
  stage_name = "test"
  description = "This is a test"

  variables = {
    "a" = "2"
  }
}

Feedback and critic are very welcome.

Acceptance Test Status:

=== RUN   TestAccAWSAPIGatewayResource_basic
--- PASS: TestAccAWSAPIGatewayResource_basic (12.72s)
=== RUN   TestAccAWSAPIGateway_basic
--- PASS: TestAccAWSAPIGateway_basic (9.81s)
PASS
ok      github.com/hashicorp/terraform/builtin/providers/aws    22.541s

[nicolai86]

@radeksimko I still need to add acceptance tests for both aws_api_gateway_method and aws_api_gateway_model - what is missing to land this PR otherwise?

[nicolai86]

There are still a some things left to do:

• write documentation
• support deployments
• add more tests
• cleanup code

but the overall structure has settled now.

[hlandau]

What's the status of this?

[nicolai86]

@hlandau it works, I need to add some more tests and refactor it a little. Would love a review though, b/c it's my first terraform PR with lots of changes.

[ryansroberts]

@nicolai86 I have done some additional work with this PR to support integration responses, which changes the syntax a little. Also it's quite difficult to create a dependency from gateway to deployment that works right now as depends_on doesn't seem to interpolate.

resource "aws_api_gateway_method" "method" {
  api_id = "${aws_api_gateway.apigateway.id}"
  resource_id = "${element(aws_api_gateway_resource.resource.*.id, count.index)}"
  http_method = "${lookup(var.methods,count.index)}"
  authorization = "AWS_IAM"
  request = {
    models = {
      "application/json" = "${element(aws_api_gateway_model.request.*.name, count.index)}"
    }
    templates = {
      "application/json" = "${template_file.map_requestparams.rendered}"
    }
  }
  integration = {
    type = "AWS"
    http_method = "POST"
    credentials = "${aws_iam_role.iam_for_gateway.arn}"
    uri = "arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${element(aws_lambda_function.lambda.*.arn, count.index)}/invocations"
  }
  response = {
    status_code = "200"
    models = {
      "application/json" = "${element(aws_api_gateway_model.response.*.name, count.index)}"
    }
    headers = {
      "Access-Control-Allow-Origin" = "'*'"
    }

  }
  count = "${var.endpoints}"
}

[ryansroberts]

Is the complexity of properly supporting all the PATCH requests needed to update a method in place worth it? Any api method is likely to be near the leaves of the dependency tree

[nicolai86]

@RyanRoberts you probably based your work off of an earlier version of the code - I've split thinks up into multiple resources since then. Take a look at the PR top, there's the current syntax noted.

This also reduced most of the issues with PATCHing. For an initial version I'd even remove this entirely and just recreate everythin when attributes change, just to keep the diff smaller

[ryansroberts]

Ah, yes. This looks much better.

[ajlanghorn]

This is exactly the PR I've been looking for! Hopefully it can get merged and out in to a release soon, as I'd very much like to use these new resources to configure API Gateway for a stack I'm helping to manage.

[brycefisher]

How would I integrate this with a Lambda function?

[ryansroberts]

@brycefisher You need to use the AWS integration type, construct the lambda invocation URI (arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${functionname}/invocations ) and post to it, unsure if it's provided by the lambda resource itself. You will also likely end up using request templates to pass configuration / context
I have a fork of this PR.. so syntax is different

resource "template_file" "map_requestparams" {
  template = <<EOF
{
    "identity" : "$context.identity",
    "data" : $input.json('$'),
     "config" : {
      "db" : {
        "someTable" : "${aws_dynamodb_table.someTable.id}",
      }
    }
}
EOF
}


resource "aws_api_gateway_method" "method" {
  api_id = "${aws_api_gateway.apigateway.id}"
  resource_id = "${element(aws_api_gateway_resource.resource.*.id, count.index)}"
  http_method = "${lookup(var.methods,count.index)}"
  authorization = "${lookup(var.authorization,count.index)}"
  request = {
    models = {
      "application/json" = "${element(aws_api_gateway_model.request.*.name, count.index)}"
    }
    templates = {
      "application/json" = "${template_file.map_requestparams.rendered}"
    }
    headers = {
      "X-Presciense-Router-Token" = false
    }
  }
  integration = {
    type = "AWS"
    http_method = "POST"
    credentials = "${aws_iam_role.iam_for_gateway.arn}"
    uri = "arn:aws:apigateway:${var.region}:lambda:path/2015-03-31/functions/${element(aws_lambda_function.lambda.*.arn, count.index)}/invocations"
  }
  response = {
    status_code = "200"
    models = {
      "application/json" = "${element(aws_api_gateway_model.response.*.name, count.index)}"
    }
    headers = {
      "Access-Control-Allow-Origin" = "'*'"
    }

  }
  count = "${var.endpoints}"
}

resource "aws_lambda_function" "lambda" {
  filename = "../dist/${lookup(var.lambdanames,count.index)}.js.zip"
  function_name = "${var.tag}${lookup(var.lambdanames,count.index)}"
  role = "lambda_dynamo"
  handler = "${lookup(var.lambdanames,count.index)}.handler"
  count = "${var.endpoints}"
}

[nicolai86]

@brycefisher @ryansroberts is correct, you'd need to construct an url for that. However, instead of ${functionname} you should use ARN of the lambda: arn:aws:apigateway:${aws_region}:lambda:path/2015-03-31/functions/${aws_lambda_function.my-lambda.arn}/invocations

[ryansroberts]

I should possibly put a PR in to export a computed invocation uri from the
lambda resource?
On Wed, 3 Feb 2016 at 18:15, Raphael Randschau notifications@github.com
wrote:

@ryansroberts https://github.com/ryansroberts is correct, you'd need to
construct an url for that. However, instead of ${functionname} you should
use ARN of the lambda:
arn:aws:apigateway:${aws_region}:lambda:path/2015-03-31/functions/${aws_lambda_function.my-lambda.arn}/invocations

—
Reply to this email directly or view it on GitHub
#4295 (comment).

[ryansroberts]

There are upcoming changes to the API Gateway ..API that make both of our approaches pretty pointless. We should have the ability to create a much cleaner and simpler resource for this next month

[nicolai86]

@ryansroberts do you have a commit to check out?

[shikasta-kashti]

@ryansroberts what are the upcoming changes?

[fuwjax]

Even if there are potentially changes in the pipeline, a working api gateway resource today and a refactoring (maybe) in a month is far more pragmatic than waiting for "should." Is there any reason to not pull these resources?

[ryansroberts]

My branch works for my use case, but It's going to be quite a large rework. Depends on your use case / how soon you need, this is currently quite hard to manage due to implicit dependencies. A closer intermediate solution would probably be to use the AWS labs swagger importer with https://www.terraform.io/docs/providers/template/r/file.html

[brycefisher]

👍 for the aws swagger importer + the terraform template. The web console can export a swagger definition to help you understand how it works (the documentation on how to use swagger with aws is a bit lacking IMHO). Not being a Java / maven expert, I simply used the Dockerfile provided by AWS Labs and it worked like a charm.

I agree with @fuwjax about the value of having api gateway support directly in terraform sooner rather than later. I quickly scanned through the files on this PR and I didn't see the implicit dependencies, only the aws-go-sdk. What are the implicit dependencies you speak of?

[ryansroberts]

By implicit dependencies I meant between the method / resource / deployment resources. Currently you need to use interpolation to ensure things are constructed in the right order.

[radeksimko]

Is there any reason why can't we call this resource aws_api_gateway_rest_api and make it match the AWS API naming convention?

[nicolai86]

I found aws_api_gateway_rest_api quite long. But I've changed it anyhow.

[radeksimko]

The link title here should be aws_api_gateway_deployment (including the aws_).

[nicolai86]

corrected!

[radeksimko]

I think it's very close to land in master, good job @nicolai86 and thank you for being so responsive! 👍

We also usually keep all validation functions in builtin/providers/aws/validators.go and have tests for most in builtin/providers/aws/validators_test.go, but since I marked that as "nice to have", I won't insist on fixing this. I can take care of it as part of the merging process in a separate commit.

One last thing, that would be very helpful before I merge this - I may be asking for too much (depending on how skilled you're in git 😃 ) - but would you mind squashing those 40+ commits into either single one or a few logical ones (e.g. per resource or three - docs/code/tests or any other logical separation)?

https://help.github.com/articles/about-git-rebase/

If you don't feel confident enough about rebasing, I'll be happy to take care of that - just let me know.

[nicolai86]

@radeksimko I've moved the validation function, added a very tiny test & clean up the commits according to your suggestion.

[radeksimko]

I just rescanned it thoroughly again, reran all tests, and it looks ok. There is a few nitpicks, but we can deal with those in a separate PR.

🚢

[nicolai86]

@radeksimko thanks a lot for your review! Great help for my very first terraform PR!

[brycefisher]

Whoa!!! So awesome.

[alexander-bobin]

Well done guys!

[nicolai86]

Now that #2143 is fixed it's time to revisit the parameter handling and probably also remove the X_in_json hack...

[radeksimko]

@nicolai86 Agreed. PRs welcomed.
Related: #7654 (also merged 😺 )

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.