CORS headers duplicated when connecting to standby server in HA mode

[nathanhoel]

Environment:

• Vault Version: Vault v0.8.1 ('8d76a41854608c547a233f2e6292ae5355154695')
• Operating System/Architecture: Alpine (Vault Docker image tag: 0.8.1)

Vault Config File:

{
  "listener": [
    {
      "tcp": {
        "address": "0.0.0.0:8200",
        "cluster_address": "0.0.0.0:8201",
        "tls_cert_file": "/vault/certs/vault.crt",
        "tls_key_file": "/vault/certs/vault.key"
      }
    },
    {
      "tcp": {
        "address": "0.0.0.0:8280",
        "tls_disable": 1
      }
    }
  ],
  "telemetry": {
    "statsd_address": "datadog.domain.vpc:8125"
  },
  "default_lease_ttl": "168h",
  "max_lease_ttl": "720h"
}

Startup Log Output:

==> Vault server configuration:

                     Cgo: disabled
         Cluster Address: https://staging-1a.vault.domain.com:8080
              Listener 1: tcp (addr: "0.0.0.0:8200", cluster address: "0.0.0.0:8201", tls: "enabled")
              Listener 2: tcp (addr: "0.0.0.0:8280", cluster address: "0.0.0.0:8281", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: true
        Redirect Address: https://staging-1a.vault.domain.com:443
                 Storage: dynamodb (HA available)
                 Version: Vault v0.8.1
             Version Sha: 8d76a41854608c547a233f2e6292ae5355154695

==> Vault server started! Log data will stream in below:

Expected Behavior:
Making a vault API auth request to /v1/auth/aws/login within a browser using CORS with allowed origins set to * and the appropriate content types configured would allow the browser to authenticate.

Actual Behavior:
The OPTIONS request is successful but the POST request to /v1/auth/aws/login returns duplicate headers from vault.
These 3 headers are duplicated access-control-allow-origin, cache-control, vary.
The duplicate access-control-allow-origin causes the browser to ignore the success request and give this error.

Failed to load https://staging.vault.domain.com/v1/auth/aws/login: The 'Access-Control-Allow-Origin' header contains multiple values 'https://vault-ui-staging.domain.com, https://vault-ui-staging.domain.com', but only one is allowed. Origin 'https://vault-ui-staging.domain.com' is therefore not allowed access.

Steps to Reproduce:
I believe this occurs because of the High Availability and request forwarding.
If you connect to a stand-by vault server while using CORS request headers this error occurs.
Whenever someone connects to the active server while using CORS the duplicate headers are not present.

Important Factoids:
You may notice that we connect to staging.vault.domain.com not staging-1a.vault.domain.com. We use a CNAME DNS entry to round robin between the nodes. We are changing our DNS strategy in the mean time to avoid the stand-by server.

Opinion
Forgive me as I am quite unfamiliar with GO and GO libs - but I believe that this line
https://github.com/hashicorp/vault/blob/master/http/handler.go#L87
Where it wraps the response in CORS headers may be the issue when the request was forwarded. I couldn't determine through code inspection alone but it appears as if the headers from the forwarded request would still contain the CORS headers and then the forwarding host will wrap the response with CORS headers again.

If I am correct my opinion is that the CORS headers from the remote host should be used and the forwarding host should not attempt to wrap the response on CORS headers.

[jefferai]

Any chance you can build from the issue-3485 branch and see if it's fixed for you? It seems to be fine in my testing but I'm not coming from a browser.

[nathanhoel]

Wonderful - I will be able to test in our staging environment tomorrow to tell you for sure.
I feel fairly confident about it.

[jefferai]

Note that I made a minor force push to the branch -- if you have checked it out before right now, please make sure to update.

[nathanhoel]

It appears the branch fixes our issue, awesome!