[S3] presigned PutObject - Enforce Content-Type

[rayrutjes]

This is a follow up of #467 .

Actually, it looks like that the Content-Type can not be constrained by the presigned url.
Here is a quick (untested) example that illustrates the purpose:

package main

import (
    "bytes"
    "crypto/md5"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    buf := bytes.NewReader(make([]byte, 10*1024*1024))
    h := md5.New()
    io.Copy(h, buf)

    //
    // Generate presigned PutObject URL
    svc := s3.New(session.New())
    r, _ := svc.PutObjectRequest(&s3.PutObjectInput{
        Bucket: aws.String(os.Args[1]),
        Key:    aws.String(os.Args[2]),
        ContentType: aws.String("image/png"),
    })
    r.HTTPRequest.Header.Set("Content-MD5", fmt.Sprintf("%x", h.Sum(nil)))
    url, err := r.Presign(15 * time.Minute)
    if err != nil {
        fmt.Println("error presigning request", err)
        return
    }
    fmt.Println("URL", url)
    buf.Seek(0, 0)

    //
    // Use the presigned URL to put a object to S3
    req, err := http.NewRequest("PUT", url, buf)
    req.Header.Set("Content-Type", "image/gif")
    if err != nil {
        fmt.Println("error creating request", url)
        return
    }

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        fmt.Println("failed making request")
        return
    }
    defer resp.Body.Close()

    //
    // Print out the response.
    fmt.Println("Status", resp.StatusCode, resp.StatusCode)
    o := &bytes.Buffer{}
    io.Copy(o, resp.Body)
    fmt.Println(o.String())
}

This results in the file being correctly uploaded but the Content-Type would be set to image/gif.
For consistency matters, I would like to ensure that the declared Content-Type while fetching the presigned url is the same as the final real Content-Type attached to the file on S3. In that way I can ensure that any clients asking my API for presigned urls do not forget to correctly add the correct Content-Type header. Hope that makes sense.

Thank you.

[jasdel]

This issue was incorrectly closed.

[jasdel]

@rayrutjes Thanks for creating this issue. I think it points out a generally issue with the current way the SDK generates pre-signed URLs. Content-Type is actually being explicitly ignored when calculating the signature even though the sig v4 spec states it as a valid optional parameter to include in the signature.

I think part of the issue is the SDK's normal request signing verse rules for pre-signing are getting conflated. The ignoredHeaders in v4.go are used to excluded headers from both the query string for pre-signing, and normal signature for SDK requests.

Marking this as a bug. We should take a look at this and see how changing this method will impact backwards compatibility. Or potentially the correct solutions is the idea mentioned in #458 to not hoist any headers but auth related, and removing/trimming the blacklist down.

[jasdel]

I think to start here we need to clarify and document which headers can and cannot be hoisted to the URL path for presigning. In addition a pass over which headers are blacklisted from (pre)signing will be helpful. Specifically we can blacklist headers that can by modified by proxies.

In this we should also include a list of headers which "could" be hoisted to the query string but probably shouldn't specifically these are some of the S3 headers around policy and encryption. Bucket policy filters do not inspect query string parameters, and only verify against headers.

[xibz]

Fixed in v1.1.0. Let us know if any other issues or feedback regarding the change.